Google Deepmind and OpenAI claim gold medal/1st place performance in top programming competition - human codemonkeys BTFO?

Destacado
grump

grump

kiwifarms.net
Registrado
14 de Abr, 2016
Both Google Deepmind and OpenAI have claimed gold medal performance in ICPC algorithmic programming competition. In the ICPC competition a 3 student team works on a set of problems for 5 hours.

Google claims to have solved 10/12 problems beating all but three human teams.

OpenAI claims 12/12 problems which would place it as the winner.

Google announcement

OpenAI announcement

Human results

Ars article on Google

Questions

Deepmind's answers
 
grump dijo:
Both Google Deepmind and OpenAI have claimed gold medal performance in ICPC algorithmic programming competition. In the ICPC competition a 3 student team works on a set of problems for 5 hours.

Google claims to have solved 10/12 problems beating all but three human teams.
OpenAI claims 12/12 problems which would place it as the winner.
Presiona para expandir...
SAAR DO NOT REDEEM THE AI SAAR.
SAAR PEOPLE BETTER CODERS THAN AI SAAR.
Good morning saar.
 
>GPT has become a coding tool of tool-coders and other IT nerds
So yet another Omni Tool meant to better humanity reduced to the one niche where it generates profits.
 
This isn't that exciting to me because it's one of the things that something like Gemini, ChatGPT, Claude, etc should excel at when they have presumably been trained on every major programming competition's questions and various solutions spanning decades. If those students could use the internet or even pull up all the previous ICPC questions and solutions then I'm sure many or all would have been able to solve more than they did.
 
Any "speed programmer" will tell you all these leetcode-style problems just boil down into several categories. LLMs are good at small data transformation without much state change. In more real world use I'm constantly trying get LLMs to spit out code without it hallucinating high level API functions that don't actually exist. Often mixing and matching different versions of APIs, it's a mess.

What's the point of leetcode? We're at that stage where programmers can say "I'd get the LLM to do it", similar to kids saying they'd use a calculator to solve math problems. When an employer tests you for leetcode, they're making you prove you can do stuff that is increasingly irrelevant. They may even have mandatory requirements for minimum LLM usage, and if they catch you writing basic data transformation functions by hand you'll get the "you're wasting time" look.
 
CrunkLord420 dijo:
LLMs are good at small data transformation without much state change. In more real world use I'm constantly trying get LLMs to spit out code without it hallucinating high level API functions that don't actually exist. Often mixing and matching different versions of APIs, it's a mess.
Presiona para expandir...
To coin a phrase, "you need to build a fence around a field before letting the animals graze on it". I'd imagine that the majority of AI "development" is going to be based around telling the LLM what not to do using increasingly precise models; in other words, we grow smarter by realizing how stupid our computers can be if we let them.
 
Destacado
Debes iniciar sesión o registrarte para responder aquí.
Atrás
Top Abajo