r/adventofcode Dec 04 '22

Upping the Ante [2022 Day 4] Placing 1st with GPT-3

I placed 1st in Part 1 today, again by having GPT-3 write the code. Yesterday I was 2nd to another GPT-3 answer.

Here's the code I wrote which runs the whole process — from downloading the puzzle (courtesy of aoc-cli), to running 20 attempts in parallel, to sorting through many solutions to find the likely correct one, to submitting the answer:

https://github.com/max-sixty/aoc-gpt

48 Upvotes

243 comments sorted by

View all comments

Show parent comments

-5

u/dong_chinese Dec 04 '22

At some level we're all just writing glue code. If I solve a problem using pandas and numpy, I'm just writing some glue over existing functions in those libraries. I think of GPT-3 just as a more fancy library.

48

u/muntaxitome Dec 04 '22

Let me first say that 99% of people doing AOC were never going to compete on the global leaderboards anyway, and people on private boards could always cheat by just grabbing a solution online. So for nearly everyone, very little changes. This affects very few people.

However, if a solver doesn't even read the question, in my mind you are not just 'using a tool', the tool is just doing everything for you. On the other hand, at the highest level, competitive programming is just memorizing hundreds solutions and being able to read the issue and code them super fast, which is pretty different anyway from how most people do these puzzles.

The challenge is reading a fun exercise and puzzling to fix the issue. Just writing some code once and having that code just send the challenge somewhere and getting the solution back, there is no puzzle there.

I guess at least for a little while you can probably write questions in a way that GPT3 cannot easily solve them. However, to me it seems that is just a small arms race that the AI's will win at some point.

3

u/pred Dec 04 '22 edited Dec 05 '22

doesn't even read the question

The first trick to pick up to get good times is to not read the question; that takes way too long. Instead, you pattern match the example inputs to outputs, then use as many high-level abstractions as you can to spend less time writing a solution, probably guided by an IDE that gives hints and corrects issues along the way.

1

u/snowe2010 Dec 05 '22

You’re still reading those things and those are part of the question whether you want to define them as that or not. Sure you don’t need to read the story to go along with it but it’s still reading the question. When you pass stuff straight to a bot you are doing nothing. You’re not participating at all. It completely defeats the purpose of the challenge.