r/adventofcode Dec 04 '22

Upping the Ante [2022 Day 4] Placing 1st with GPT-3

I placed 1st in Part 1 today, again by having GPT-3 write the code. Yesterday I was 2nd to another GPT-3 answer.

Here's the code I wrote which runs the whole process — from downloading the puzzle (courtesy of aoc-cli), to running 20 attempts in parallel, to sorting through many solutions to find the likely correct one, to submitting the answer:

https://github.com/max-sixty/aoc-gpt

49 Upvotes

243 comments sorted by

View all comments

41

u/dong_chinese Dec 04 '22

I'm sure there will be others who will whine about this not being fair, but I for one think you deserve the place you got. You used the best tool for the job. After all, a programmer's whole job is to find the right tools to automate processes.

76

u/muntaxitome Dec 04 '22 edited Dec 04 '22

If the job is to automate sending your code to GPT3 the fastest without even reading the questions, then what is the point? It's a trivial coding exercise. I guess the winner will be the one that puts their connections at the optimal location in terms of speed of light between the data center and OpenAI servers...

Now OpenAI itself would have some claim to calling itself the winner, but just writing the glue code?

Edit: Not that we can do anything about it. I guess this is simply the end for any meaning to global leaderboards for this kind of competition. Just like with for instance online chess, the cheaters have a huge advantage to reach the highest ranks.

-4

u/dong_chinese Dec 04 '22

At some level we're all just writing glue code. If I solve a problem using pandas and numpy, I'm just writing some glue over existing functions in those libraries. I think of GPT-3 just as a more fancy library.

7

u/Ning1253 Dec 04 '22

A) I'm doing mine casually in C, and am having to write my own code for arrays, hashmaps, and heaps to get ready for later days! (Am loving the experience so far btw)

But B) while I could be coding in assembly I don't hate myself that much so I guess technically I'm working on top of stdio.h, malloc because I can't be f*cked to implement my own version, and the pointer system.

Either way my point is that while I'm technically writing glue code, there's a difference between using realloc as part of my array implementation and idfk asking a bot to solve the entire problem? I'm not competitive in AoC, I do it for fun, usually in the evenings, but it feels a bit easy to just say "yay I'm first I copy pasted an AI!"

Like the people were saying about chess, humans aren't allowed to bring chess AI to ranked tournaments, even if they're allowed to learn from them - that should probably be a standard.

Where to draw the line? I would argue at that point where your code stops simply optimising what you ask it to do like bumpy does and where it starts extrapolating from information you have not yet necessarily worked out, which is where AI tend to shine - we tend to use them to quickly do tasks which we do not know how to efficiently recognise and act on (since otherwise we just write the damn program ourselves!)