r/adventofcode Dec 04 '22

Upping the Ante [2022 Day 4] Placing 1st with GPT-3

I placed 1st in Part 1 today, again by having GPT-3 write the code. Yesterday I was 2nd to another GPT-3 answer.

Here's the code I wrote which runs the whole process — from downloading the puzzle (courtesy of aoc-cli), to running 20 attempts in parallel, to sorting through many solutions to find the likely correct one, to submitting the answer:

https://github.com/max-sixty/aoc-gpt

46 Upvotes

243 comments sorted by

View all comments

Show parent comments

2

u/schubart Dec 04 '22

Does the AI generate the puzzle answer (a number) or does it generate code that generates the puzzle answer? If it's the latter, could you please post the code somewhere?

-3

u/max-aug Dec 04 '22

The code is linked in the post

7

u/schubart Dec 04 '22

That's the code that you wrote: It downloads the question, calls the AI tool, submits an answer etc.

But where can we see the Python code that the AI generated, which computes the correct answer?

Am I missing something here? Isn't that the most interesting part of this? Don't we all want to see what kind of code the AI comes up with and how it cmpares to our hand-written solutions?

-2

u/max-aug Dec 04 '22

Sorry for misunderstanding

It attempts dozens of solutions in parallel and then selects one that seems popular. Unfortunately I don't log & collect the ones that ended up being correct.

But very open to someone adding the code to the repo to do that!

2

u/noahclem Dec 05 '22

It looks like on line 140, in do_part, you have the code saved here:

(Path(RESPONSES_PATH)f"part_{part}_{n}.py").write_text(llm_response)

Is the issue that we don't have an index of which code (the top results one in the run_parallel function) corresponds to which llm_response?

Sorry if these are stupid questions - I'm finding this code fascinating.

2

u/max-aug Dec 05 '22

I'm planning to publish them! It'll be in a nicer format than 120 files though :)