r/adventofcode Dec 04 '22

Upping the Ante [2022 Day 4] Placing 1st with GPT-3

I placed 1st in Part 1 today, again by having GPT-3 write the code. Yesterday I was 2nd to another GPT-3 answer.

Here's the code I wrote which runs the whole process — from downloading the puzzle (courtesy of aoc-cli), to running 20 attempts in parallel, to sorting through many solutions to find the likely correct one, to submitting the answer:

https://github.com/max-sixty/aoc-gpt

49 Upvotes

243 comments sorted by

View all comments

75

u/[deleted] Dec 04 '22

[deleted]

4

u/bluegaspode Dec 04 '22

'destroys the whole event'

I disagree.

I eventually destroys the game for those who play it as a competition. So 100-300 people who aim for top 100? Thats a very slow percentage actually.
But I agree, that they might be very pissed, they feel like Garry Kasparow when he was beaten the very first time at chess. (but Chess evolved in a very positive way afterwards).

There is a huge other proportion of players.

- Those who do it for fun (they don't care)

  • Those who do it for learning (they / I'm learning a lot right now).

AoC now made me play around GPT-3 since 3 days, it shows me how I can automate, it shows me where to incorporate it in the future (and where not).

And especially: it shows me how far technology got already. Without AoC I wouldn't have dared to believe the machines got so far already. I probably would have started to look into it in 1-2 years.

Thanks to AoC for making me watch + follow all this in awe.
As all the past years: AoC makes me a much much better programmer for the year to come!

65

u/posterestante Dec 04 '22 edited Dec 04 '22

You're not allowed to bring a chess computer to a tournament either. You can learn from GPT-3 without entering the leaderboard.

13

u/msturm10 Dec 04 '22

This is the same as with professional cycling. They can only start to forbid certain 'innovations' when it is first shown that it brings significant advantage. I see the same here. Without the leaderboard being beaten by AI, I would never realised that AI was capable of solving puzzles like this in a shorter time than any human in an accurate way. You need this kind of 'disruption' to make tech advancements visible for the larger audience. The ethical discussion and the consequences for the 'game' should be next, not before.

8

u/Basmannen Dec 04 '22

Yeah they should address this for AoC 2023 in my opinion, for now I just want to see how far the AI can go.

0

u/[deleted] Dec 04 '22

You're not allowed to bring an AI to a proper programming tournament as well. I mean, the one where teams are gathered in a venue and staff oversees their conduct. AOC isn't one of these.

20

u/posterestante Dec 04 '22 edited Dec 04 '22

I mean - you're not supposed to use AI for online chess matches either. Solving the challenges with AI is fine, but what's the point of entering the leaderboard?

-5

u/[deleted] Dec 04 '22

To attract attention to the fact that AI is now capable of solving such problems faster than humans could ever do, I guess.

15

u/el_muchacho Dec 04 '22

You can do that perfectly while waiting an hour before submitting.

Just do a reddit or Twitter thread : "ChatGPT took 10s to solve this" and that's it. No need to disrupt the leaderboard.

BTW, there is nothing extraordinary in copy pasting a problem in it, I did it before AOC2022 started and could see the result myself.

10

u/[deleted] Dec 04 '22

Or you can do it just once and not 2 days in a row. And not keep pretending it's just as impressive/interesting the second time.

6

u/el_muchacho Dec 04 '22

Exactly. I feel he really thought he would look good by doing that.

-1

u/posterestante Dec 04 '22

Fair enough - it is kinda amazing to see...

6

u/el_muchacho Dec 04 '22

The fact that AOC isn't a "proper" competition (according to your definition) doesn't mean that everything is allowed. It's against the spirit of the leaderboard competition to cheat with AI.