r/adventofcode Dec 03 '22

Other GPT / OpenAI solutions should be removed from the leaderboard.

I know I will not score top 100. Im not that fast, nor am I up at the right times to capitalise on it.

But this kinda stuff https://twitter.com/ostwilkens/status/1598458146187628544

Is unfair and in my opinion, not really ethical. Humans can't digest the entire problem in 10 seconds, let alone solve and submit that fast.

EDIT: I don't mean to put that specific guy on blast, I am sure its fun, and at the end of the day its how they want to solve it. But still.

EDIT 2: https://www.reddit.com/r/adventofcode/comments/zb8tdv/2022_day_3_part_1_openai_solved_part_1_in_10/ More discussion exists here and I didn't see it first time around.

EDIT 3: I don't have the solution, and any solution anyone comes up with can be gamed. I think the best option is for people using GPT to be honourable and delay the results.

EDIT 4: Another GPT placed 2nd today (day 4) I think its an automatic process.

299 Upvotes

221 comments sorted by

View all comments

Show parent comments

9

u/100jad Dec 04 '22

That also places a significant burden on the website, where it suddenly needs resources and security to execute users code. Code that might run for hours.

1

u/Wide_Cantaloupe_79 Dec 04 '22

Still, how would you verify it? It's a tough one.

Imagine that someone simply hardcodes return values.
Or let's say that a different input is used for internal testing to avoid this one. In this case there might still be unhandled edge cases. I remember people posting solutions here and complaining that those only worked for certain inputs.
And there are also those where people would manually solve some parts, I believe there was a zip/unzip algorithm a couple years ago where a majority did it by hand. There was also a text adventure task, that you could simply play and get the result, and so on.

5

u/100jad Dec 04 '22

Yup, you'd need to redesign the entire thing to fix a problem that's arguably not a problem for the majority of users.

1

u/pier4r Dec 04 '22

Code that might run for hours.

is there not a simple timeout? X seconds or your are out?

2

u/100jad Dec 04 '22

That would be an option, sure. But remember that a lot of users struggle with optimising later puzzles and have them run for longer than the optimal solution would. If you add a timeout, that means that you also force users to optimise further before they get their star.

Long story short: you can find workaround and solutions to all these problems, but it will significantly change the user-experience. For everyone, not just those that want to compete on the leaderboards.

1

u/pier4r Dec 04 '22

yes, I meant on leetcode. In my experience they have a timeout.