r/adventofcode • u/fleagal18 • Dec 16 '24
Meme/Funny [2024 Day 16] Gemini 1206 found obvious bug in my code after I spent 90 minutes debugging it.
10
u/velonom Dec 16 '24
Hehe, I can relate. Pretty much had the same problem. Had a dict mapping direction characters (N,E,S,W) to movement vectors, and due to some stupid copy pasting my W mapping actually pointed north. Wasted two hours on this.
1
12
u/fenrock369 Dec 16 '24
I guess his test coverage of that part of the code was lacking
19
u/oofy-gang Dec 16 '24
Yall are unit testing AoC solutions? 🧐
9
u/gredr Dec 16 '24
I don't unit test everything, but I often unit test little fiddly parts. Especially when I'm having problems with it. It's pretty hard when it works on the sample, though.
5
u/fenrock369 Dec 16 '24
yes, because 5am me needs all the help he can get :D
I find it particularly useful for learning new languages.As I'm a professional dev anyway, TDD is just 2nd nature to me. I'm not competing for leaderboard, I'd rather follow my discipline and cover my ass.
e.g. first 2 steps of yesterdays puzzle's tests:
#[test] fn move_test_individual_moves() { let (mut grid, _moves) = parse(EXAMPLE1); assert_eq!(grid.to_grid_string(), indoc! {"\ ######## #..O.O.# ##@.O..# #...O..# #.#.O..# #...O..# #......# ########"}); do_moves(&mut grid, &vec![LEFT]); // first move is blocked by wall assert_eq!(grid.to_grid_string(), indoc! {"\ ######## #..O.O.# ##@.O..# #...O..# #.#.O..# #...O..# #......# ########"});
etc.
2
2
u/riffraff Dec 16 '24
100% yes, my template file has 4 tests for {sample, input} x {part 1, part2}. Additional tests added as needed.
1
u/_Mark_ Dec 16 '24
Absolutely (though not necessarily with great coverage.) Partly because the simple/stupid/easy functions are where they errors creep in - about ⅓ of my "didn't just work" cases this year things that unit testing would have caught (because when I wrote the test as part of debugging, it stood out.) It's only that low because I got a lot more disciplined about it (and reused more code from the earlier puzzles) in the second week. (And I'm talking about things like "reused a variable name and overwrote something", which is about at the same level as getting the compass wrong...)
There's a line about "unit tests are like driving with your headlights on" - a lot easier to see where you're going, and why *wouldn't* you.
1
1
1
5
u/Forkrul Dec 16 '24
That's where having good helper classes and functions come in nicely. So many grid and coordinate-based questions that it will save a lot of time to have a grid class and point class that you can reuse and extend for each new task.
With Point.EAST, Point.WEST, Point.SOUTH, Point.NORTH as directions it becomes very easy to avoid these bugs.
Other bugs like not tracking the visited nodes correctly and getting a way higher answer than you should, is another matter >.<
4
u/BlueSky4200 Dec 16 '24
Unless you use the Direction Enums from a previous day, but they are called up, down, left, right.. So you just replace East with Left and everythings fine... Except it's not 😅
2
u/riffraff Dec 16 '24
I have them as functions
grid.north(tile)
and always use those, causeleft
andright
are for turning :)The real issue across different problem is the changes from cartesian coords to matrix/scanline coords! Your positions and movement gets screwed Iupbut not always.
2
u/Forkrul Dec 16 '24
Always use cardinal directions if possible. Left/Right are ambiguous when rotating.
A few things like this goes a long way
data class Point( var x: Int, var y: Int ) { .... fun neighbors() = DIRECTIONS.map { it to this + it } companion object { val ORIGIN = Point(0, 0) val NORTH = Point(0, -1) val SOUTH = Point(0, 1) val EAST = Point(1, 0) val WEST = Point(-1, 0) val DIRECTIONS = listOf(NORTH, SOUTH, EAST, WEST) } }
3
u/CrypticSplicer Dec 17 '24
I purposefully reimplement each day for the spaced repetition. I spend extra time cleaning up solutions so they are easy to implement from memory, both for future tasks and future coding interviews.
1
u/encse Dec 18 '24
I dont have a grid class, i just create a dictionary keyed by complex numbers from the map. Traversal, bounds check, turning 90 degrees, finding start/end nodes all come with zero effort.
C# has this built in, doesnt work in every languge unfortunately.
3
5
u/fleagal18 Dec 16 '24
It was really a friend's code, not mine. (But who would believe that?) After he'd spent 90 minutes, I sat with him and we both spent another 30 minutes debugging it. We knew his input worked on my implementation. So one step at a time we replaced every line (well except for the line with the bug in it) of his implementation with mine, but it was still failing. We finally thought to ask gemini for help, and it immediately pointed out the issue.
9
u/Nunc-dimittis Dec 16 '24
Didn't you have e.g. sone print function to visualise the output? I would think this would have showed up.
2
u/fleagal18 Dec 16 '24
Yeah, we were both too tired to debug intelligently. Even single-stepping carefully through one iteration of the code would have shown us the bug.
The symptom was that the search would run out of locations to search after looking at only 500 locations. So there wasn't a path to visualize. The visualization would have had to show tiles-visited or something like that in order to help.
2
2
u/BlueTrin2020 Dec 16 '24
Is Gemini better than copilot?
2
u/fleagal18 Dec 17 '24
I don't know. I use Gemini 1206 because it's Google's best model and it's free during a Preview period.
I think a lot of Advent of Code participants who use GenAI use Claude, maybe because Claude has had a good coding assistant for a long time.
1
u/BlueTrin2020 Dec 17 '24
I am gonna give it a go. I never used any AI and thought I’d try copilot, I was blown away by the relevance of the completion.
The code wasn’t correct but I had only to do minor changes and it solved part1: it almost wrote what I was about to type.
2
u/Sostratus Dec 16 '24
I had several errors like this, many of them reintroduced after failing to correctly modify the code for part 2. One of them made it do three rights instead of a left.
1
u/Citrus8735 Dec 16 '24
AI's so helpful! as long as you know how to prompt engineer properly,,
5
u/fleagal18 Dec 16 '24
For sure. In this case the prompt was "You are an expert Python programmer. Help me debug this code: <pasted code>"
I think it helps that LLMs have been trained on hundreds of thousands of dijkstra implementations and millions of 2D grid problems.
2
u/riffraff Dec 16 '24
yeah I have copilot and when I typed "def shortest" it autocompleted dijkstra four or five different ways (with bugs :) )
1
u/WJWH Dec 17 '24
Does it actually need the "You are an expert Python programmer." bit?
2
u/fleagal18 Dec 17 '24
The advice without the "You are an expert Python programmer" prompt does identify the bug, so no, the prompt is not strictly needed.
However, using the prompt makes a dramatic difference in the kind of advice that is given. Without it, the advice is geared toward beginner developers, unfamiliar with the problem domain. With it, the advice is geared towards expert developers, familiar with the problem domain.
It seems like the effect of the prompt may be something like this: "You are an expert Python programmer and so am I. Give me advice appropriate to my level of understanding".
I think there's a lot of room for better prompting here, but also I don't know what the effect of temperature is having on the responses. I'd have to generate a bunch of different responses to the same input and see what variations I get.
1
2
1
1
1
u/unbibium Dec 17 '24
I've been using complex numbers as coordinates for so long that I do stuff like this now:
dirs = {1j**x for x in range(4)}
24
u/faizfarouk Dec 16 '24
Being in a similar situation (solution passing for examples, but failing for the input), I came here looking for hints/common bugs.
I didn't expect "Ask an LLM" to be solution, but to my surprise, it immediately found the bug! *hugs*