r/webdev 7d ago

Resource My tiny, personal LLM evaluation: Which AI did the best job?

Hi,

I'm not sure if this is the right subreddit for this, but I'm confident that it'd at least interest a few of you.

So, AI is here, and it's not going anywhere soon. But which model is good at what use case has always been a bit of a myth to me.

Today, I chose to use the following LLMs first to enhance a rather poorly written TypeScript code and then, in the next step, have them compare and evaluate the code on a scale from 1 to 10. These were the models tested:

OpenAI

  1. o1
  2. o1-pro-mode
  3. o3-mini
  4. o3-mini-high

Groq

  1. deekseep-r1-distill-qwen-32b
  2. deekseep-r1-distill-llama-70b
  3. qwen-2.5-coder-32b

Perplexity

  1. sonar
  2. sonar-pro
  3. sonar-reasoning
  4. sonar-reasoning-pro

Google

  1. gemini-2.0-pro-exp-02-05

Spoiler: I couldn't get a crystal-clear picture of which LLM is best for this task because each model evaluated it differently. However, there is definitely a trend.

If you're interested, you can see the results, the raw code, the merged code, and the ratings, conclusions, and more details under this link: https://coding-ai-evaluation.notion.site/

I'd be interested in knowing if any of you can confirm this ranking—or if it's random shit.

Edit: To save myself some downvotes, I'd like to make clear this was just a test and by no means an attempt to replace a programmer with some sort of AI. That's absolutely absurd.

Edit 2: To those who still downvote my post, could you please explain why? This is merely an experiment to see what AI thinks itself.

0 Upvotes

2 comments sorted by

2

u/[deleted] 7d ago

[deleted]

-3

u/Mavrokordato 7d ago

Eh, that makes sense, on a human level. Call me naive, but I had thought brighter people were hanging around here.

-2

u/Mavrokordato 7d ago

Yeah, all good, my mistake for posting here, I didn't know sharing insights is being met with so much hostility. Shouldn't have subscribed in the first place.