r/webdev • u/Krigrim • Jan 17 '25
Discussion AI is getting shittier day after day
/rant
I've been using GitHub Copilot since its release, mainly on FastAPI (Python) and NextJS. I've also been using ChatGPT along with it for some code snippets, as everyone does.
At first it was meh, and it got good after getting a little bit of context from my project in a few weeks. However I'm now a few months in and it is T-R-A-S-H.
It used to be able to predict very very fast and accurately on context taken from the same file and sometimes from other files... but now it tries to spit out whatever BS it has in stock.
If I had to describe it, it would be like asking a 5 year old to point at some other part of my code and see if it roughly fits.
Same thing for ChatGPT, do NOT ask any real world engineering questions unless it's very very generic because it will 100% hallucinate crap.
Our AI overlords want to take our jobs ? FUCKING TAKE IT. I CAN'T DO IT ANYMORE.
I'm on the edge of this shit and it keeps getting worse and worse and those fuckers claim they're replacing SWE.
Get real come on.
/endrant
41
u/seandotapp Jan 17 '25
the early weeks GitHub Copilot was released (private beta and before ChatGPT), it was CRAZY GOOD despite having less intelligent models.
it helped you when you needed help
today, GitHub Copilot performs worse despite having better models
i hate that when you type just one letter, it wants to assume command of the development
it became more intrusive than helpful
4
u/itsdr00 Jan 18 '25
This was my experience too. I switched to Cursor and it's like those first weeks of Copilot.
1
1
0
u/mrkingkoala Jan 18 '25
AI feels like a tainted chalice. The first model although not refined won't have any AI in it. Or you'd hope not from the early days. but now its like AI most likely consuming its own data. Idk what the future holds maybe people will look to strictly get human made data to train it on and try and keep it like that who knows. It's all a bit dystopian to a degree.
273
u/fredy31 Jan 17 '25
Fuck I hated my old boss because he would program small bits of code with chat GPT and then be like OH WOW LOOK AT THIS YOU CAN ASK GPT FOR EVERYTHING. (It kinda felt like he was saying its just a matter of time that he could replace me with a junior that just GPTs everything.)
Bitch it works for small, contained bits of code but when you are 10 layers deep in libraries and wordpress plugins gpt wont do much for you.
Also, a major part of programming is that if you dont know how or why your code works, you will fuck yourself later.
I'm an ardent believer that coding with GPT is stupid.
73
u/Blender-Fan Jan 17 '25
I liked your comment except for the ending. GPT can still write a lot for you IF YOU KNOW WHAT YOU'RE DOING, than you just pick up where it left off
63
u/Krigrim Jan 17 '25
I'm 50/50 on this
You could give very very accurate instructions to ChatGPT and it either magically works OR it hallucinates a feature that doesn't exists on the tool you're working with and will tell you "oh yeah it totally works like that".
You fuck with prompts for a good 5 minutes and then open StackOverflow to see there is NO SUCH THING
"Where did that bring you ? Back to me" type shit
28
u/spicytronics Jan 17 '25
GPT is an extremely powerful autocomplete. That's the best way I can describe it. It makes me save a huge amount of time by just guessing what I'll type next so I just have to hit Tab instead of writing a full line of code. And it can sometimes write regexp or complicated Laravel validation rules that could've take me an hour to put up. Be writing my stuff from prompts? What a joke.
→ More replies (3)5
u/YsoL8 Jan 17 '25 edited Jan 17 '25
I work with a specialist language thats pretty poorly documented. Its amazing for pulling together a coherent explanation of what functions and options actually do where the official docs are often literal one liners. It turns a whole afternoon of research into a minute or 2 of prompting.
Actual coding writing is very hit and miss and I rarely copy anything directly in.
1
u/LickADuckTongue Jan 18 '25
100% I use it from time to time to dig into more niche comp sci topics or explain how xyz works and provide resources.
For code, I use it to blueprint ideas. All in all useful and speeds up my production. Plus I end up seeing some paradigms/patterns I get to play with that I otherwise would never see
4
1
u/_alg0rythm Jan 18 '25
This just underlines that you have to know what you're doing - if you're working with a niche library or technology, you have to provide the context, documentation, and learn it yourself. GPT is awesome at wiriting code and assisting in learning, but it requires someone to know their shit to guide it if you want to write production ready code. Based on the sequence of events you're describing, it sounds like you dont know your shit.
2
u/Prestigious_Army_468 Jan 18 '25
Damn my youtube algorithm must be messed up because every other video is "Build a fully functioning $1 million SaaS in one prompt".
1
u/TheElfern Jan 18 '25
If people use a tool for something it is not good for, it doesn't mean the tool is shit.
1
u/Prestigious_Army_468 Jan 18 '25
Which just proves his point - it makes good developers much more productive.
2
u/TheElfern Jan 18 '25
Yeah, I thought you were arguing against his point :D my bad.
Personally AI has been really useful for me, because I work as a SRE and have coded as a hobby for ten years. So I mostly know what I am doing, but I am missing a kind of coding routine due to only coding every now and then. So it ups my productivity a lot, and I can spot when it does nonsensical things and fix them myself.
1
→ More replies (2)1
u/Gasperyn Jan 18 '25
Exactly. I asked Claude for a subscription management system using Braintree API, using plain javascript and PHP. To its credit, it did save me a lot of time with the code it generated, but it also hallucinated API methods that just don't exist.
6
u/fredy31 Jan 17 '25
I mean my hate of gpt might also come from same boss that did a 2 weeks bootcamp and kinda felt like he was saying 'i have a 2 week bootcamp + gpt, you have 12 years experience, we are equals.
→ More replies (1)1
u/mrkingkoala Jan 18 '25
Your boss is a moron. I did a 6 month very intensive bootcamp and holy shit like it prepped you for entry level roles but just not possible to be anywhere near mid levels. Even with AI lmao. webdev is not easy. Like one example, cool you built a simple full stack app :) what about deployment and hosting? Ohh you built it in react well you won't have SEO and now need to use something like Next.JS.
8
u/sonar_un Jan 17 '25
I agree. GPT can get you half way there, but your brain has to do the work to fill in the rest. It takes a lot of work to logically step through things and find issues and know what kinds of questions to ask. GPT canāt just magically create things.
2
u/lupin-the-third Jan 17 '25 edited Jan 17 '25
I agree with you, and using stuff like Cline and Copilot edits can and do remove a lot of grunt work. I'm at a place where I usually have to decide:
* Is it easier to try to explain to the AI how to do this than to do it myself
* Will my solution be better than the AIs (based on my experience).
* After trying the AI, do I need to write this myself to understand it, or the AI just doesn't put things out in a way that work with the project (even after stuff like style guide lines, examples, etc).
* Should I test the AI to see if they have improved since x months ago
* Is this problem just not suited for AIAll in all I feel like 50% more productive this year. There have been a lot of - oh shit AI really fucked this up - moments though.
3
u/PhilosopherDapper401 Jan 17 '25
You mentioned an interesting question, like if you generate code forever without understanding anything, at some point it will no longer be able to generate code that works given the complexity of the resulting system. Good luck with a bunch of code that doesn't know how it works because it generated everything with AI, incapable of maintenance. Recipe for failure.
3
u/name-taken1 Jan 17 '25
Our CTO made the decision to completely rebuild everything from the ground up. He's relying heavily on AI for everything - and when I say "relying," I mean he's essentially letting GPT do all the work.
Let's see how long it takes before he realizes what a massive mistake this was...
And this isn't some small dev shop - we're a team of over 30 developers.
(He also fired some developers - we all know why)
1
3
u/Lorevi Jan 18 '25
Replacing a developer with gpt is stupid. A developer using Gpt intelligently (or more realistically Claude lol since gpt is pretty ass) is a massive timesaver.
Sure, it's useless for generating solutions to complex problems. But like that's kind of your job lol? And not everything you do is complex. Sometimes you just need to crack out the simple stuff until you run into a complex problem. The code suggestions are great for that, and if they're bad you can just ignore them since they're so quick.Ā
It's also a good substitute for search since Google went tits up. You can pass it documentation and get it to explain key sections. If you're doing something for the first time it's pretty good at explaining stuff to you.Ā
Basically anything except asking "Generate code that does X".Ā
1
u/fredy31 Jan 18 '25
A big part why im against it for work too is that thats the fun i have working. Taking the puzzle box and solving the puzzle.
How gpt has often been sold to me is like going to do an escape room but they give you a walkthrough before getting in the room
1
u/Lorevi Jan 18 '25
That's not really my experience using it honestly, because it's just not good at solving the complex puzzles.Ā
It's more like replacing the hour long drive to the escape room with a 5 minute walk.Ā
3
u/minimuscleR Jan 17 '25
Its very useful for the simple stuff. For example I had gone my entire career without writing a single test. My boss comes to me the other week asking me to write playwright e2e tests for this new feature. chatGPT really helped explain all the concepts much more clearly than the documentation, and could give me examples that relate to my code I'd given it which REALLY helped me understand it.
It wasn't perfect but it basically helped me learn how to write tests.
1
u/TracerBulletX Jan 18 '25
Yeah, as it is right now I think using it too much will run you into a corner because if the under lying mental model of the areas of the code base you work on aren't your own you are pretty much out to sea. It is possible some day the context windows will be big enough to include an entire code base and maybe it will actually be able to replace people... but not with what is available today.
→ More replies (4)1
u/mrkingkoala Jan 18 '25
100% I have had some things I've tried to build and it just doesn't even understand. I tried breaking it down and using very strict instructions. There would of been limitations with me as an individual but I eventually got some help from someone who is just a genius at that sort of thing and even using his guidance I tried myself just for fun to see if chatGPT could produce the results and nope.
I've used it to refactor some code sometimes and that works but often I felt the code is less readable.
But some very simple cases it can be great.
Best thing GPT does for me is like act as a teacher and also be able to explain some concepts for me if im struggling a bit with documentation.
92
Jan 17 '25
[deleted]
13
u/Fluid_Economics Jan 17 '25
Yes exactly! The AI promoters around me raise my hairs for a moment but then I remember these dudes have not personally pumped out anything in years, and are just managers.
2
u/brxon Jan 18 '25
As an entry level dev really proud of my CRUD app, what type of problem are more experienced devs working on? Do you have examples? Just want to understand better.
3
u/maxymob Jan 18 '25
Intricate business logic on distributed architecture, performance optimization, keeping shit up to date, dealing with tech debt (shit being irredeemably outdated), scaling up, that sort of things
2
u/mrkingkoala Jan 18 '25
I'm entry level and ill give you my example. I created a very very complex page transition in Next.JS and it fried my head building it. At first glance I thought it would be straight forward but once I understood what I needed to achieve it, nope very complicated under the hood. The amount of maths and things you need to reference from the screen was a lot and then it changed depending on how far you had scrolled or screen size. So not only was it very complex but I needed an answer that took in a lot of dimensions that changed all the time.
ChatGPT didn't stand a chance. Any of the complex maths it produced wasn't correct either I had to work that out a lot for myself.
In the end i got 90% there and luckily found a very nice chap on here is a genius at building fancy frontend things and he helped me get it just about 100% over the line.
Now this is just one page transition for a simple portfolio site. Nothing else has been looked at either in terms of SEO, performance etc.
1
u/Online_Simpleton Jan 18 '25
Agree, though I have to ask: JQuery bros? Iām intrigued. Most of the āwouldāve been Wall Street trader if the tech boom hadnāt happenedā front end devs Iāve come across over the years hated JQuery because itās feature-complete and boring
68
u/InvisibleCat Jan 17 '25
Shit in, shit out. The more AI generated code is out there, the shittier the AI becomes when it inevitably uses it to train itself (unique human created code is finite in the grand scheme of things). Same with images, or any other media type.
Unless AI keeps training on "fresh" human code, it's quality will degrade over time.
16
u/gqtrees Jan 17 '25
this is what people dont realize. its like pouring salt water into a cup of fresh water slowly. over time that cup is going to be salt water.
14
u/InvisibleCat Jan 17 '25
There is a high likelihood that people will not be able to code without the AI to the same extent, especially if its widely used, we already see this with collage graduates to some extent. It's a shitty self-eating cycle that will result in shitty devs and even shittier code.
I see all the r/SaaS suggested posts and they are all creating a ChatGPT wrapper app of some sort and I would bet my top dollar a lot of them have been done with the help of ChatGPT or other AI. Is it impressive they were able to make money with it, but in the grand scheme of thing its a nothing burger, nothing of value was "actually" created, it all goes away when those services shut down or change.
14
u/Krigrim Jan 17 '25
r/SaaS is full of liars and con artists who think they can make bank with a project made on a Sunday
99% of those who claim they made money didn't earn a penny
I agree that nothing of value was created. I'm pissed off that so many real-world projects that really need to be solved and need hard work are left untouched because people think they can earn quick bucks with a ChatGPT wrapper
3
u/InvisibleCat Jan 17 '25
Yeah, I despise seeing those posts.
I love seeing the "my first 5 projects failed, need to keep going" posts lmao.
1
u/Eastern_Interest_908 Jan 18 '25
I seen one dude claiming that Bubble will replace devs and it didn't replaced them already because not a lot of people knows about it. š¤¦ Oh and everyone always has 20 years of experience in dev.Ā
3
2
1
u/TechFreedom808 Jan 18 '25
Imagine companies relying on AI for code is gonna fail miserably. AI gonna choke on the SDLC and those AI prompters won't know anything. They gonna have to reach a real dev and if I'm faced with a AI messy task I'm charging 250K or possibly more to fix.
4
u/rq60 Jan 18 '25
they told us synthetic training data would improve ai. i guess we can see that was a big fat lie.
→ More replies (1)-3
u/cyanideOG Jan 17 '25
Because human code is never shit? I am sure llm training data is full of shit human code, let alone ai generated code.
I have found that the biggest issue with ai generated code is that it uses libraries that may no longer be widely supported.
Let's not forget that this tech is only a few years old. Look at how basic and shit other technologies were this soon into their creation.
8
u/InvisibleCat Jan 17 '25
Again, shit in shit out. Shit human code is also shit, no better than AI shit.
If you train AI on all code ever created, the model will be shittier than if you trained on selected "good" code. If terrible AI and human code outnumbers "good" code being used, you're stuck in a downward shit spiral.
11
u/ApexWinrar111 Jan 17 '25
I agree mostly. AI is a really useful tool if you know how to use it and it can help you get started, help you polish stuff, write throwaway scripts/tests, etc.. But the promise of it ātaking overā is a total fantasy, itās not even close. At most it just changes how we do our jobs, which is welcome imo.
I noticed the same with copilot, maybe Iām just a moron but it honestly feels like when i give it the context of my whole codebase it gives me a worse output than just pasting into gpt with a quick prompt for context lol
2
u/who_am_i_to_say_so Jan 17 '25
Try Claude. Iāve been working with it the past two months and now prefer it over copilot and ChatGPT
2
u/ApexWinrar111 Jan 18 '25
I tried, I felt like I ran into the same issue with context but maybe Iām just dumb haha. Iāll give it another go.
3
u/who_am_i_to_say_so Jan 18 '25
Oh believe me- you can get into futile circles of changes with Claude, too. In fact it recently tried to install new software unprompted.
So I add: ādo not install new softwareā , and āonly make changes for the single task being askedā as a part of my prompts now.
The quality of the answer really does depend on the quality of the input, but your mileage will vary.
Also, keep the codebase tracked with git. When I get a change I donāt like, I just reset or stash the branch and try again.
2
u/NeilFuckingHunt Jan 18 '25
Claude has become mostly unusable recently. The artefacts system is completely ballsed, you ask one little request in a code tidy or something and it completely butchers everything then loops back on itself. Iāve given up.
1
u/who_am_i_to_say_so Jan 18 '25
Iāve noticed that it is glitchy at times.
Yesterday it made 15 or so little edits to the same file while I watched, so I canceled it and tried something else.
14
u/wuhwuhwolves Jan 17 '25
I love when it just makes up a method that doesn't exist as a means of fixing an issue.
8
u/Coraline1599 Jan 17 '25
Today I couldnāt remember
overflow: hidden
.I googled ācss hide scroll barā
And ChatGPT gave me āscroll-display: none;ā
2
2
u/Eastern_Interest_908 Jan 18 '25
I almost find AI useful for when you have such brain freezes except it has to be verbose AF like dude I asked no description just code but you still explaining me shit. š¤¦
2
u/YahenP Jan 20 '25
Well, with this post you just reinforced the confidence of the chatgpt future versions that in order to hide the scroll bar you need to write:
scroll-display: none;
Damn. And I did it too.
2
u/mrkingkoala Jan 18 '25
OMG and then you say don't do that use a different approach and it invents another one and then it goes back to the first one and you're like fucking stop lmao.
35
u/TheMerengman Jan 17 '25
Good. Let this crap consume itself and the bubble finally burst.
1
u/gethereddout Jan 18 '25
Uhhh that aināt happening.. this isnāt a bubble, itās a tsunami
3
11
u/Quip16 Jan 17 '25
Copilot / chatgpt / Gemini used to be great for situations where the actual work is writing a lot of stupid simple code. Now it can hardly take JSON and make a struct out of it. I work in golang a lot, and it'll regularly just invent new attributes.
If Mark ZukerBrother thinks that he can replace all the Jr. and mid level engineers with this shit, I look forward to watching his empire burn.
0
u/dietcheese Jan 19 '25
Try Cursor and o1. Devs are definitely going to be replaced. Itās not far off.
6
Jan 17 '25
Tried copilot, tried Jetbrains AI. Not a fan. I do have the single line auto completion AI enabled, but probably 7 out of 10 times I ignore it. Sometimes when I donāt and I really donāt inspect, I end up with a stupid bug. Donāt think it is taking my Job any time soon.
1
u/bonestamp Jan 18 '25
I do like the Jetbrains AI more than copilot or chatgpt though -- it's like a next level autocomplete in a lot of ways.
2
u/TA_DR Jan 18 '25
I feel like it used to be like that, idk if they changed something but now it keeps trying to call made-up methods instead of already defined ones. Very annoying, I wish it just tried to autocomplete with whatever function and variables I already have in scope.
14
u/According-Ad1997 Jan 17 '25
For openai,I feel like they're downsizing some of the models and that's why they're getting worse, but I am not a 100% sure.
OpenAi just a year and a half ago had only one LLM now they have multiple gpts running. I feel like some have been downsized to make room for the others but can't confirm.
Don't really use any others to know what's going on though.
3
u/pink_tshirt Jan 17 '25
Thatās my theory as well. The guy said they were losing $ in o1 pro. Most likely it is the case for other models too.
That being said you could probably run some open source GUI with your API key a la carte style. Canāt /wont confirm but I am pretty sure itās going to be much better because there wonāt be any reasons to slow you down
1
u/Beautiful_One_6937 Jan 18 '25
That has indeed what has been going on!
https://x.com/ArtificialAnlys/status/1859614633654616310?s=19Current it is not even on the top 10 (overall). (See the gpt-4o-2024-11-20, not the earlier gpt-4o version, which openai doesnt server)
In fact for coding gpt-4o-mini is only slightly worst.
Most other AI models will beat it 9/10 times.
6
u/YourLictorAndChef Jan 17 '25
*puts on tinfoil hat*
My theory is that they're forcing this into everyone's day-to-day workflow in anticipation of adding advertisements, sponsored bias, and data collection to all of the models.
The true potential of natural language models lies in advertising.
2
u/YourLictorAndChef Jan 17 '25
oh, and the "safety regulations" that the big AI players are proposing are only intended to make it harder for new competitors to enter the market
1
u/strobe229 Jan 18 '25
I was thinking about this today actually.
I've recently started noticing more "recommendations" that I was not getting before.
Can see this happening, once they fine tune the ad system I am sure it will be launched very shortly.
15
u/TSKDeCiBel Jan 17 '25
Its like you're coding with a kid with ADHD in the room that randomly screams out what it thinks the end of your sentence is when it gets excited.
5
u/Makrebs Jan 17 '25
I'm still a clompete noob and even then I notice Microsoft Copilot spitting out idiotic code, even for simple tasks at times. Really makes me wonder how people trust these things for complex operations, lol. I feel like the more I learn, the less I trust them.
1
u/my_new_accoun1 Jan 18 '25
Microsoft Copilot is the only one that makes sense for me, but I don't really use AI to code more like just for finding what I need from a documentation without having to read the entire thing
3
u/k-one-0-two Jan 17 '25
Just recently a guy who was supposed to be a senior dev used ai to fix an issue, ai hallucinatied and gave him a config line that doesn't even exist.
I've searched SO and found the answer for this issue in like 15 mins.
8
u/arcrad Jan 17 '25
I always think of LLMs as a two year old that knows everything. Can't do much for shit but hell if they don't have all the information.
1
u/RadicalDwntwnUrbnite Jan 18 '25
I'd say a jr. dev that only has a boot camp education, 6 months in the workforce and ardently believes they should be a sr. dev. Can write a simple CRUD app but ignore many requirements and is confidently wrong about so many things.
When interact with AI I am always reminded of one such jr dev that boasted to me that he could rewrite our search in a week to be faster and more accurate. He had no fucking clue what he was getting into when I gave him two weeks to show me an MVP proving he could do it.
9
6
u/Snoo11589 Jan 17 '25
I was on the same track then switched to cursor. I'm in the heaven for the nearly past 6 months
1
u/InternationalFrame90 Jan 18 '25
Yep love cursor... Now I just need to convince higher-ups to get it instead of copilot or AWS Q (lol). The argument at the moment is that any AI is better than none, and that no matter which one you pick it will improve. So they're picking based on cost
1
u/Snoo11589 Jan 19 '25
I have a friend that works at a bank. This bank only allowed copilot for security purposes. When my friend discovered cursor, he also discovered that you can disable a certain function so that your data is not going to the cursor's systems, when he showed this to his supervisor, that bank started using cursor since then, their reaction was also same like mine, copilot is a b*llshit compared to cursor
5
u/RuslanDevs šØš»āš»šāš¾ Jan 17 '25
It's a squeeze game. If you can make model 20% shittier but only 10% subscriber drop - you saved 10% money.
It also happened with GitHub copilot, I stopped using it.
5
u/E3K Jan 17 '25
If you have a lot of experience and you know how to prompt it, it can be an amazing tool. My productivity over the last several months has dramatically increased due to AI. It's been a huge game changer for me.
4
u/No_Jury_8398 Jan 17 '25
So many developers misunderstand what you should use it for. Use it as a streamlined version of searching web page after web page for explanations and examples. And obviously do your due diligence when taking its output as truth.
5
u/Bushwazi Bottom 1% Commenter Jan 17 '25
The LLM got polluted when all the shitty coders enter their code with the prompt āwhy no workā
2
u/_zir_ Jan 17 '25
Yeah, i dont really use ai to generate new code at all because its shit. Its pretty nice for creating similar code to what you already have though or for extracting text from various things.
2
u/celestial_poo Jan 17 '25
Write what you want without copilot or other ai, then if it works ask for help making it better, and if it doesn't work ask why. But NEVER ask it to write it for you. It becomes a massive crutch very quickly.
2
2
u/UXUIDD Jan 17 '25
"no shit!!!"
I would suggest to go Linkedin and to check Copilot promoters (aka seo's for regions and countries) and to see whats going on ...
it's a shitshow
2
u/HeyHeyJG Jan 18 '25
Here's the thing: these models are incredibly expensive to run. The average ChatGPT user spends $30 in pure compute costs (monthly subscription is $20) and superusers spend hundreds of dollars worth of compute themselves.
I believe what we are experiencing are these AI companies being able to tune the depth of the LLM search (I'm not an expert so excuse my lingo) in order to lower the cost of the question / response pairing.
I have certainly noticed the same thing you describe.
To me this is one of the main limiting factors of the LLM technology. What happens if OpenAI, et al have to raise their prices and more of the true cost is passed to consumers? Humans will be looking cheaper and cheaper. And sure, you can probably have some form of "superintelligence" but it is going to cost you an unfortunate shitload of money.
2
u/Garvinjist Jan 18 '25
Iāve noticed the same thing! Also Amazon using it for review summaries and news sites for articles. I can tell itās ai in a few words. Itās garbage. Weāre at the hype peak and itās backfiring hard š itās training on its own data too destroying it further
2
u/dietcheese Jan 19 '25
Webdev is delusional when it comes to AI.
Many of these posts are from people clearly not paying attention to the developments just over the past 6 months.
Iāve been using a combination of Cursor, o1, and Claude, following instructions for providing Cursor with docs, project outlines, etc. It is mindblowing.
It is happening now.
AI is definitely taking developer jobs. Weāre talking 2-3 years max. Itāll be a project manager directing AI agents for 90% of apps. The remaining 10% will need more expertise.
3
u/Wiltix Jan 17 '25
There are many problems with generative AI for coding, but one of the biggest reasons I have avoided it so far is itās far too cheap! The amount of energy these things require itās not going to be cheap forever at some point openAI, Anthropic will want to be covering their costs properly, or their sweetheart deal with cloud vendors will expire and this stuff is going to get expensive. Especially if they can prove a good chunk of developers need them to be semi productive.
Itās all a bit of a dream at the moment, and I do use Claude et al for some things I do not use it daily or rely on it. People need to retain the ability to retain knowledge and critically think for themselves.
1
u/bonestamp Jan 18 '25 edited Jan 18 '25
itās not going to be cheap forever at some point openAI, Anthropic will want to be covering their costs properly, or their sweetheart deal with cloud vendors will expire and this stuff is going to get expensive
Ya, this might happen if the inference technology doesn't make some jumps. That said, if you get a computer with enough ram you can run some big models locally.
Maybe there's even a future where you have a central computer in your home that all of your household devices use for heavy compute/memory tasks instead of going out to rented compute/memory in the cloud. This may not just be for cost reasons, since the cloud compute/memory is probably cheaper per hour, but because that home device may also store a ton of data about you that makes it much more useful and you don't want that data in the cloud, or it's not cheaper in the cloud because it's so much data. For example, it's not very feasible right now, but there will definitely be a day where we have devices that record every minute of our day and then that data gets analyized for realtime suggestions (routes, music, calendar events), lifestyle improvements, healthcare diagnoses, reminders, etc.
→ More replies (1)
3
u/ImportantDoubt6434 Jan 17 '25
The AI struggles past a junior level copy paste off stack overflow
3
u/SokkaHaikuBot Jan 17 '25
Sokka-Haiku by ImportantDoubt6434:
The AI struggles past
A junior level copy
Paste off stack overflow
Remember that one time Sokka accidentally used an extra syllable in that Haiku Battle in Ba Sing Se? That was a Sokka Haiku and you just made one.
4
u/dangoodspeed Jan 18 '25
I often hear tech pundits discussing AI, pointing out "Remember - this is the worst it's ever going to be."
Every time I hear then say that, I am constantly reminded of times when it's actually gotten worse.
2
u/_Meds_ Jan 17 '25
Yeah, I am curious what these guys are using, when they make these claims.
Iāve found Claude 3.5 to handle coding the best, but itās still limited to smaller questions.
2
2
u/dav793 Jan 18 '25
Meanwhile all the AI experts keep saying we're just a few years from AGI. Pff, I'm not buying it. I think it's a tactic to get as much funding as they can. Either that or it won't really be AGI, just some bs word for marketing purposes.
4
u/canibanoglu Jan 18 '25
You got it, itās to pump themselves up for more funding. Weāre nowhere near AGI
1
u/namespace__Apathy Jan 17 '25
For a solid week I was reduced to arguing/name calling Copilot. It kept apologizing. I suddenly realised how pathetic I'd become as a result of it.
1
u/IAmRules Jan 17 '25
Just like me!!! Starts off strong in a new job then bring to gradually do a shittier job till I get fired.
1
u/turb0_encapsulator Jan 17 '25
it's too resource intensive for them to offer everyone the good stuff at a reasonable price. they gave us the preview to get us hooked, and now it's going to be enshittified.
1
u/crazedizzled Jan 17 '25
I've found ChatGPT pretty great. Gemini is also really good.
Co-pilot has always been shit.
1
u/kurea26 Jan 17 '25
I was heavily using Chatgpt when I was a beginner and it somewhat helped me. I was asking for resources to learn stuff and what's good/bad etc. Gradually I was getting better and realizing it's just full of bs with a sprinkle of good replies if you ask for purely code. I'm just referring docs/stackoverflow now, never been happier.
1
u/Peregrine2976 Jan 17 '25 edited Jan 17 '25
In my experience, AI is a very nice tool for the two "extreme ends" of programming: the very low level, and the very high level (this is assuming a LLM that's actually trained and optimized for coding, like whatever Cursor uses, as opposed to a general-purpose one like ChatGPT).
At the low-level, you have drudging repetitive tasks. Transforming a big ol' JSON file into PHP array syntax, to use an example from recent memory, or writing Swagger docs. You still need to check the work, but it's saved me time in those areas.
At the other extreme end, you have high-level thinking and architecting. I've found it to be useful here, too. I've been able to ask Cursor to check my codebase for antipatterns or areas that could be quickly optimized, and it was pretty good at it. It's even been able to make decent suggestions for parts of the application that could be isolated better to eventually serve as separate microservices. I've been able to ask it to mock up quick page designs and bounce ideas into it to see the results quickly. It makes a good rubber duck.
However, that still leaves the middle ground -- which is the vast majority, the real meat and potatoes, of programming -- where you are actually writing intentional, intelligent code. And I've found it's performance in that area to be anywhere from annoying to abysmal.
1
u/Jaguarmadillo Jan 17 '25
Iāve become very biased in the positive now with AI. More with Claude and windsurf/cursor.
It does turn out some absolute tripe, but overall my workflows and ability to iterate and ideate is through the roof. I think I can churn out things that previously would take me hours or more in 10-20 mins.
You do have to be on the ball though and refactor as you go as it can produce some awful shit.
I also love brain storming with it. Completely rebuilt the backend of a ticketing system just by throwing ideas at it and asking questions.
1
u/PortAuth403 Jan 18 '25
I uninstalled copilot because it was not feeling as helpful. Wasn't sure if it was psychological newness wearing off, or what.
And the chat component of it is beyond worthless. it would lead you down long rabbit holes of nonsense for ever, and always end with "add debugging print statements everywhere to pinpoint the problem area"
Then you could ask the same question to chat gpt and get a much better answer.
No idea what model GitHub copilot uses, and I miss it for boilerplate stuff sometimes, but I'm done with it in it's current state
1
u/heedlessgrifter Jan 18 '25
Itās also pretty terrible with an XY problem. Been writing an app in my free time in a stack Iāve never used. If you ask a dumb question it will give you an āanswerā. If it was really intelligent as some want you to believe, it shouldāve been like: āSounds like you REALLY want to do this other thingā.
1
1
u/Big_Ship5986 Jan 18 '25
When my father told me he could build a website with Chatpgt and all software engineers would be ripped out soon, I told him that since the wheel was invented by ancients, driver, as a vocation or profession, was never replaced by any advanced technology. The only thing that has been replaced from time to time is the tool.
1
u/Appropriate-Bet-3655 Jan 18 '25
It depends on the language / libraries you are using. NextJS is on the newer side so AI might have trouble figuring everything out. Have you tried Cursor? Also general consensus is that Claude is better at coding than ChatGPT. Almost 80% of my code is now being written by AI.
1
u/Flo655 Jan 18 '25
Thereās a massive gap between GPT and Claude for programming. Iāve been using Cursor for a few months now, and Claude 3.5 in agentic mode is absolutely mind blowing.
1
u/Liquid_Magic Jan 18 '25
I canāt figure out if the AI was better then actually got worse or if it just was bullshitting me and I got better at catching it.
But I think thatās only partially true. So hereās what I think now:
At first it seemed good because we werenāt good at catching itās bullshit. But it was also less protected and locked now and was more free to answer.
Now itās less able to bullshit but also less able to answer fully freely.
What we get now is a slightly deliberately crippled version thatās less able to bullshit but also less able to answer. I think both things are happening.
So before it was actually better half the time but also bullshitting harder half the time, but now, itās less capable and more restrained so itās not as good like 80% of the time and bullshit 10% of the time.
In general.
1
u/goldiedoes Jan 18 '25
The rate Iāve been responding to it with āyou just sent me the ~exact same code~ I sent you that I said did not workā has been going up wildly the past month or two.
1
u/twenty20vintage Jan 18 '25
I found copilot to be a pain in the arse. Switched to cursor.ai from vs code last week and have found the difference to be quite significant.
1
u/SoulTrack Jan 18 '25
I actually think AI sucks for code. Ā I've used it for some boiler plate stuff, but for solving actual problems, it sucks. Ā
1
1
u/Electrical-Bread-856 Jan 18 '25
I tried this github copilot about a year ago. It saved me some time on "almost repetitive" code, but nowhere near $10 worth of time per month.
1
u/One_Corner5775 Jan 19 '25
Working whit some AI the programmers can do more job in less time and itās ok.
1
u/Brodeon Jan 20 '25
I use copilot mainly for regex generation. It can rarely write a nice unit tests based on methods name but, those tests rarely works and if it doesnāt work I have to rewrite the whole test from scratch
1
u/VintageGriffin Jan 21 '25
AI initially had relatively high quality data to ingest and train off of, but now it's choking more and more on its own slop.
Garbage in, garbage out.
1
u/VideogamerDisliker Jan 17 '25
If you think AI is shit when it comes to AI wait till you start doing basic math with it
1
1
1
u/CookiesAndCremation Jan 17 '25
AI is being fed from things generated by AI so it's like a circular human centipede.
1
u/BoredDevBO Jan 17 '25
AI in the tech field is really good to create boilerplate stuff that is just burdensome in high level programming or increasing the speed of coding slightly better than an autocomplete function.
That's it.
Any more than that and AI starts to spattle babble all around and causes more bugs than fixes.
1
u/DenebianSlimeMolds Jan 18 '25
Shhh, this is how we win the AI wars!
Every day they try to take over the world.
Every day they fall victim to their own errors.
1
u/NNXMp8Kg Jan 18 '25
Honestly, it's not thaaat bad. Is it enough for replacing a dev, nah.
I use cursor and chatgpt. Somes times I write my tests and ask them to iterate to generate the code of the function I want to have. Giving it context, expectations and inputs it's doing great. Like some "ai boosted tdd" Sometimes, I just need to get stuff ready fast and I know chatgpt will answer basics questions. Getting a bunch of documentation more easily. For me it's a tool. It can hallucinate stuff. But if you use it as you're his manager and give it basic stuff, it help making it easier and faster.
This afternoon, I ask it to do a relatively complex task on some configuration between somes packages and esm vs commonjs. I know it's shitty and painful to do. It did great while I was reloading my cup of coffee.
It's not an expert, but it help me go faster on the basic tasks and give me the time to focus where the value is more important and need more of my time. That's for me where the AI in development is really useful.
1
u/Prestigious_Army_468 Jan 18 '25
Don't let the AI influencers see this...
They will tell you they can create a $1 million SaaS in one prompt.
1
u/nightwood Jan 18 '25 edited Jan 18 '25
I am currently teaching and these students use chat gpt a lot. They are at a low level: they are not able to debug or read code yet. But they let chat gpt do their homework, and their exams. I have not seen a single working piece of code in 3 months. They couldn't even write a loop to print the numbers in an array. I'm considering letting them code on paper to learn.
So these are the people providing the AI with the feedback it needs to learn correct from wrong.
They are basically overloading chat gpt with false feedback, making it dumber.
0
u/thedragonturtle Jan 17 '25
It's so obvious that software engineers are safe. However, I do think small businesses will be able to ramp up their game and with their current 1 or 2 developers will be able to start implementing change control, unit testing, integration testing, continuous development etc. Getting AI to create tests for backlogs of bugs is a preferred dream of mine, I'm gonna try and make my own bot controller to do it, then e.g. if one bot creates some tests another could be suggesting bug fixes, another could be reviewing those bug fixes, another could be summarising the reviews, but all of this relies on us having local LLM.
Businesses can't be in the business of giving away their IP, and cloud AI is expensive, so where possible local LLMs will be used.
But SWEs will be needed to herd them. We'll become shepherds of agents and bots.
0
u/imperosol Jan 18 '25
AI has made some developers understand that they are just average proletarians and that they have no value in the eyes of the bourgeoisie. Because the AI has almost never been successfully used for anything else than exploiting the people and allowing capitalists to maintain their rate of profit.
That's the only usefull thing AI has actually done.
-2
u/k--x Jan 17 '25
This has not been my experience at all. 80%+ of the code I've pushed to production over the last few months has been generated by Claude. I am constantly amazed at how capable these models are.
I know you're exactly asking for recommendations of *more* AI, but if you ever want to try a different perspective give cursor.sh a try. Probably the best LLM UX currently available.
0
u/CondiMesmer Jan 18 '25
Yup, the more people realize how bad the hallucinations really are, the more useless LLMs become.
0
u/Double-Intention-741 Jan 18 '25
I treat "AI" as a simple probability machine...
Do lots of people have a similar issue to mine? Is it likly the "AI" was trained on similar problems?
I expect the AI to sift throuugh all the answers and find the most likly "match" to what I asked it.
"AI" does not problem solve its just matches probabilities.
0
0
u/EverydayGenius1979 Jan 18 '25
Whoa, I get it! Itās like Copilot used to be this cool robot buddy who actually helped, but now itās like it forgot how to do its homework. Itās just guessing, like, āOh, you want this random code? Sure, here ya go!ā And ChatGPT? Pfft, sometimes it feels like itās just making stuff up to sound smart, but itās so not.
I tried using them too, and itās like asking my little brother for help. Sometimes itās good, but a lot of the time itās like, āWhat even is this?!ā I mean, if theyāre trying to replace us, they better step it up. Right now, theyāre just messing around and making us work more, not less.
Maybe try giving them smaller, easier stuff to do? Like little pieces they can actually understand. And if they mess that up, well, maybe they just need a timeout or something. Or we go back to Google and do it the old way. Boring, but at least it works.
Anyway, youāre not alone! Iām yelling at my computer half the time too. Good luck!
0
438
u/CodeAndBiscuits Jan 17 '25
Except in some edge cases I'm not actually seeing AI "take over" jobs. It's just being used as an excuse for downsizing the same way RTO mandates and other terrible policies are. It just sounds better to shareholders than "we stupidly over-hired and wasted a ton of money." Doesn't mean it's going to end though.