Whether it's vibe coding, agentic coding, or copy pasting from the web interface to your editor, it's still sad to see the normalization of private (i.e., paid) LLM models. I like the progress that LLMs introduce and I see them as a powerful tool, but I cannot understand how programmers (whether complete nobodies or popular figures) dont mind adding a strong dependency on a third party in order to keep programming. Programming used to be (and still is, to a large extent) an activity that can be done with open and free tools. I am afraid that in a few years, that will no longer be possible (as in most programmers will be so tied to a paid LLM, that not using them would be like not using an IDE or vim nowadays), since everyone is using private LLMs. The excuse "but you earn six figures, what' $200/month to you?" doesn't really capture the issue here.
Once it becomes economical to run a Claude 4 class model locally you'll see a lot more people doing that.
The closest you can get right now might be Kimi K2 on a pair of 512GB Mac Studios, at a cost of about $20,000.
> The top-end Ryzen AI Max+ 395 configuration with 128GB of memory starts at just $1999 USD. This is excellent for gaming, but it is a truly wild value proposition for AI workloads. Local AI inference has been heavily restricted to date by the limited memory capacity and high prices of consumer and workstation graphics cards. With Framework Desktop, you can run giant, capable models like Llama 3.3 70B Q6 at real-time conversational speed right on your desk. With USB4 and 5Gbit Ethernet networking, you can connect multiple systems or Mainboards to run even larger models like the full DeepSeek R1 671B.
I'm futsing around with setups, but adding up the specs would give 384GB of VRAM and 512GB total memory, at a cost of about $10,000-$12,000. This is all highly dubious napkin math, and I hope to see more experimentation in this space.
There's of course the moving target of cloud costs and performance, so analysing break-even time is even more precarious. So if this sort of setup would work, its cost-effectiveness is a mystery to me.
[0] https://frame.work/be/en/blog/introducing-the-framework-desk...
It will run some big MoEs at a decent speed (eg, Llama 4 Scout 109B-A17B Q4 at almost 20 tok/s). The other issue is its prefill - only about 200 tok/s due to having only very under-optimized RDNA3 GEMMs. From my testing, you usually have to trade off pp for tg.
If you are willing to spend $10K for hardware, I'd say you are much better off w/ EPYC and 12-24 channels of DDR5, and a couple fast GPUS for shared experts and TFLOPS. But, unless you are doing all-night batch processing, that $10K is probably better spent on paying per token or even renting GPUs (especially when you take into account power).
Of course, there may be other reasons you'd want to inference locally (privacy, etc).
I use local LLMs as much as possible myself, but coding is the only use case where I still entirely defer to Claude, GPT, etc. because you need both max speed and bleeding edge model intelligence for anything close to acceptable results. When Qwen-3-Coder lands + having it on runpod might be a low end viable alternative, but likely still a major waste of time when you actually need to get something done properly.
In my experience, something Llama 3.3 works really well for smaller tasks. For "I'm lazy and want to provide minimal prompting for you to build a tool similar to what is in this software package already", paid LLMs are king.
If anything, I think the best approach for free LLMs would be to run using rented GPU capacity. I feel bad knowing that I have a 4070ti super that sits idle for 95% of the time. I'd rather share an a1000 with bunch of folks and have that run at close to max utilization.
In the mid to long term the question is, is the subscription covering the costs of the LLM provider. Current costs might not be stable for long.
I will switch to whatever is best for me at a good price, and if thats not sustainable then I'll be fine too; I was a developer before these existed at all, and local models only help from there.
You can plug in the $60 Coral to a Raspberry Pi and get real-time image recognition running in Frigate.
When I can have:
1) Something similar inside my computer/laptop
2) Something I can plug in to my computer via USB-C
3) Something I can buy and install to my LAN so all devices in my home can connect to it
I'll buy it instantly.
What I don't want is a massive generic GPU that just happens to be good at AI workloads, I want custom hardware that's more efficient and cheaper.
(Off topic, but my guess is that Apple is aiming for #3 with an Apple TV variant so you can have more power than your phone, but still keep it 100% local)
By that time Claude 5 (or whatever) will be available over API.
I am grateful for upward pressure from models with published binaries - I do believe this is fundamental floor-raising technology.
Choosing frontier-1 for the sake of privacy, autonomy, etc will always be a hard sell and only ever to a pretty niche market. Even me - I'm ideologically part of this market, but I'm already priced out hardware wise.
(If you need even more memory you could equip one of those servers with 6TB of DDR3 but you’ll lose a bit of bandwidth if you go over 2TB. DDR4 is also a slightly faster option but you’re spending 4x as much for the same capacity.)
People were buying stores empty of GPUs to mine for BTC.
Then people built custom ASICs that couldn't do anything but mine BTC, but did it a lot cheaper and with a lot less electricity required -> nobody GPU mines anymore pretty much.
I'm waiting for a similar thing to happen to local AI.
Edit: And also, to get even half as much memory, you need to spend $10k. If you want to host actually-large LLMs (not quantized/distilled versions), you'll need to spend close to that much. Maybe you can get away with 256GB for now, but that won't even host full Deepseek now (and I don't know if 512GB either, with OS/etc overhead, and a large context window).
I trust AWS, but we also pay big bucks to them and have a reason to trust them.
I use Qwen2.5 coder for auto complete and occasional chat. I don't want AI to edit my code and so this works well for me.
I agree that the hardware investment for local AI is steep but IMO the local models are good enough for most experienced coders who just want a better autocomplete than the one provided by the IDE by default.
Total newb here.
Considering as you said, that LLMs are doing a form of compression, and assuming generously that you add extra compression on top, yeah, now I understand a bit more. Even if you focus on non-similar code to get the most coverage, I wouldn't be shocked if a modern, representative source code training data from Github weighed 1TB, which obviously is a lot more than consumer grade hardware can bear.
I guess we need to ramp up RAM production a bunch more :-(
Speaking of which, what's the next bottle neck except for storing the damned things? Training needs a ton of resources but that part can be pooled, even for OSS models, it "just" need to be done "once", and then the entire community can use the data set. So I guess inference is the scaling cost, what's the most used resource there? Data bandwidth for RAM?
Historically these sorts of things happened because of Moores law. Moores law is dead. For a while we have scaled on the back of "more cores", and process shrink. It looks like we hit the wall again.
We seem to be near the limit of scaling (physics) we're not seeing a lot in clock (some but not enough), and IPC is flat. We are also having power (density) and cooling (air wont cut it any more) issues.
The requirements to run something like claud 4 local aren't going to make it to house hold consumers any time soon. Simply put the very top end of consumer PC's looks like 10 year old server hardware, and very few people are running that because there isn't a need.
The only way we're going to see better models locally is if there is work (research, engineering) put into it. To be blunt that isnt really happening, because Fb/MS/Google are scaling in the only way they know how. Throw money at it to capture and dominate the market, lock out the innovators from your API and then milk the consumer however you can. Smaller, and local is antithetical to this business model.
Hoping for the innovation that gives you a moat, that makes you the next IBM isnt the best way to run a business.
Based on how often Google cancels projects, based on how often the things Zuck swear are "next" face plant (metaverse) one should not have a lot of hope about AI>
This is exaggeration. CPUs are still getting faster. IPC is increasing, not flat. Cooling on air is fine unless you’re going for high density or low noise.
This is just cynicism. Even an M4 MacBook Pro is substantially faster than an M1 from a few years ago, which is substantially faster than the previous versions.
Server chips are scaling core counts and bandwidth. GPUs are getting faster and faster.
The only way you could conclude scaling is dead is if you ignored all recent progress or you’re expecting improvements at an unrealistically fast rate.
Benchmarks going up is not IPC increasing. These are separate things.
Please look IPC for the latest GPU's from Nvidia, the latest CPU's from AMD. The IPC is flat. See intel loosing credibility with failing processors due to power problems from clocking because IPC is flat.
> Even an M4 MacBook Pro is substantially faster than an M1
Again, clocking. m4 (non pro) vs m1 are so close in IPC on common tasks that its negligible. The performance gains between the two are from memory bandwidth not core performance.
> Server chips are scaling core counts
Parallelism is not the same as performance. Intel dropping the "core duo" 20 year ago was that RUNNING at 2ghz was an admission that single threading was ending. 20 years on were 20 cores deep (consumer), and only at 4ghz with "boost clocks" (back to that pesky power and cooling problem).
And this product still exists today: the N150 (close enough). Its has lower power consumption and more cores. And what was the single core performance gain? 35% Improvement in 20 years.
None of these things are running any of the LLM's that power the tools were talking about. Those are in the datacenter. 700 core CPU's, 400-800gbps top of rack switching are the bleeding edge. This is where "power" and cooling have hit the wall. The spacing requirements of a bleeding edge NVIDIA install are impacting the costs of interconnect between systems. Lots of fiber and needing to be spaced out because of power/heat adds up to a boat load of extra networking costs. Having half empty racks because of density is now a reality.
And you see these same issues at home: power demands of GPU's for consumers and workstations are thought he roof. Were past what the PCI spec can provide, all that power is heat and has to go somewhere. Sometimes it burns up poorly designed connectors. The latest gen is consumes even more power, to push clocks higher, for very little gain (see flat IPC nvida).
Moores law is dead, has been for along time. There is nothing to outpace.
> That's what DeepSeek V3 was about.
This would be a foundational shift! What problem in complexity theory was solved that the rest of computing missed out on?
Don't get me wrong MOE is very interesting but breaking up one large model into independent chunks isn't a foundational breakthrough its basic architecture. It's 1960's time sharing on unix basics. It's decomposition of your application basics.
All that having been said, there is a ton of room for these sorts of basic, blood and guts engineering ideas to make systems more "portable" and "usable". But a shift in thinking to small, targeted and focused will have to happen. Thats antithetical to everything in one basket throw more compute at it and magically we will get to AGI. That clearly isnt the direction the industry is going... it wont give any one a moat, or market dominance.
None. We're still in the "if you spend enough effort you can make things less bad" era of LLMs. It will be a while before we even find out what are the theoretical limits in that area. Everyone's still running on roughly the same architecture after all - big corps haven't even touched recursive LLMs yet!
Right now, hardware wise, we need more RAM in GPUs than we really need compute. But it's a breakpoint issue: you need enough RAM to hold the model. More RAM that is less than the model is not going to improve things much. More RAM that is more than the model is largely dead weight.
I don't think larger models are going to show any major inference improvements. They hit the long tail of diminishing returns re: model training vs quality of output at least 2 years ago.
I think the best anyone can hope for in optimizing current LLM technology is improve the performance of inference engines, and there at most I can imagine only about a 5x improvement. That would be a really long tail of performance optimizations that would take at least a decade to achieve. In the 1 to 2 year timeline, I think the best that could be hoped for is a 2x improvement. But I think we may have already seen much of the low hanging optimization fruit already picked, and are starting to turn the curve into that long tail of incremental improvements.
I think everyone betting on LLMs improving the performance of junior to mid level devs and that leading to a Renaissance of software development speed is wildly over optimistic as to the total contribution to productivity those developers already represent. Most of the most important features are banged out by harried, highly skilled senior developers. Most everyone else is cleaning up around the edges of that. Even a 2 or 3x improvement of the bottom 10% of contributions is only going to grow the pie just so much. And I think these tools are basically useless to skilled senior devs. All this "boilerplate" code folks keep cheering the AI is writing for them is just not that big of a deal. 15 minutes of savings once a month.
But I see how this technology works and what people are asking it to do (which in my company is basically "all the hard work that you already weren't doing, so how are you going to even instruct an LLM to do it if you don't really know how to do it?") and there is such a huge gap between the two that I think it's going to take at least a 100x improvement to get there.
I can't see AI being all that much of an improvement on productivity. It still gives wrong results too many times. The work needed to make it give good results is the same sort of work we should have been doing already to be able to leverage classical ML systems with more predictable performance and output. We're going to spend trillions as an industry trying to chase AI that will only end up being an exercise in making sure documents are stored in a coherent, searchable way. At which point, why not do just that and avoid having to pressure the energy industry to firing up a bunch of old coal plants to meet demand?
Yep... Copy and paste with find and replace already had the boilerplate code covered
Every one wants this to happen they are all trying but...
EUV, what has gotten us down to 3nm and less is HARD. Reduction in chip size has lead to increases in density and lower costs. But now yields are DOWN and the design concessions to make the processes work are hurting costs and performance. There are a lot of hopes and prayers in the 1.8 nodes but things look grim.
Power is a massive problem for everyone. It is a MASSIVE a problem IN the data center and it is a problem for GPU's at home. Considering that locally is a PHONE for most people it's an even bigger problem. With all this power comes cooling issues. The industry is starting to look at all sorts of interesting ways to move heat away from cores... ones that don't involve air.
Design has hit a wall as well. If you look at NVIDIA's latest offering its IPC, (thats Instructions Per Clock cycle) you will find they are flat. The only gains between the latest generation and previous have come from small frequency upticks. These gains came from using "more power!!!", and thats a problem because...
Memory is a problem. There is a reason that the chips for GPU's are soldered on to the boards next to the processors. There is a reason that laptops have them soldered on too. CAMM try's to fix some of this but the results are, to say the least, disappointing thus far.
All of this has been hitting cpu's slowly, but we have also had the luxury of "more cores" to throw at things. If you go back 10-15 years a top end server is about the same as a top end desktop today (core count, single core perf). Because of all of the above issues I don't think you are going to get 700+ core consumer desktops in a decade (current high end for server CPU)... because of power, costs etc.
Unless we see some foundational breakthrough in hardware (it could happen), you wont see the normal generational lift in performance that you have in the past (and I would argue that we already haven't been seeing that). Someone is going to have to make MAJOR investments in the software side, and there is NO MOAT by doing so. Simply put it's a bad investment... and if we can't lower the cost of compute (and it looks like we can't) its going to be hard for small players to get in and innovate.
It's likely you're seeing a very real wall.
OP is right, these people are posers and fakers, not programmers.
The models are somewhat getting there: even the smaller ones like Qwen3-30B-A3B and Devstral-23B are okay for some use cases and can run decently fast. They’re not amazing, but better than much larger models a year or two ago.
The hardware is absolutely not there: most development laptops will be too weak to run a bunch of tools, IDEs and local services alongside a LLM and will struggle to do everything at the pace of those cloud services.
Even if you seek compromise and get a pair of Nvidia L4 cards or something similar and put them on a server somewhere, the aforementioned Qwen3-30B-A3B will run at around 60 tokens/second for a single query but slow down as you throw a bunch of developers at it that all need chat and autocomplete. The smaller Devstral model will more than halve the performance at the starting point because it’s dense.
Tools like GitHub Copilot allow an Ollama connection pretty easily, Continue.dev also does but can be a bit buggy (their VS Code implementation is better than their JetBrains one), whereas the likes of RooCode only seem viable with cloud models cause they generate large system prompts and need more performance than you can squeeze out of somewhat modest hardware.
That said, with more MoE models and better training, things seem hopeful. Just look at the recent ERNIE-4.5 release, their model is a bit smaller than Qwen3 but has largely comparable benchmark results.
Those Intel Arc Pro B60 cards can’t come soon enough. Someone needs to at least provide a passable alternative to Nvidia, nothing more.
I have 128 GB RAM on my laptop, and regularly run multiple multiple VMs and several heavy applications and many browser tabs alongside LLMs like Qwen 3 30B-A3B.
Of course there’s room for hardware to get better, but the Apple M4 Max is a pretty good platform running local LLMs performantly on a laptop.
If you have no interest in the inner workings of LLMs and you just want the machine to spit out some end result while putting in minimal time and effort, then yes, absolutely don't waste your time with smaller, less capable models.
Even the big online models have very specific styles and preferences for similar tasks. You can easily test this by giving them all some generic task without too many limits, each of them will gravitate towards a different solution to the same problem.
There’s simply so much going on under the hood at these LLM providers that are very hard to replicate locally.
Just like every other subscription model, including the one in the Black Mirror episode, Common People. The value is too good to be true for the price at the beginning. But you become their prisoner in the long run, with increasing prices and degrading quality.
I think the only people worried about lock-in or Black Mirror themes are the people who are thinking about these subscriptions in an abstract sense.
It’s really easy to change providers. They’re all improving. Competition is intense.
People are reasonably worried that the same will happen to AI.
It absolutely does not.
Your use of social network derives value from your network. If you switch, you have to convince everyone else to switch as well.
It's a tremendous barrier to switching.
LLMs are for the most part interchangeable commodity.
One option is to add adverts to the generated output, and making the product cheaper than the competition. Another is to have all your cloud data preprocessed with LLMs in a non-portable way, so that changing will incur a huge cost.
As of now, specifically for coding assistance LLM, workflows remain generic enough to have relatively low switching costs between the different models.
Netflix/Hulu were "losing money on streaming"-level cheap.
Uber was "losing money on rides"-level cheap.
WeWork was "losing money on real-estate" level cheap.
Until someone releases wildly profitable LLM company financials it's reasonable to expect prices to go up in the future.
Course, advances in compute are much more reasonable to expect than advances in cheap media production, taxi driver availability, or office space. So there's a possibility it could be different. But that might require capabilities to hit a hard plateau so that the compute can keep up. And that might make it hard to justify the valuations some of these companies have... which could also lead to price hikes.
But I'm not as worried as others. None of these have lock-in. If the prices go up, I'm happy to cancel or stop using it.
For a current student or new grad who has only ever used the LLM tools, this could be a rougher transition...
Another thing that would change the calculation is if it becomes impossible to maintain large production-level systems competitively without these tools. That's presumably one of the things the companies are betting on. We'll see if they get there. At that point many of us probably have far bigger things to worry about.
Eventually, these things should get closer. Eventually the hosted solutions have to make money. Then we’ll see if the costs of securing everything and paying some tech company CEO’s wage are higher than the benefits of centrally locating the inference machines. I expect local running will win, but the future is a mystery.
Locally I need to pay for my GPU hardware 24x7. Some electricity but mostly going to be hardware cost at my scale (plus I have excess free energy to burn).
Remotely I probably use less than an hour of compute a day. And only workdays.
Combined with batching being computationally more efficient it’s hard to see anything other than local inference ALWAYS being 10x more expensive than data centre inference.
(Would hope and love to be proven wrong about this as it plays out - but that’s the way I see it now).
They will. And when they do it will hit hard, especially if you’re not just a consumer but relying on it for work.
One vector is personalization. Your LLM gets to know you and your history. They will not release that to a different company.
Another is integrations. Perhaps you’re using LLMs for assistance, but only Gemini has access to your calendar.
Cloud used to be ”rent a server”. You could do it anywhere, but AWS was good & cheap. Now how is is it to migrate? Can you even afford the egress? How easy is it to combine offerings from different cloud providers?
259. Anthropic tightens usage limits for Claude Code without telling users (techcrunch.com)
395 points by mfiguiere 2 days ago | hide | 249 comments
It isn’t specific to software/subscriptions but there are plenty of examples of quality degradation in the comments
What I think is more likely is people will realize that every line of code written is, to an extent, a liability, and generating massive amounts of sloppy insecure poorly performing code is a massive liability.
That's not to say that AI's will go away, obviously, but I think when the hype dies down and people get more accustomed to what these things can and can't do well we'll have a more nuanced view of where these things should be applied.
I suppose what's still not obvious to me is what happens if the investment money dries up. OpenAI and Anthropic, as far as I know, aren't anywhere near profitable and they require record breaking amounts of capital to come in just to sustain what they have. If what we currently see is the limit of what LLM's and other generative techniques can do, then I can't see that capital seeing a good return on its investment. If that's the case, I wonder if when the bubble bursts these things become massively more expensive to use, or get taken out of products entirely. (I won't be sad to see all the invasive Copilot buttons disappear..)
If the gains plateau, then there's really no need to make productivity sacrifices here for the societal good, because there's so much competition, and various levels of open models that aren't far behind, that there will be no reason to stick with a hostile and expensive service unless it's tooling stays leaps ahead of the competition.
If the gains don't plateau, well then we're obsolete anyways, and will need to pivot to... something?
So I sympathize, but pragmatically I don't think there's much point in stressing it. I also suspect the plateau is coming and that the stock of these big players is massively overvalued.
I think there's room for more nuance here. It could also be a situation of diminishing returns but not a sharp plateau. That could favour the big players. I think I find that scenario most likely, at least in between major breakthroughs.
Search engine tech isn't that much of a secret nowadays? Still it's prohibitively expensive for almost everyone to build a competitive search engine. What if really good AI turns out to be more like that (both training and inference)
For AI, I think that ship has sailed already. OpenAI is the closest to dominance, but not currently the best at all tasks (claude and gemini for some tasks), and everyone else is nipping at their heels, followed by open / cheap models anywhere from 6 months to 1 year behind. Maybe I'm wrong, but as far as I can tell, all evidence points to it becoming a commodity (or utility), similar to cloud computing.
For example AWS is pretty dominant in the cloud computing space, but the differences aren't really that big of a deal for most people and there are services that will extract the cloud for you as a generic services layer. Like OpenRouter does for AI models. Is AWS really better than other cloud providers? Maybe, in some situations, with some requirements, but it's definitely not the general rule. Their focus since the beginning has been layering value on top of this base cloud offering (I'd argue to the point where there are so many services it's confusing AF), and I think it's the same with AI providers. It's even the same players for a lot of it (Google Cloud, AWS, Azure all offer AI services).
Yet JetBrains has been a business longer than some of my colleagues have been alive, and Microsoft’s Visual Basic/C++/Studio made writing software for Windows much easier, and did not come cheap.
How is that any different than JetBrains versus vim?
Calling LLMs a strong dependency or a lock-in also doesn’t make sense. It’s so easy to switch LLMs or even toggle between them within something like Copilot.
You can also just not use them and write the code manually, which is something you still do in any non-trivial app.
I don’t understand all of these people talking about strong dependencies or vendor lock in, unless those comments are coming from people who haven’t actually used the tools?
Can you make vim work roughly the same way? Probably you can get pretty close. But how many hours do I have to sink into the config? A lot. And suddenly the PyCharm license is cheap.
And it's exactly the same thing with LLMs. You want hand crafted beautiful code, untainted by AI? You can still do that. But I'm paid to solve problems. I can solve them faster/solve more of them? I get more money.
The reason I don't like those arguments is that they merge two orthogonal stuff: Solving problems and optimizing your tooling. You can optimize PyCharm just as much you can fiddle with Vim's config. And people are solving with problems with Vim just as you do with an IDE. It's just a matter of preference.
In my day job, I have two IDEs, VSCode, and Emacs open. I prefer Emacs to edit and git usage, but there's a few things that only the IDEs can do (as in I don't bother setting emacs to do the same), and VSCode is there because people get dizzy with the way I switch buffers in Emacs.
But you’re ignoring that the “optimizing tooling” is for the goal of making it easier for you. Its spending time now to decrease time spent in the long term.
I spent over a decade with Emacs as my sole editor and have since spent over a decade with PyCharm. Day 1 of PyCharm already had practically everything that it took a decade to get working for Emacs, and more. It was pre-optimized for me, so I was able to spend more time working on my code. Did I need to spend time optimizing Emacs? No. But doing so added intellisense and the ability to jump around the codebase very quickly. You _can_ spend just as much time optimizing Emacs, but I didn’t _have_ to in order to get the same result. Or have I spent that much time optimizing it since, for even more functionality.
I can tell you in my case it was because I did want to play with emacs and get my hands dirty. But that does shift the blame to me since it's hardly fair to blame emacs for being so extensible and fun.
What you may need is something others can do without. So what’s best is always subjective.
I think it’s odd for you to criticize someone’s reasons when you apparently just reject their underlying premise to begin with (which is, effectively, my experience with a particular open source tool isn’t the best for me and the one I found that does isn’t open source).
Frankly, the idea of having 4 different ways of editing code, as you’ve described above, just seems like a nightmare to me. I like to have one tool that does it all and learn to use that tool well. The Jetbrains tooling allows me to do this with little of my time spent configuring it.
If what you are using works well for you, then I think we should all be happy here.
Everyone works differently. For most tasks, I prefer Emacs. But for pairing, VS code may be more familiar to others. And it’s hard to get rid of Android Studio and XCode if you’re doing mobile development.
No one can judge workflow and tooling choices as long as the results are in.
You won’t be able to switch to a meaningful vim if you channel your support to closed source software, not for long.
Best to put money where the mouth is.
The alternative is to restrict yourself to “not as good” ones already now.
And how they don't mind freely opening up their codebase to these bigtech companies.
And how they don't mind opening up their development machines to agents driven by a black-box program that is run in the cloud by a vendor that itself doesn't completely understand how it works.
Also, many of my personal projects are open sourced with pretty permissive licensing, so I’m not all that mad someone else has the code.
As of now, I’m seeing no lock-in for any LLM. With tools like Aider, Cursor, etc., you can swim on a whim. And with Aider, I do.
That’s what I currently don’t get in terms of investment. Companies (in many instances, VCs) are spending billions of dollars and tomorrow someone else eats their lunch. They are going to need to determine that method of lock-in at some point, but I don’t see it happening with the way I use the tools.
What you're describing is really just called "Offering a product for sale" and yes typically the people doing it will do, say, and offer things that encourage using their product over the competitors. That isn't "lock in" in any sense of the word. What are they supposed to do? Say "Our shit sucks and isn't price effective compared to others and we bring nothing to the table?" while giving you stuff for free?
Even if they make their tooling cheaper, that’s not going to lock me in. It has to be the best model or have some killer feature. Which, again, could be usurped the next day with the rate these tools are advancing.
I don’t understand how people consider this a strong dependency.
Changing LLMs is trivial. Some times I’ll switch between LLMs on a whim to experiment. I can close one coding agent app and open another in seconds.
These claims about vendor lock-in and strong dependencies seem to mostly be coming from people watching from a distance, not the people on the ground using these tools.
That problem does not even include lock-in, surveillance, IP theft and all other things that come with SaaS.
Why?
If I want to pick up many hobbies, not to mention lines of professional work, I have to pay for tools. Why is programming any different? Why should it be?
Complaining that tools that improve your life cost money is... weird, IMO. What's the alternative? A world in which people gift you life-and-work-improving tools for free? For no reason? That doesn't exist.
> Programming used to be (and still is, to a large extent) an activity that can be done with open and free tools.
Btw, I think this was actually less true in the past. Compilers used to cost money in the 70s/80s. I think it's actually cyclical - most likely tools will cost money today, but then down the line, things will start getting cheaper again until they're free.
I don't see it as an inevitable cycle. Free tools (gcc, emacs) and OS (Linux) came about as idealistic efforts driven by hobbyists, with no profit goal/expectation, then improved as large companies moved to support them out self-interest. Companies like RedHat have then managed to make a business model out of selling support for free software.
Free LLMs or other AI-based tools are only going to happen where there are similar altruistic and/or commercial interests at play, and of course the dynamics are very different given the massive development costs of LLMs. It's not a given that SOTA free tools will emerge unless it is the interest of some deep-pocketed entity to make that happen. Perhaps Meta will develop and release SOTA models, but then someone would have to host them which is also expensive. What would the incentive be for someone to host them for free or at-cost?
Local models are not quite there yet. For now, use the evil bad tools to prepare for the good free tools when they do get there. It's a self-correcting form of technical debt that we will never have to pay down.
Why do I have to prepare? Once the good free tools are available, it should just work no?
As an Emacs user, I anticipate further programming to refine how the glove fits the hand. The solutions we will want as individuals have a lot of dispersion, so I cannot rely on others for everything.
There are plenty of times where I download what others have written and use it as it is, within the bounds of some switches and knobs. Do you want to have such a hands off approach with your primary interface to a computer?
But the price we're paying (and I don't mean money) is very high, imho. We all talk about how good engineers write code that depends on high-level abstractions instead of low-level details, allowing us to replace third party dependencies easily and test our apps more effectively, keeping the core of our domain "pure". Well, isn't it time we started doing the same with LLMs? I'm not talking about MCP, but rather an open source tool that can plug into either free and open source LLMs or private ones. That would at least allow us to switch to a free and opensource version if the companies behind the private LLMs go rogue. I'm afraid tho that wouldn't be enough, but it's a starting point.
To put an example: what would you think if you need to pay for every single Linux process in your machine? Or for every Git commit you make? Or for every debugging session you perform?
Fortunately there are many of these that can integrate with offline LLMs through systems like LiteLLM/Ollama/etc. Off the top of my head, I'd look into Continue, Cline and Aider.
https://github.com/continuedev/continue
There are open source tools that do exactly that already.
But every single post I read don't mention them?
Does every single post about a Jetbrains feature mention that you can easily switch from Jetbrains to an open source editor like VS Code or vim?
Because the models are so much worse that people aren't using them.
Philosophical battles don't pay the bills and for most of us they aren't fun.
There have been periods of my life where I stubbornly persisted using something inferior for various reasons - maybe I was passionate about it, maybe I wanted it to exist and was willing to spend my time debugging and offer feedback - but there a finite number of hours in my life and often I'd much rather pay for something that works well than throw my heart, soul, time, and blood pressure at something that will only give me pain.
I have been building that for a couple of years now: https://llm.datasette.io
Has someone computed/estimated what is at cost $$$ value of utilizing these models at full tilt: several messages per minute and at least 500,000 token context windows? What we need is a wikipedia like effort to support something truly open and continually improving in its quality.
Almost all providers and models can be used with the OpenAI api and swapping between them is trivial.
It just so happens that closed models are better today.
Code written will continue to run, you can use alternate LLM to further write code. Unless you are a pure vibe coder, you can still type the code. IF programmer stop learning how to write code, that's on them and not Claude's or antirez responsibility.
We are using best tool available to us, right now it is Claude Code, tomorrow it may be something from OpenAI, Meta or Deepseek.
But that is my personal value judgement. And it doesn't mean other people will think the same. Luckily tech is a big space.
This was largely not the case before 2000, and may not be the case after, say 2030. It may well be that we are living in extraordinary times.
Before 2000 one had to buy expensive compilers and hardware to do serious programming. The rise of the commercial internet has made desktop computers cheaper, and the free software movement (again mostly by means of commercial companies enjoying the benefits) has made a lot of free software available.
However, we now have mostly smartphones, and serious security risks to deal with.
We can use cheap hardware that can be fixed with soldering irons and oscilloscopes.
People said for decades our projects just become weird DSLs. And now whatever little thing I want to do in any mainstream language involves learning some weird library DSL.
And now people be needing 24h GPU farm access to handle code.
In 50 years my grandkids that wish to will be able to build, repair and program computers with a garage workbench and old wrinkled books. I know most of the software economy will end up in the hands of major corporations capable of paying through the nose for black box low code solutions.
Doesn't matter. Knowledge will set you free if you know where to look.
My general thinking is that we're using graphics processors which _work_ but aren't really designed for AI (lack of memory, etc.).
Anecdotally, I recently find myself building a simple enough Android app for a client based on an iOS app I have already built.
I don’t really know Android dev, but know enough to get started having patched an App somewhat recently.
So started from scratch and having heard that the Android SDK had something similar to SwiftUI that’s what I went for.
In building the App I have found that Gemini is far more useful than Googles own documentation and tutorials.
The tutorials for basic things like NavBars is just wrong in places, maybe due to being outdated.
I’ve reported issues using the feedback tool. But still….
What I thought you were going to say is “Gemini- wha?”.
I’ve used Gemini 2.5 PRO and would definitely not use it for most of my coding tasks. Yes, it’s better at hard things, but it’s not great at normal things.
I’ve not used Claude 4 Opus yet- I know it’s great at large context- but Claude 4 Sonnet Thinking is mostly good unless the task is too complex, and Claude 4 Sonnet is good for basic operations in 1-2 files- beyond that it’s challenged and makes awful mistakes.
Feels like this is a byproduct of a poor work-life balance more than an intrinsic issue with programming itself. I also can't really relate since I've always enjoyed discussing challenging problems with colleagues.
I'm assuming by "die" you mean some future where autonomous agentic models handle all the work. In this world, where you can delete your entire programming staff and have a single PM who tells the models what features to implement next, where do you imagine you fit in?
I just hope for your sake that you have a fallback set of viable skills to survive in this theoretical future.
I've been programming professionally since 2012 and still love it. To me the sweet spot must've been the early mid 2000s, with good enough search engines and ample documentation online.
If you feel it is stealing your life, then please feel free to reclaim your life at any time.
Leave the programming to those of us who actually want to do it. We don't want you to be a part of it either
> [...] get lost [...]
> [..] We don't want you to be a part of it either. [...]
He's being rude.
Honesty would be, something like:
> I (and probably many others) like programming a lot. Even if you're frustrated with it, I think a great deal of people will be sad if somehow programming disappeared completely. It might be best for you if you just found a job that you love more, instead.
Also the original comment makes a point that's SUPER valid and anyone working as a professional programmer for 10+ years can't really deny:
> poorly if at all maintained libraries, tools, frameworks
Most commercial code just sucks due to misaligned incentives. Open Source is better, but not always, as a lot of Open Source code is just commercial code opened up after the fact.
Sure it does. Reads incredibly honestly to me.
Responding to the original comment with 'get lost' and 'we don't want you either' is not constructive in my opinion.
People who are cheering for LLM coding because they hate actually coding themselves are cheering for programmers to lose their livelihoods
I am not going to be polite and constructive to people who don't care if my livelihood is destroyed by their new tools. Why should I? They are cheering for my ruin
It's not like OP is Sam Altman and one of the actual AI bros. This thing will happen or not happen regardless of what OP wants.
Amusingly, you've committed the same basic mistake as LLMs, which just don't know when to shut up :-p
Rudeness being considered a <<good>> rhetorical choice reflects poorly on its source's rhetorical prowess.
I understand your frustration but the problem is mostly people. Not the particular skill itself.
I hope this will change before (captured) regulation strangles open models.
I wish the stuff on top - deep research modes, multimodal, voice etc. had a more unified API as well but there's very little lock-in right now.
Apple's fee is like that Visual Studio purchase I made, a fee that lets me compile for their platforms. It's not a subscription without which I can't code anything at all.
Creating a new dependency on monthly subscriptions to unsustainable companies or products is a huge step away from accessible programming of the last 50 years and one that should not so casually be dismissed.
Not without a lot of hard thankless work by people like RMS to write said tools. Programming for a long while was the purview of Microsoft Visual Studio family, which cost hundreds, if not thousands of dollars. There existed other options, some of which was free, but, as is the case today with LLMs you can run at home, they were often worse.
This is why making software developer tools is such a tough market and why debugging remains basically in the dark ages (though there are the occasional bright lights like rr). Good quality tools are expensive, for doctors and mechanics, why do we as software developers expect ours to be free, libre and gratis?
And once the tool has been made, many people just give it away for others to benefit from too.
There is a 0% chance that I'm going to subscribe to being able to program, because its actively a terrible idea. You have to be very naïve to think that any of these companies are still going to be around and supporting your tools in 10-20 years time, so if you get proficient with them you're absolutely screwed
I've seen people say that AI agents are great because instead of using git directly, they can ask their AI agent to do it. Which would be fine if it was a free tool, but you're subscribing to the ability to even start and maintain projects
A lot of people are about to learn an extremely blunt lesson about capitalism
I've been spending time with my team, just a few hours a week, on training them on foundational things, vs every other team in the company just plodding along, trying to do things the same way they always have, which already wasn't working. It's gotten to where my small team of 4 is getting called in to clean up after these much larger teams fail to deliver. I'm pretty proud of my little junior devs.
Yeah, coding (and to a lesser extent IT in general) at one point was a real meritocracy, where skill mattered more than expensive/unnecessary academic pedigree. Not perfect of course, but real nevertheless. And coders were the first engineers who really said "I won't be renting a suit for an interview, I think an old t-shirt is fine" and we normalized that. Part of this was just uncompromisingly practical.. like you can either do the work or not, and fuck the rest of that noise. But there was also a pretty punk aspect to this for many people in the industry.. some recognition that needing to have money to make money was a bullshit relic of closeted classism.
But we're fast approaching a time where both the old metrics (how much quality code are you writing how fast and what's your personal open source portfolio like?) and the new metrics (are you writing a blog post every week about your experience with the new models, is your personal computer fast enough to even try to run crappy local models?) are both going to favor those with plenty of money to experiment.
It's not hard to see how this will make inequality worse and disadvantage junior devs, or just talented people that didn't plan other life-events around purchasing API credits/GPUs. A pay-to-play kind of world was ugly enough in politics and business so it sucks a lot to see it creeping into engineering disciplines but it seems inevitable. If paying for tokens/GPU ever allows you to purchase work or promotion by proxy, we're right back to this type of thing https://en.wikipedia.org/wiki/Purchase_of_commissions_in_the...
You have very powerful open weight models, they are not the cutting edge. Even those you can't really run locally, so you'd have to pay a 3rd party to run it.
Also the competition is awesome to see, these companies are all trying hard to get customers and build the best model and driving prices down, and giving you options. No one company has all of the power, its great to see capitalism working.
Or Windows?
Or an Apple Developer License?
There are free/free-ish, options, but there have always been paid tools.
Further, I can write and compile an application for Mac, Windows, or Linux with entirely free tools including tools provided directly by Apple and Microsoft.
This discussion is about is a monthly subscription, without which, most up and coming coders, certainly "vibe coders," are completely dead in the water.
These two dependencies are not the same. If that's not obvious to you, I don't know what else to say to that.
There is no dependency at all.
A cabbie is still dependent on the car if they can still drive a horse buggy and order of magnitude slower.
You are only not dependent if you are not doing this professionally and can enjoy the leisure of tending to your horses.
That's why I drink the whole tools kool-aid. From TFA:
> In this historical moment, LLMs are good amplifiers and bad one-man-band workers.
That's how I use them: write a function here, explain an error message there. I'm still in control.
I don't depend on LLMs: they just amplify.
I can pull the plug immediately and I'm still able to code, as I was two years ago.
Shall DeepSeek release a free SOTA model? I'll then use that model locally.
It's not because I use LLMs that I have a strong dependency on them.
Just like I was already using JetBrains' IntelliJ IDEA back when many here were still kids (and, yup, it was lightyears better than NetBeans and Eclipse) didn't make me have a strong dependency on JetBrains tools.
I'm back to Emacs and life is good: JetBrains IDEs didn't make me forget how to code, just as LLMs won't.
They're just throwaway tools and are to be regarded as such.
I added a "disclosures" section to my own site recently, in case you're interested: https://simonwillison.net/about/#disclosures
To be fair, change is not always good. We still haven't fixed fitness/obesity issues caused (partly) by the invention of the car, 150 years later. I think there's a decent chance LLMs will have the same effect on the brain.
Since when? It starts with computers, the main tool and it's architecture not being free and goes from there. Major compilers used to not be free. Major IDEs used to not be free. For most things there were decent and (sometimes) superior free alternatives. The same is true for LLMs.
> The excuse "but you earn six figures, what' $200/month to you?" doesn't really capture the issue here.
That "excuse" could exactly capture the issue. It does not, because you chose to make it a weirder issue. Just as before: You will be free to either not use LLMs, or use open-source LLMs, or use paid LLMs. Just as before in the many categories that pertain to programming. It all comes at a cost, that you might be willing to pay and somebody else is free to really does not care that much about.
There were and are a lot of non-free ones, but since the 1990s, GCC and interpreted languages and Linux and Emacs and Eclipse and a bunch of kinda-IDEs were all free, and now VS Code is one of the highest marketshare IDEs, and those are all free. Also, the most used and learned programming language is JS, which doesn't need compilers in the first place.
And my point is that's simply not the case. Different products have always been not free, and continue to be not free. Recent example would be something like Unity, that is not entirely free, but has competitors, which are entirely free and open source. JetBrain is something someone else brought up.
Again: You have local LLMs and I have every expectation they will improve. What exactly are we complaining about? That people continue to build products that are not free and, gasp, other people will pay for them, as they always have?
There's never been anything stopping you from building your own
Soon there will be. The knowledge of how to do so will be locked behind LLMs, and other sources of knowledge will be rarer and harder to find as a result of everything switching to LLM use
How are LLMs equivalent? People posting their prompts on bulletin boards at cafes?
This phrasing can be misleading and points to a broader misunderstanding about the nature of doctoral studies, which it has been influenced by the marketing and hype discourse surrounding AI labs.
The assertion that there is a defined "PhD-level knowledge" is pretty useless. The primary purpose of a PhD is not simply to acquire a vast amount of pre-existing knowledge, but rather to learn how to conduct research.
What sticks out to me is Gemini catching bugs before production release, was hoping you’d give a little more insight into that.
Reason being is that we expect ai to create bugs and we catch them, but if Gemini is spotting bugs by some way of it being a QA (not just by writing and passing tests) then that perks my interest.
Another area we've started doing is having it look at build failures and writing a report on suggested root causes before even a human looks at it - saves time.
Or (and we haven't rolled this out automatically yet but are testing a prototype) having it triage alarms from our metrics, with access to the logs and codebase to investigate.
I feel many also try "review and fix automatically", as it's tempting to "just" pass the generated comments to a second agent to apply them.
But that opens a whole other can of worms and pretty soon you're just another code assistant service.
Further, I always assumed PhD level of knowledge meant coming up with the right questions. I would say it is at best a "Lazy Knowledge Rich worker", it won't explore hypothesis if you don't *ask it* to. A PHD would ask those questions to *themselves*. Let me give you a simple example:
The other day Claude Code(Max Pro Subscription) commented out a bunch of test assertions as a part of a related but separate test suite it was coding. It did not care to explore — what was a serious bug — why it was commenting it out because of a faulty assumption in the original plan. I had to ask it to change the plan by doing the ultra-think, think-hard trick to explore why it was failing, amend the plan and fix it.
The bug was the ORM object had null values because it was not refreshed after the commit and was fetched before by another DB session that had since been closed.*
I use Claude Code with Opus, and had same experience - was pushing it hard to implement complex test, and it gave me an empty test function with test plan inside in a comment (lol).
I do want to try Gemini 2.5 Pro, but I don't know a tool which would make experience compatible to Claude Code. Would it make sense to use with Cursor? Do they try to limit context?
~/.nvm/versions/node/v22.16.0/lib/node_modules/@anthropic-ai/claude-code $ npx prettier cli.js | ack ultrathink -C 20
var jw1 = { HIGHEST: 31999, MIDDLE: 1e4, BASIC: 4000, NONE: 0 },
Yk6 = {
english: {
HIGHEST: [
{ pattern: "think harder", needsWordBoundary: !0 },
{ pattern: "think intensely", needsWordBoundary: !0 },
{ pattern: "think longer", needsWordBoundary: !0 },
{ pattern: "think really hard", needsWordBoundary: !0 },
{ pattern: "think super hard", needsWordBoundary: !0 },
{ pattern: "think very hard", needsWordBoundary: !0 },
{ pattern: "ultrathink", needsWordBoundary: !0 },
],
MIDDLE: [
{ pattern: "think about it", needsWordBoundary: !0 },
{ pattern: "think a lot", needsWordBoundary: !0 },
{ pattern: "think deeply", needsWordBoundary: !0 },
{ pattern: "think hard", needsWordBoundary: !0 },
{ pattern: "think more", needsWordBoundary: !0 },
{ pattern: "megathink", needsWordBoundary: !0 },
],
BASIC: [{ pattern: "think", needsWordBoundary: !0 }],
NONE: [],
},Anyway I don't think this is ""PhD-knowledge"" questions, but job related electrical engineering questions.
It’s not like once you have a PhD anyone cares about the subject, right? The only thing that matters is that you learnt to conduct research.
IMHO those two variables are 10x (maybe 100x) more explanatory than any vibe coding setup one can concoct.
Anyone who is befuddled by how the other person {loves, hates} using LLMs to code should ask what kind of problem they are working on and then try to tackle the same problem with AI to get a better sense for their perspective.
Until then, every one of these threads will have dozens of messages saying variations of "you're just not using it right" and "I tried and it sucks", which at this point are just noise, not signal.
After all the effort getting to the point where the generated code is acceptable, one has to wonder, why not just write it yourself? The time spent typing is trivial to all the cognitive effort involved in describing the problem, and describing the problem in a rigorous way is the essence of programming.
Because it is still way, way, way faster and easier. You're absolutely right that the hard part is figuring out the solution. But the time spent typing is in no way trivial or cognitively simple, especially for more complex tasks. A single prompt can easily generate 5 - 10x the amount of code in a few seconds, with the added bonus that it:
a) figures out almost all the intermediate data structures, classes, algorithms and database queries;
b) takes care of all the boilerplate and documentation;
c) frequently accounts for edge cases I hadn't considered, saving unquantifiable amounts of future debugging time;
d) and can include tests if I simply ask it to.
In fact, these days once I have the solution figured out I find it frustrating that I can't get the design in my head into the code fast enough manually. It is very satisfying to have the AI churn out reams of code, and immediately run it (or the tests) to see the expected result. Of course, I review the diff closely before committing, but then I do that for any code in any case, even my own.
And creates new ones you wouldn't even consider before, creating just as much, if not more future debugging :D
That said, I've adopted a specific way of working with AI that is very effective for my situation (mentioned in my comment history, but echoes a lot of what TFA advises.)
This has been a surprise to me, as I expected code review of AI-generated code to be much more difficult than it has been in practice. Maybe this has been because I only really use LLMs to write code that is easy to explain, and therefore probably not that complicated. If code is more complicated, then I will write it myself.
At the most generous I will allow, it's an illusion where producing 20-50x the amount of code/hour introduces a higher raw count of bugs relative to what you are used to in that timeframe - but this notion of AI coders being more bugprone than humans is utter nonsense. The only way thats not true is on very niche systems or when the human has conceptualized and planned out their code extensively beforehand - in which case AI would still be the superior next step.
Are these very niche? Yeah, there is a category of coding where what you are doing is essentially translating from English to some high-level language with high-level APIs. This is not significantly different than translating to Spanish, of course LLMs will be successful here.
But there are endless other domains with complex reasoning where LLMs absolutely suck. Like please tell me how will an LLM reason about concurrent access. And prompting it so that it will reply with "Oh you are right, here is Atomic blahblah" is not reasoning, it's statistical nonsense.
Don't get me wrong, I do think LLMs are a very useful tool, but it is as much overhyped by some as it is underhyped by others.
Here's a recent ShowHN post (a map view for OneDrive photos), which documents all the LLM prompting that went into it:
You can get images that can go straight to print or end up with pure slop based on your prompt
You know, I would often ask myself that very question...
Then I discovered the stupid robots are good at designing a project, you ask them to produce a design document, argue over it with them for a while, make revision and changes, explore new ideas, then, finally, ask them to produce the code. It's like being able to interact with the yaks you're trying to shave, what's not to love about that?
When the change is very small, self-contained feature/refactor it can mostly work alone, if you have tests that cover the feature then it is relatively safe (and you can do other stuff because it is running in an action, which is a big plus...write the issue and you are done, sometimes I have had Claude write the issue too).
When it gets to a more medium size, it will often produce something that will appear to work but actually doesn't. Maybe I don't have test coverage and it is my fault but it will do this the majority of the time. I have tried writing the issue myself, adding more info to claude.md, letting claude write the issue so it is a language it understands but nothing works, and it is quite frustrating because you spend time on the review and then see something wrong.
And anything bigger, unsurprisingly, it doesn't do well.
PR reviews are good for small/medium tasks too. Bar is lower here though, much is useless but it does catch things I have missed.
So, imo, still quite a way from being able to do things independently. For small tasks, I just get Claude to write the issue, and wait for the PR...that is great. For medium (which is most tasks), I don't need to do much actual coding, just directing Claude...but that means my productivity is still way up.
I did try Gemini but I found that when you let it off the leash and accept all edits, it would go wild. We have Copilot at work reviewing PRs, and it isn't so great. Maybe Gemini better on large codebases where, I assume, Claude will struggle.
Glad to see my experience is reflected elsewhere. I've found Gemini 2.5 PRO to be the best bang-for-buck model: good reasoning and really cheap to run (counts as 1 request in cursor, where opus can blow my quotas out of the water). Code style works well for me too, its "basic" but thats what I want. If I have only one model to take to my deserted island this is the one I'd use right now.
For the heady stuff, I usually use o3 (but only to debug, its coding style is a bit weird for me), saving Opus 4 for when I need the "big guns".
I don't have Claude Code (cursor user for now), but if I did I'd probably use Opus more.
https://x.com/pashmerepat/status/1946392456456732758/photo/1
Another contender in the "big idea" reasoning camp: DeepSeek R1. It's much slower, but most of the time it can analyze problems and get to the correct solution in one shot.
Frontier LLMs are easiest to work with for now. Open models _will_ catch up. We can be excited for that future.
You are able to learn things from LLMs, and you can ask them for recommendations for an approach to implement something. Or just tell your LLM the approach to take. Sometimes it overcomplicates things. You’ll develop an instinct for when that’s likely. You can head off the overcomplication ahead of time or ask for refactorings after the initial cut is built for you. After a while you get an instinct for which way will get the work done soonest. Most fascinating of all, it’ll all change again with the next round of frontier models.
You don’t need frontier models for every task. For instance I’ve been positively surprised by Github Copilot for straightforward features and fixes. When it’s obvious how to implement, and you won’t need to go back and forth to nail down finer design details, getting an initial PR from Copilot is a great starting place.
To everyone starting out, enjoy the ride, know that none of us know what we’re doing, and share what you learn along the way!
> more/better in the same time used in the past — which is what I do), when left alone with nontrivial goals they tend to produce fragile code bases that are larger than needed, complex, full of local minima choices, suboptimal in many ways.
And this part felt like a "bitter lesson" anti-pattern:
> Avoid any RAG that will show only part of the code / context to the LLM. This destroys LLMs performance. You must be in control of what the LLM can see when providing a reply.
Ultimately I think cli agents like claude-code and gemini-cli and aider will be controlling the context dynamically, and the human should not be spending premature optimization time on this activity.
If anyone's interested I've got some very exact stats on prompts and accepted solution linked in my LLM proof of concept repo: https://github.com/sutt/agro/blob/master/docs/dev-summary-v1...
I find it very sad that people who have been really productive without "AI" now go out of their way to find small anecdotal evidence for "AI".
If the AI is doing the coding then that is a threat to some people. I am not sure why, LLMs can be good and you can enjoy coding...those things are unrelated. The logic seems to be that if LLMs are good then coding is less fun, lol.
Consider cilantro. I’m happy to admit there are people out there who don’t like cilantro. But it’s like the people who don’t like cilantro are inventing increasingly absurd conspiracy theories (“Redis is going to add AI features to get a higher valuation”) to support their viewpoint, rather than the much simpler “some people like a thing I don’t like”.
https://metr.org/blog/2025-07-10-early-2025-ai-experienced-o...
Personally, I find the current tools don’t work great for large existing codebases and complex tasks. But I’ve found they can help me quickly make small scripts to save me time.
I know, it’s not the most glamorous application, but it’s what I find useful today. And I have confidence the tools will continue to improve. They hardly even existed a few years ago.
> AI systems do not currently speed up many or most software developers
> We do not claim that our developers or repositories represent a majority or plurality of software development work
Not to mention in the study less than 1/2 have ever used it before the study.
There's an entire class of investment scammers that string along their marks, claiming that the big payoff is just around corner while they fleece the victim with the death of a thousand cuts.
But I would stake my very life on the fact that the movement by developers we call open-source is the single greatest community and ethos humanity has ever created.
Of course it inherits from enlightenment and other thinking, it doesn't exist in a vacuum, but it is an extension of the ideologies that came before it.
I challenge anyone to come up with any single modern subcultures that has tangibly generated more that touches more lives, moves more weight, travels farther, effects humanity more every single day from the moment they wake up than the open source software community (in the catholic sense obviously).
Both in moral goodness and in measurable improvement in standard of living and understanding of the universe.
Some people's memories are very short indeed, all who pine pine for who they imagined they were and are consumed by a memetic desire of their imagined selves.
good lord.
people who believe in open source don't believe that knowledge should be secret. i have released a lot of open source myself, but i wouldn't consider myself a "true believer." even so, i strongly believe that all information about AI must be as open as possible, and i devote a fair amount of time to reverse engineering various proprietary AI implementations so that i can publish the details of how they work.
why? a couple of reasons:
1) software development is my profession, and i am not going to let anybody steal it from me, so preventing any entity from establishing a monopoly on IP in the space is important to me personally.
2) AI has some very serious geopolitical implications. this technology is more dangerous than the atomic bomb. allowing any one country to gain a monopoly on this technology would be extremely destabilizing to the existing global order, and must be prevented at all costs.
LLMs are very powerful, they will get more powerful, and we have not even scratched the surface yet in terms of fully utilizing them in applications. staying at the cutting edge of this technology, and making sure that the knowledge remains free, and is shared as widely as possible, is a natural evolution for people who share the open source ethos.
The "race against China" is a marketing trick to convince senators to pour billions into "AI". Here is who is financing the whole bubble to a large extent:
LLMs are useful—but there’s no way such an innovation should be a “guarded secret” even at this early stage.
It’s like saying spreadsheets should have remained a secret when they amplified what people could do when they became mainstream.
This post has nothing to do with Redis and is even a follow up to a post I wrote before rejoining the company.
Is author suggesting manually pasting redis C files into Gemini Pro chat window on the web?
The fundamental requirement for the LLM to be used is: don’t use agents or things like editor with integrated coding agents.
So right, is he like actually copying and pasting stuff into a chat window? I did this before Co-Pilot, but with cursor I would never think of doing that. He never mentioned Cursor or Claude Code so I wonder if he's even experienced it.
This is exactly why he's doing it the way he is and why what he describes is still the most effective, albeit labor intensive, way to work on hard/complex/difficult problems with LLMs.
Those tricks are for saving money. They don't make the LLM better at its task. They just make it so the LLM will do what you could/should be doing. We're using agents because we're lazy or don't have time or attention to devote, or the problems are trivial enough to solved with these "tricks" and added complexities. But, if you're trying to solve something complex or don't want to have a bunch of back and forth with the LLM or don't want to watch it iterate and do some dumb stuff... curate that context. Actually put thought and time into what you provide the LLM, both in context and in prompt - you may find that what you get is a completely different product.
Or, if you're just having it build views and buttons - keep vibing.
Now Cursor's agent mode does this for me, but it can be a hit or miss.
Before claude code was around I had already learned very quickly that exactly what antirez is describing, literally copy/pasting code into the chat verbatim, was _wildly_ effective. So much so that I had Claude build me a jetbrains plugin to streamline the process for me - I would select the files I wanted to include in context in a tree-style view of the project and it simply had a 'Copy' button that would, when pressed, compile the files into a single markdown document with simple metadata about each included file (filename, path) and a simple "divider" between files.
And, also like OP states, it's wildly more effective to take this approach combined with a new context/session/chat for each new task. (So, with Claude, I used projects heavily for this and this copied context markdown document was added to the project files and then I'd just start a new chat in that project any time I wanted to work on something that included them/that context I had crafted)
Claude Code and the "agent" approach is wildly effective, no doubt. But it doesn't really support this sort of very specific and "heavy" approach the the "problem". Rather - it's actually working pretty hard to use as few tokens as possible and weave through the maze of trying to be token efficient while getting/finding/using context that _is_ available to it but not presented directly to it. Rather - if you just feed it the whole ass class/module/package and provide a detailed, focused and intentful prompt you might be really surprised what you get.
These agents are _really_ going to thrive when the context window isn't a problem anymore. Claude (not code, but the desktop app, so using a "project") essentially one-shot implementing the necessary updates to phpredis (PHP extension in C for Redis) to support the new vector set data type. I first cloned the phpredis repo locally, initialized claude code and told it what I wanted to do and what the repo it was working with was and asked it to analyze the repo and the code and provide me a list of the files that would be essential for it to have in context when building out that new functionality. It provided me the list of files - I packed them up in a claude project and started over there. The only issues were related to it mixing up/assuming some stuff related to standard Redis commands and versions. And it's own tests caught that, fixed it. I only used this extension in my local/dev environments as I wanted to test the vector sets in Redis from within the same ecosystem my application was already working within (phpredis is significantly more performant that it's alternatives (which support the new commands) and it, also, serves as a perfect example of another of antirez's points... the utility provided by these tools to do things as "throwaway" proof of concepts is just amazing. I'm not a C developer, I didn't know anything about compiling php extensions (or how weird that whole ecosystem is in PHP-land) - I would have never gone through that effort to test just one method of doing something prior. I would have just used what was available rather than even contemplating doing something like forking an extension in a language I'm not familiar with and fixing it up the way I needed it to work (only because of time, I'm always down for a challenge, but I can't always afford one).
Context is king. Your ability to communicate to the box what you want is next in line for the throne.
Is there any good tooling for making this part easier and less error prone, short of going to a full-fledged agent system?
Copying and pasting code may work but when the agent is right there and can go read a file on its own I don't see the point in copying and pasting this way.
Really though, like another comment said, I'm probably working on different problems than antirez, so your mileage may vary.
"Don’t use agents or things like editor with integrated coding agents."
He argues that the copy/paste back and forth with the web UI is essential for maintaining control and providing the correct context.
I agree with this, but this is why I use a CLI. You can pipe files instead of copying and pasting.
Ex: Implementing a spec, responding to my review comments, adding wider unit tests, running a role play for usability testing, etc. The main time we do what he describes of manually copying into a web ide is occasionally for a better short use of a model, like only at the beginning of some plan generation, or debug from a bunch of context we have done manually. Like we recently solved some nasty GPU code race this way, using a careful mix of logs and distributed code. Most of our job is using Boring Tools to write Boring Code, even if the topic/area is neato: you do not want your codebase to work like an adventure for everything, so we invest in making it look boring.
I agree the other commenter said: I manage context as part of the skill, but by making the AI do it. Doing that by hand is like slowly handcoding assembly. Instead, I'm telling Claude Code to do it. Ex: Download and crawl some new dependency I'm using for some tricky topic, or read in my prompt template markdown for some task, or generate and self-maintain some plan.md with high-level rules on context I defined. This is the 80% case.
Maybe one of the disconnects is task latency vs throughput as trade-offs in human attention. If I need the LLM to get to the right answer faster, so the task is done faster, I have to lean in more. But my time is valuable and I have a lot to do. If rather spend 50% less of my time per task, even if the task takes 4x longer, by the LLM spinning longer. In that saved human time, I can be working on another task: I typically have 2-3 terminals running Claude, so I only check in every 5-15min.
We do this ~daily for:
* Multitier webapps
* DevOps infrastructure: docker, aws, ci systems, shell scripts, ...
* Analytics & data processing
* AI investigations (logs, SIEMs, ..) <--- what we sell!
* GPU kernels
* Compilers
* Docs
* Test amplification
* Spec writing
I think ~half the code happening by professional software engineers fits into these, or other vibes friendly domains. The stuff antirez does with databases seems close to what we do with compilers, GPU kernels, and infra.
We are still not happy with production-grade frontend side of coding, though by being strong on API-first design and keeping logic vs UI seperated, most of our UI code is friendly to headless.
I am on board to agree that pure LLM + pure original full code as context is the best path at the moment, but I’d love to be able to use some shortcuts like quickly applying changes, checkpoints, etc.
My persistent (and not unfounded?) worry is that all the major tools & plugins (Cursor, Cline/Roo) all play games with their own sub-prompts and context “efficiency”.
What’s the purest solution?
My augmented workflow is to “chat” with GPT because it’s free and powerful, to refine my ideas and surface things I hadn’t thought about. Then I start writing code to get in to the flow of things. I’ve found if I use the LLM straight away, I disengage, become lazy and lose context and understanding over the code. In these situations, I’ve had to redo the code more often than not. Lack of understanding is one part of why, but more importantly, disengaged prompting leads to vague and incorrect outcomes.
When I’m very clear in my head about my goal, I create a prompt either directly from my cursor, or if the changes are larger, I ask the LLM to not apply the changes but instead show them to me. I do both these things within the IDE in the chat window. I review the code and sometimes I’m happy applying it as is, other times I copy and paste it and tweak it manually.
I’ve got barebones rules set up; I haven’t felt the need to go overboard. Jetbrains Assistant does a good job of passing relevant context to the model.
I keep my prompts top-down (https://en.wikipedia.org/wiki/BLUF_(communication)) and explicit. Sometimes I’m much more detailed than others, and I’ve found that extra detail isn’t always necessary for a good result.
For each task I always start with a new (empty) context and manually tag relevant files to include (this is trivial since I know the codebase well).
First I use Claude 4 Sonnet in thinking mode (I could also use Gemini 2.5 Pro or Opus as per antirez' recommendations) to come up with a detailed plan on how to implement something (research/planning phase). I provide feedback and we iterate on the plan.
Then, in the same conversation I switch to Sonnet 4 non-thinking and tell it to implement what we just devised.
I manually review the changes and trst them. If something needs fixing or (more often) if I notice I missed some edge case/caveat, I tell it to do that (still same convo).
Commit, clear convo, next task.
For research that isn't directly tied to the code, I use ChatGPT or Claude (web apps) to brainstorm ideas, and sometimes copy/pasre these into the editor agent as starting point.
If your codebase fits in the context window, you can also just turn on "MAX" mode and it puts it all in the context for you.
It’s a far cry away from “vibe code everything”, “this will eliminate jobs” that the current hype train is pushing, despite clearly using the agentic approach with large context provieee by Opus.
You can use an LLM to help document a codebase, but it's still an arduous task because you do need to review and fix up the generated docs. It will make, sometimes glaring sometimes subtle, mistakes. And you want your documentation to provide accuracy rather than double down on or even introduce misunderstanding.
In future I'll go "In the name of our new darling bot, let us unit test and refactor this complicated thing".
----
Several codebases I've known have provided a three-stage pipeline: unit tests, integration tests, and e2e tests. Each of these batches of tests depend on the creation of one of three environments, and the code being tested is what ends up in those environments. If you're interested in a particular failing test, you can use the associated environment and just iterate on the failing test.
For humans with a bit of tribal knowledge about the project, humans who have already solved the get-my-dev-environment-set-up problem in more or less uniform way, this works ok. Humans are better at retaining context over weeks and months, whereas you have to spin up a new session with an LLM every few hours or so. So we've created environments for ourselves that we ignore most of the time, but that are too complex to be bite sized for an agent that comes on the scene as a blank slate every few hours. There are too few steps from blank-slate to production, and each of them is too large.
But if successively more complex environments can be built on each other in arbitrarily many steps, then we could achieve finer granularity. As a nix user, my mental model for this is function composition where the inputs and outputs are environments, but an analogous model would be layers in a docker files where you test each layer before building the one on top of it.
Instead of maybe three steps, there are eight or ten. The goal would be to have both whatever code builds the environment, and whatever code tests it, paired up into bite-sized chunks so that a failure in the pipeline points you a specific stage which is more specific that "the unit tests are failing". Ideally test coverage and implementation complexity get distributed uniformly across those stages.
Keeping the scope of the stages small maximizes the amount of your codebase that the LLM can ignore while it works. I have a flake output and nix devshell corresponding to each stage in the pipeline and I'm using pytest to mark tests based on which stage they should run in. So I run the agent from the devshell that corresponds with whichever stage is relevant at the moment, and I introduce it to onlythe tests and code that are relevant to that stage (the assumption being that all previous stages are known to be in good shape). Most of the time, it doesn't need to know that it's working stage 5 of 9, so it "feels" like a smaller codebase than it actually is.
If evidence emerges that I've engaged the LLM at the wrong stage, I abandon the session and start over at the right level (now 6 of 9 or somesuch).
It's complicated. You have to understand that when you ask an LLM something, you have the model itself, which is kind of like a function: put something in, get something out. However, you also pass an argument to that function: the context.
So, in a literal sense, no, they do not learn as they go, in the sense that the model, that function, is unchanged by what you send it. But the context can be modified. So, in some sense, an LLM in a agentic loop that goes and reads some code from GitHub can include that information in the context it uses in the future, so it will "learn" within the session.
> If the latter, what happens as less and less code gets written by human experts?
So, this is still a possible problem, because future trainings of future LLMs will end up being trained on code written by LLMs. If this is a problem or not is yet to be seen, I don't have a good handle on the debates in this area, personally.
Verification for code would be a formal proof, and these are hard; with a few exceptions like seL4, most code does not have any formal proof. Games like chess and go are much easier to verify. Math is in the middle; it also needs formal proofs, but most of math is doing these formal proofs themselves, and even then there are still unproven conjectures.
You can incorporate proofs with Coq or Dafny or use model checkers or TLA+ to actually verify your code.
This will be required for any software where correctness matters.
Something that was meant for humans, has now been consumed by AI and he is being repaid for that openness in a way. It comes full circle. Consistency, clarity and openness win again.
antirez is a big fuggin deal on HN.
I’m sort of curious if the AI doubting set will show up in force or not.
But just because I’ve not been lazy…
https://cloud.google.com/gemini/docs/codeassist/write-code-g...
(We may have differing definitions of "leaving" ones IDE).
I think this is key here. Whoever has the best UX for this (right now, it's Cursor IMO) will get the bulk of the market share. But the switching costs are so low for this set of tooling that we'll see a rapid improvement in the products available, and possibly some new entrants.
I used a similar setup until a few weeks ago, but coding agents became good enough recently.
I don’t find context management and copy pasting fun, I will let GitHub Copilot Insiders or Claude Code do it. I’m still very much in the loop while doing vibe coding.
Of course it depends on the code base, and Redis may not benefit much from coding agents.
But I don’t think one should reject vibe coding at this stage, it can be useful when you know what the LLMs are doing.
That said, are there tools that make going through a codebase easier for LLMs? I guess tools like Claude Code simply grep through the codebase and find out what Claude needs. Is that good enough or are there tools which keep a much more thorough view of the codebase?
Untrustworthy is worse than useless.
This is why Claude Code rocks - it quite often finds relevant parts itself.
It tried a few ways to read the PDF before coming up with installing PyPDF2, using that to parse the PDF, then updated all the JSON files. It took about 5 minutes to do this, but it ended up 100% correct, updating 7 different fields across two dozen JSON files.
(The reason for the PDF export was to get past the Confluence page being behind Okta authentication. In retrospect, I probably should've saved the HTML and/or let Claude Code figure out how to grab the page itself.)
How would I have done that with Gemini using just the web interface?
Been there, done that!
for those one-off small things, LLMs are rather cool. Especially Cloude Code and Gemini CLI. I was given an archive of some really old movies recently, but files were bearing title names in Croatian instead of original (mostly English ones). So I claude --dangerously-skip-permissions into the directory with movies and in a two-sentence prompt I asked it to rename files into a given format (that I tend to have in my archive) and for each title to find original name and year or release and use it in the file.. but, before commiting rename to give me a list of before and after for approval. It took like what, a minute of writing a prompt.
Now, for larger things, I'm still exploring a way, an angle, what and how to do it. I've tried from yolo prompting to structured and uber structured approaches, all the way to mimicking product/prd - architecture - project management / tasks - developer/agents.. so far, unless it's rather simpler projects I don't see it's happening that way. Most luck I had was "some structure" as context and inputs and then guiding prompting during sessions and reviewing stuff. Almost pair-programming.
I thought large contexts are not necessarily better and sometimes have opposite effect ?
On coding you need to aggressively prune it, and only give minimum adjacent context, or it'll start going on useless tangents. And if you get stuck just refresh and start from 0, changing what is included. It's often faster than "arguing" with the LLM in multi-step sessions.
(the above is for existing codebases. for vibe-coding one-off scripts, just go with the vibes, sometimes it works surprisingly well from a quick 2-3 lines prompt)
I've been going down to sonnet for coding over opus. maybe i am just writing dumb code
Opus is also way more expensive. (Don’t forget to switch back to Sonnet in all terminals)
> * Gemini 2.5 PRO > * Claude Opus 4
I think trying out all the LLMs for each task is highly underappreciated. There is no pareto optimal LLM for all skills. I give each prompt to 8 different LLMs using a Mac app. In my experience while Gemini is consistently in top 3 of 8, the difference between best output and Gemini Pro could be huge.
One example, I had a PR up that was being reviewed by a colleague. I was driving home from vacation when I saw the 3-4 comments come in. I read them when we stopped for gas, went to OpenAI / codex on my phone, dictated what I needed and made it PR to my branch. Then got back on the road & PR'd it. My colleague saw the PR, agreed and merged it in.
I think of it as having a ton of interns, the AI is about the same quality. It can help to have them, but they often get stuck, need guidance, etc. If you treat the AI like an intern and explain what you need it can often produce good results; just be prepared to fallback to coding quickly.
...one day.