I’ve been in software for nearly two decades. The industry has fundamentally changed with the advent of language model tools. I don’t like it, but Pandora’s box is wide open and I want to understand what the path forward is.
I don’t write code for work anymore, I babysit a language model. Maintaining a sense of code ownership and comprehension is a real risk, and we are pushed to use tools that turn our brains into pudding.
Is this what the old timers thought when fancy pants compilers showed up? The reason I disagree with this analogy is that the new tooling is fundamentally incompatible with software freedom that cannot be solved with gcc et al. I can’t afford a $10k computer to run local models.
The idea is you’re basically expected to be a manager so the LLM is now your junior engineer except you don’t get the joy of building someone up and watching them grow you get an expensive robot that never learns and can’t remember what you said yesterday.
Two of my coworkers are constantly adding to and tweaking our repo’s skills folders. I guess that’s the equivalent of mentoring for them and it actually has made the responses significantly higher quality.
I can’t afford a $10k computer to run local models.
Tbh you don’t really need to either. With models like Qwen 3.6 35B A3B (which is quite close to the performance of frontier models from a year ago) enough of it fits onto a 8GB GPU - with the rest sitting in RAM or swapped out - to have it running at a speed that is very workable with. On my work laptop with a 4060 equivalent and llama.cpp + CUDA I get around 15-20 t/s for coding tasks. The equivalent Qwen 3.8 variant should be even better once that releases. I strongly believe small models like these running locally is going to be the way to go once the bubble bursts.
It’s pretty funny watching software go from arguing about functional programming, total test coverage, side effect free code, and mathematical proofs of correctness to “eh just sorta run your eyes over code you didn’t write and hope that 6 hours into your shift you’re not letting anything critical by”
Erlang, Haskell, my beloved. I will still love you while the others are lost to a storm of madness.
The reason I disagree with this analogy is that the new tooling is fundamentally incompatible with software freedom that cannot be solved with gcc et al. I can’t afford a $10k computer to run local models.
That’s actually a good point but if we consider history the barrier to entry to software engineering was huge 40-50 years ago, not many people could afford to do it locally. Then again it’s different because software as a whole wasn’t really that important 50 years ago.
Then again “software freedom” as we know it may just as well be a coincidental transient phenomenon because capitalist markets arranged themselves in such a way to make the hardware and software required to build and maintain software accessible enough to the general public, that “right” can conceivably be taken away by the same system again, there is no reason to assume it’s here to stay.
Honestly I’m just rambling at this point I have no idea where any of this leads I’m pretty much hoping the whole thing is gonna be a dud and we’ll return to the same old way of doing things, capitalism is a fuck, death to america.
It’s not magic these models are big and you can’t just squeeze them onto smaller devices. Quantisation isn’t magic either it’s literally reducing the accuracy of a larger model to fit on a smaller device reducing its performance. If the data doesn’t exist in the model weights then the model will perform worse and you can only compress data so much. The only meaningful tech shift in the space is unified memory but it’s not going to be enough for anything this is fundamentally going to need silly quantities of ram for any effective model to run.
The main limitation is just RAM on the GPUs. Unfortunately the bubble itself drove up prices on that exact thing dramatically but this not a particularly expensive thing normally.
GPU prices shot up in ~2018 and 2020 and [etc] and never came back down. Nowadays a 9060XT (the useful version, mind) costs on its own what a decent entry level PC used to. We are never going back to normal.
ⓘ Remember to enable JPEG-XL support in your browser, even on your phone!
We’re at a really different point technologically and given the whole thing of Google’s paper and the architecture is more data more layers more good there is a huge incentive to run on as much ram as possible which is expensive.
We’re not going from hand soldered resistors to nanoscale transistors again. We’re looking at some serious physics walls. Even if photonic chips leave the lab.
I’ve been in software for nearly two decades. The industry has fundamentally changed with the advent of language model tools. I don’t like it, but Pandora’s box is wide open and I want to understand what the path forward is.
I don’t write code for work anymore, I babysit a language model. Maintaining a sense of code ownership and comprehension is a real risk, and we are pushed to use tools that turn our brains into pudding.
Is this what the old timers thought when fancy pants compilers showed up? The reason I disagree with this analogy is that the new tooling is fundamentally incompatible with software freedom that cannot be solved with gcc et al. I can’t afford a $10k computer to run local models.
The idea is you’re basically expected to be a manager so the LLM is now your junior engineer except you don’t get the joy of building someone up and watching them grow you get an expensive robot that never learns and can’t remember what you said yesterday.
I really do miss mentoring junior engineers. At least they never rambled on and on about “load bearing seams”
“You are absolutely right!”
Two of my coworkers are constantly adding to and tweaking our repo’s skills folders. I guess that’s the equivalent of mentoring for them and it actually has made the responses significantly higher quality.
Tbh you don’t really need to either. With models like Qwen 3.6 35B A3B (which is quite close to the performance of frontier models from a year ago) enough of it fits onto a 8GB GPU - with the rest sitting in RAM or swapped out - to have it running at a speed that is very workable with. On my work laptop with a 4060 equivalent and llama.cpp + CUDA I get around 15-20 t/s for coding tasks. The equivalent Qwen 3.8 variant should be even better once that releases. I strongly believe small models like these running locally is going to be the way to go once the bubble bursts.
It’s pretty funny watching software go from arguing about functional programming, total test coverage, side effect free code, and mathematical proofs of correctness to “eh just sorta run your eyes over code you didn’t write and hope that 6 hours into your shift you’re not letting anything critical by”
Erlang, Haskell, my beloved. I will still love you while the others are lost to a storm of madness.
It’s not funny at all. I’m gonna develop male pattern baldness from pulling my hair out every time I have to interact with LLM-using ‘programmers’.
This is why I chose to laugh. I am making games as a hobby now, there’s nothing left in computer touching for money that I used to love.
That’s actually a good point but if we consider history the barrier to entry to software engineering was huge 40-50 years ago, not many people could afford to do it locally. Then again it’s different because software as a whole wasn’t really that important 50 years ago.
Then again “software freedom” as we know it may just as well be a coincidental transient phenomenon because capitalist markets arranged themselves in such a way to make the hardware and software required to build and maintain software accessible enough to the general public, that “right” can conceivably be taken away by the same system again, there is no reason to assume it’s here to stay.
Honestly I’m just rambling at this point I have no idea where any of this leads I’m pretty much hoping the whole thing is gonna be a dud and we’ll return to the same old way of doing things, capitalism is a fuck, death to america.
Hopefully that is only temporary.
I imagine few people could afford a Unix system in 1987 when Stallman first wrote GCC.
It’s not magic these models are big and you can’t just squeeze them onto smaller devices. Quantisation isn’t magic either it’s literally reducing the accuracy of a larger model to fit on a smaller device reducing its performance. If the data doesn’t exist in the model weights then the model will perform worse and you can only compress data so much. The only meaningful tech shift in the space is unified memory but it’s not going to be enough for anything this is fundamentally going to need silly quantities of ram for any effective model to run.
The main limitation is just RAM on the GPUs. Unfortunately the bubble itself drove up prices on that exact thing dramatically but this not a particularly expensive thing normally.
GPU prices shot up in ~2018 and 2020 and [etc] and never came back down. Nowadays a 9060XT (the useful version, mind) costs on its own what a decent entry level PC used to. We are never going back to normal.
ⓘ Remember to enable JPEG-XL support in your browser, even on your phone!
Excuse my ma’am but let me tell you about a little thing called supply and demand. smirks insufferably just before getting punched
We’re at a really different point technologically and given the whole thing of Google’s paper and the architecture is more data more layers more good there is a huge incentive to run on as much ram as possible which is expensive.
We’re not going from hand soldered resistors to nanoscale transistors again. We’re looking at some serious physics walls. Even if photonic chips leave the lab.