Individuals using AI are a tiny drop in the bucket. Think bigger: Businesses and government.
There’s government agencies racking up token usages to the point where Google had to build an entire data center on Christmas Island (the place with the cool crabs) and that still won’t be enough! And that’s just one government!
Companies are using AI in automations now and it’s working so well that they’re willing to spend tens of thousands of dollars a month (on tokens) just on that one automation.
These usages are where demand is growing. Not regular people or even developers using AI to help them write code.
Companies are using AI in automations now and it’s working so well that they’re willing to spend tens of thousands of dollars a month (on tokens) just on that one automation
it’s working so well
Citation needed. I work in a midsized IT corpo, I don’t see it working that well? Sure, some shit is shipped faster, some is shipped slower, customers churn is 2x higher than “planned” and prod bugs have doubled.
(And we do all the fancy shit, looping, custom harnesses, orchestrations, handovers, savepoints, post delivery pipelines, autoremediation, blablabla, context, graphs blabla)
(And we do all the fancy shit, looping, custom harnesses, orchestrations, handovers, savepoints, post delivery pipelines, autoremediation, blablabla, context, graphs blabla)
Yeah that’s not what I’m talking about. That’s just coding stuff and I think everyone is learning a hard lesson right now that there’s really two ways to do AI (for code):
You let the AI write all the code. Using those techniques you mentioned to try to keep it from fucking up too much.
You use the AI to fix/write a little bit of the code at a time. Manually. As in, someone who knows software architecture really well tells the AI to write a function or two or maybe even a whole module and that’s it. They do this repeatedly until their task is complete.
#1 is a very expensive way to do it but it seems to work, albeit with a huge learning curve and a much longer horizon before you get something stable. Even then, what you get can be a giant fucking mess that’s really, really hard to understand.
#2 is much safer and works quite well, IMHO. When the AI fucks something up, it’s only a tiny little thing that’s easy to fix. This method really is just a way for developers to improve their productivity.
With #2, it only takes a small amount of tokens to have the AI help you troubleshoot as well. At least, that’s been my experience.
Having said that, the places where we are seeing the most growth in AI is small shit. For example, at my work they implemented an agent that scans customer complaints and filters out false positives. I work for a huge company that has a zillion products/services and the AI has tool calls into our ticketing systems to check if there was an outage at the time the customer complained. It’s not that smart, but it’s smart enough to tell if a ticket about system X was opened when system X had an outage. It can even tell if the customer was complaining about something relevant to the outage.
Stuff like that runs 24/7 and eats up bazillions of tokens, but only on our local hardware (we have thousands of enterprise GPUs). At other companies, they’re doing similar things and slowly realizing they can replace OpenAI/Anthropic API calls with local ollama stuff.
Even if your token bill is $10,000/month for a service like that, it’s still cheaper than paying an entire team of humans to perform the same function.
That is the growing use of AI automation I was talking about.
I wouldn’t believe that hype
I am the only person in my circle using open weight models.
Normies out number us by a large amount and none of them really are running their own things.
Individuals using AI are a tiny drop in the bucket. Think bigger: Businesses and government.
There’s government agencies racking up token usages to the point where Google had to build an entire data center on Christmas Island (the place with the cool crabs) and that still won’t be enough! And that’s just one government!
Companies are using AI in automations now and it’s working so well that they’re willing to spend tens of thousands of dollars a month (on tokens) just on that one automation.
These usages are where demand is growing. Not regular people or even developers using AI to help them write code.
Citation needed. I work in a midsized IT corpo, I don’t see it working that well? Sure, some shit is shipped faster, some is shipped slower, customers churn is 2x higher than “planned” and prod bugs have doubled.
(And we do all the fancy shit, looping, custom harnesses, orchestrations, handovers, savepoints, post delivery pipelines, autoremediation, blablabla, context, graphs blabla)
Yeah that’s not what I’m talking about. That’s just coding stuff and I think everyone is learning a hard lesson right now that there’s really two ways to do AI (for code):
#1 is a very expensive way to do it but it seems to work, albeit with a huge learning curve and a much longer horizon before you get something stable. Even then, what you get can be a giant fucking mess that’s really, really hard to understand.
#2 is much safer and works quite well, IMHO. When the AI fucks something up, it’s only a tiny little thing that’s easy to fix. This method really is just a way for developers to improve their productivity.
With #2, it only takes a small amount of tokens to have the AI help you troubleshoot as well. At least, that’s been my experience.
Having said that, the places where we are seeing the most growth in AI is small shit. For example, at my work they implemented an agent that scans customer complaints and filters out false positives. I work for a huge company that has a zillion products/services and the AI has tool calls into our ticketing systems to check if there was an outage at the time the customer complained. It’s not that smart, but it’s smart enough to tell if a ticket about system X was opened when system X had an outage. It can even tell if the customer was complaining about something relevant to the outage.
Stuff like that runs 24/7 and eats up bazillions of tokens, but only on our local hardware (we have thousands of enterprise GPUs). At other companies, they’re doing similar things and slowly realizing they can replace OpenAI/Anthropic API calls with local ollama stuff.
Even if your token bill is $10,000/month for a service like that, it’s still cheaper than paying an entire team of humans to perform the same function.
That is the growing use of AI automation I was talking about.