Companies like Claude with their AI subscriptions might be losing money, but I doubt paid-per-token AI API usage is not making them money. There are several companies like e.g. DeepInfra and Fireworks that have sprung up to sell specifically that. I don’t think simply multiplying API cost with expected usage is sufficient to estimate how much will be charged however, because I suspect that OpenAI and Claude currently have a significant profit margin since they seem to be the defacto duopoly in the US.
The chart above shows that quite clearly, the vertical axis is the combined score on various benchmarks. The horizontal axis shows the price. OpenAI and Claude do score higher, but the price difference is enormous, even if it wasn’t a log scale (70$ vs 1.3k$ for similar results!). The competition of these companies could drastically reduce the margins of US companies,
I therefore think the pricing will depend on whether the large US AI companies manage to lobby the government to enact laws to cripple the competition of Chinese companies under the guise of security.
more than the cost of the human labor it replaced
I personally think that general consumers will never use LLMs in any significant number. I think that LLMs will exist in two distinct spaces, FOSS for devs and other technical people who want to run there own infra locally - and B2B for everything else.
The few big AI companies that manage to last will be selling access to their models for much higher prices. Probably similar to current proprietary commercial software like VMWare, SolidWorks, VEEAM, Splunk, etc. Companies will pay hundreds, possibly thousands of dollars per seat depending on the niche offering and amount of usage.
Suppose that a company developed an LLM that is trained & tuned specifically to do legal work, and suppose it produced work that was around 95% the quality of a typical paralegal. If that company charged $6,000 a year per license to work on their platform, that’s expensive, but if you’re a small firm with say, a dozen full time lawyers, then for the yearly price of a single average paralegal, you could have each lawyer using that software to do most of the work that the paralegal would have done. I can see those kinds of applications happening more and more.
This assumes though that LLMs will continue to improve at a significant rate for a long time into the future, (5-10 more years) which isn’t at all obvious, and there is some evidence that it’s already starting to hit a ceiling.
There are other ways it might work, like if there is a method of compression that is discovered that reduces the necessary RAM and Compute needs by 2-3 orders of magnitude. So models that are considered very large today (100-300 billion params at full quality) might be able to run effectively on a single 32GB GPU that costs a few thousand dollars.
So the cost to run these models is reduced immensely, and a single small data center could run enormous models with 1,000,000+ context windows for tens of thousands of users at once.
But that cuts both ways, which is something that any AI company is going to have to deal with. Once small free models get good enough to do the vast majority of a task, a user is going to start weighing the cost/benefits, and the prospect of just buying a box and throwing one of these models in for a few grand will be very appealing.
I think there may be a good market out there for “AI boxes”, compact computers designed to run a tuned LLM, set up with a little special sauce so the interface is user-friendly, etc. Companies could sell these with support contracts to legal firms, indie Dev studios, startups, small government agencies, etc.
Idk, it’s so up in the air right now, and everything is constantly changing so fast. It’s impossible to predict where things will be in 6 months, let alone 6 years from now.
There are other ways it might work, like if there is a method of compression that is discovered that reduces the necessary RAM and Compute needs by 2-3 orders of magnitude. So models that are considered very large today (100-300 billion params at full quality) might be able to run effectively on a single 32GB GPU that costs a few thousand dollars.
You might want to check in on how well distilled / quantized models are doing, compared to gigundo datacenter versions.
I think by the time AI becomes efficient enough to be profitable, it’s going to be efficient enough to run locally and the whole AI as a service business model is going to collapse. We’re basically in the mainframe era of AI right now, and we’ve seen this happen with many technologies before. There’s no reason to think this case will be different.
Just to give you an idea of how fast this stuff is moving. Qwen 3.6 was just released and can be run on a high end laptop, it outperforms Qwen 3.5 from February which required a commercial grade server to run. https://qwen.ai/blog?id=qwen3.6-27b
Yup. Already working on a suite of local pipeline apps and an orchestration platform for this. Happy to share if interested! Source
you should patent this!
And thereby lock it away from the underserved communities that need it most? Naw. Open source publishing is the way forward for a truly egalitarian system, which is what I’m aiming for
I was thinking of insulin.
The person who created it likewise refused to protect it from profit motives because he also felt that it belonged to humanity and it became captured to the detriment of humanity as a result.
Ah well the trouble is software patents can cost upwards of 5 figures, so yeah if I start making money I might do that, but it’s definitely not within my capacity for now, thus public publishing for copyright establishment
Maybe you could crowdfund it!
There’s no reason to think this case will be different.
Not even the end of Moore’s Law?
I’m not sure if you’re aware, but processors aren’t really getting much more efficient anymore. They’re just getting bigger (more parallel), which is why the price for the newer generations of GPUs has been skyrocketing. A new top-end GPU costs twice as much (or more) as a previous-gen one because it has twice as many (or more) compute units, since they can’t make the individual compute units much faster due to fundamental laws of physics.
I expect that software will continue to get optimized, and we’ll see new algorithms that are more efficient than what people are doing currently. However, it’s possible we’ll start seeing hardware specifically built for models as well. For example, there’s already a startup that uses ASIC chips to print the model directly to the chip. Since each transistor acts as a state, it doesn’t need DRAM and the whole chip requires a small amount of SRAM which isn’t in short supply right now https://www.anuragk.com/blog/posts/Taalas.html
The limitation with this approach is that the chip is made for a specific model, but that’s not really that different from the way regular chips work either. You buy a chip and if it does what you need, it keeps working. When new models come out, new chips get printed, and if you need the new capabilities then you upgrade.
You can see how absurdly fast their hardware version of llama 3 is here https://chatjimmy.ai/
That is indeed absurdly fast.
i suspect that we’ve neared the end of what we can get out of using silicon and the only way forward, at this point, is to switch materials altogether into something like graphene or carbon; but i bet it would take a long time to ever do that because the profit motives that keeps on silicon won’t allow for it.
From a basic physics research perspective (as opposed to an engineering process development for production perspective), are we even sure graphene semiconductors have that much potential headroom for improvement beyond the best possible silicon ones? I’m not convinced it buys us more than a couple of process nodes. I mean, we’re already making transistors so small you can damn near count the individual atoms in them today. Is making them out of atoms with one less valence level gonna be enough for a 10x, 100x, or 1000x improvement, even in the long run?
The Chinese will likely be the first ones to know for certain considering that they’ve already demonstrated a willingness to spend a metric fuck ton into public infrastructure like the United States used to do for its military.
There are a few different tracks here. One is software optimizations where models require less energy to use. That’s been moving really fast over the past few years, and there are still a lot of papers that haven’t been integrated into production systems that are really promising.
Another track is hardware architecture where the substrate stays the same, but chip design improves. A general example of this is SoC architecture like M series from Apple of Kirin 9000 from Huawei. The architecture eliminates the memory bus which is one of the main bottlenecks, and RISC instruction set facilitates parallelism much better than SISC. A more specific example would be ASIC chips like what Taalas is making which print the model directly on the chip.
And the last track is the one you mention with using a more efficient substrate. Notably this will directly benefit from the other two tracks as well. Whatever software and hardware architecture improvements people come up with, will directly apply to chips made out of graphene or other materials.
Agreed and all of those w tracks to squeeze out as much as we can from silicon.
There’s a limit that we haven’t yet reached but we will eventually because of those profits.
I bet that China will be the first to reach it since they’re willing to spend so much on all infrastructure.
I expect so as well, and China also has a lot of incentive to invest in alternative substrates since they’re behind on silicon. If one of these moonshot projects they’re pursuing delivers that would make current silicon chips look like vacuum tubes by comparison.
They’re fucked.
Local models are already winning. Those benchmarked a year behind the biggest of big boys, a year ago. Six months ago they were six months behind. Yesterday Qwen released 3.6 27B and it outperforms 3.5 397B… from February.
Either we’re plateauing toward the asymptotic limit of LLM capabilities, and the endgame runs as well on a toaster as it does on a server - or breakthroughs use big fat models as a glorified search space to be rapidly discarded. Both options point toward neural networks as a lump of algebra that sits on your hard drive and occasionally spins your fans. Remote computing loses, as it basically always must, and the drastically reduced requirements for competing on local software favor clever new competitors who aren’t a bajillion dollars in debt.
I agree with this. I have an openclaw setup since I want to own my own data and services. A few months ago Sonnet was the clear leader for general use task for me. Now Gemma 4 performs nearly as well hosted off my gaming PC. Based on resource utilization, I actually think I can run it from the same nuc that openclaw is hosted from.
I’m confused, aren’t they already charging? Something about tokens?
They’re charging but they’re burning cash by the truck load.
I’m guessing they’d need to charge north of $1000/month to get in the black.
most people are getting by on free tier or ~USD$20/MONTH
as many comments have said - this is probably a loss leader that won’t survive IPO.
Let’s do some estimates:
- An 8x H100 machine costs about $20 / hr to rent.
- With a 70B model with 4K context, a H100 node can do about 300 requests in parallel.
- A single response takes around 30 seconds to generate.
- An average user sends about 300 messages / month.
The throughput of a node is
300 concurrent * (3600 / 30) = 36 000 messages / hour.
The cost per message, then, is $20 / 36 000 = $.00055…
With 300 messages per month, the compute cost for the AI vendor is 300*$20/36000 = $0.16 / month per user. By contrast, a subscription costs $20.
So given these assumptions, it’s other things (like R&D, safety research, training runs, free accounts, etc) that represent the bulk of the cost and those could be scaled down to turn a profit. What will they do? Give how hyped AI is currently and the competitive landscape, I don’t think they’ll increase prices that much. We have products like DeepSeek on the horizon which are much cheaper, so it’s more likely that they squeeze money out of it by becoming more efficient.
It’s a weird market.
Those H100s are $25k minimum. So $200,000 just in GPUs. Drawing 700W each, or 5.6kW total. At my local prices that’s about a dollar per hour just for electricity.
It’s going to take you a couple of years to break even at $20/h. They might still hold some value at that point. Or they might be obsolete.
Well that entirely depends on your users… coding agents or in general agents that run for hours will crash your calculation
That won’t happen due to token limits. According to Anthropic, only about 5% of users hit the limit.
Exactly. Then you move up to the $100 or $200 or per token API pricing levels.
The pricing question assumes the current model (cloud inference, centralized compute, hyperscaler margins) is the only model.
Local inference flips that math entirely. If the model runs on your hardware, the marginal cost to the provider is close to zero. The pricing problem is a distribution problem, not a compute problem.
What I think actually happens: cloud AI settles at $20-50/month for power users who need the latest frontier models and don’t want to manage hardware. That’s sustainable. The “free tier” disappears or gets severely throttled.
But for a large chunk of use cases (summarization, classification, drafting, local assistants) models small enough to run on a consumer GPU are already good enough. That market doesn’t need to pay $50/month to Anthropic. It needs a good local runner and a one-time hardware investment.
The companies that will survive the pricing correction are the ones who either have genuinely differentiated frontier capability, or who make local deployment easy enough that users own their own stack.
There are also going to be issues with how bleeding edge AI gets sold. If the AI that can detect security exploits is real, the AI owner isn’t going to sell open access to that model.
I suspect that, if the AI is really that good for certain tasks, it won’t get sold on a token model but something more akin to human work.
I have created a machine
It can know all your secrets
I will sell your own secrets back to you
Because I own my machine and it now owns you too
This is something I keep asking and can’t get a good answer for. You see some of the ads as “your $20 per month gets you $300 in tokens!” But that’s not sustainable unless it’s being subsidized by low use people.
But then for that matter, what does the value of a token mean? Is it the amount of money you would save as compared to having a human work? That doesn’t help the company providing the service.
Or maybe is supposed to cover the price of the compute to execute the query. That would be ideal, but I don’t think that value is correct.
I really think it’s “the first hit is free” approach and soon they’re going to start jacking prices up. I believe all of them are operating as loss leaders, just to try to get market share and even this few initial price increases are showing how much they’re bleeding money.
Oh wow, that’s a fascinating topic.
This is something I’m looking forward to, because I assume it would be more expensive than we think. I hope to the point where it’s more prudent to pay a skilled worker than XY of money for slop. Currently AI is being indirectly “subsidized” through many high ticket investors, and that besically finances our “slop”.
I pray for the day when the AI companies have to “pay the piper” and start charging realistic prices for the use of AI.
I am biased because ATM I have to fight tooth and nail to keep my job because my CEO thinks that he can just upload the company logo to AI, and that AI would shit marketing back.
I expect consumer prices to be always a loss leader, with professional prices starting from €300/month.
The image, video, music generators will never be profitable since human artists are also paid jack shit anyway. But they will be available for their marketing value, making the real profit from business use.
The business prices need a 10-fold or raise though, at least.
I dunno on the music one. You are right human artists get paid shit but they also put out like an album a year at most whereas AI puts out literally thousands of tracks a year so theoretically with 100x more music, even if it’s only half as popular, should make plenty on streaming services.
Obviously it’s terrible music and we should all boycott music platforms that don’t aggressively remove AI content. But people are clearly doing it for a real, financial reason
Won’t that cause true economic chaos? There already is more music than one can listen to in their lifetime. You’ll never have to listen to the same track again if you chose so. That is true before AI comes in. So how to you chose what to listen to? Where do you put value in music?
AI is just going to devalue music and art in general. Flood these industries in a race to the bottom.
