- cross-posted to:
- technology@lemmy.zip
- cross-posted to:
- technology@lemmy.zip
The University of Rhode Island’s AI lab estimates that GPT-5 averages just over 18 Wh per query, so putting all of ChatGPT’s reported 2.5 billion requests a day through the model could see energy usage as high as 45 GWh.
A daily energy use of 45 GWh is enormous. A typical modern nuclear power plant produces between 1 and 1.6 GW of electricity per reactor per hour, so data centers running OpenAI’s GPT-5 at 18 Wh per query could require the power equivalent of two to three nuclear power reactors, an amount that could be enough to power a small country.
You must log in or register to comment.
And an LLM that you could run local on a flash drive will do most of what it can do.
Probably not a flash drive but you can get decent mileage out of 7b models that run on any old laptop for tasks like text generation, shortening or summarizing.
What do you use your usb drive llm for?
Fucking Doc Brown could power a goddamn time machine with this many jiggawatts, fuck I hate being stuck in this timeline.
Help me out here. What designates the “response” type? Someone asking it to make a picture? Write a 20 page paper? Code a small app?
Response Type is decided by ChatGPTs new routing function based on your input. So yeah. Asking it to “think long and hard”, which I have seen people advocating for to get better results recently, will trigger the thinking model and waste more resources.
So instead of just saying “thank you” I now have to say “think long and hard about how much this means to me”?
I have an extreme dislike for OpenAI, Altman, and people like him, but the reasoning behind this article is just stuff some guy has pulled from his backside. There’s no facts here, it’s just “I believe XYX” with nothing to back it up.
We don’t need to make up nonsense about the LLM bubble. There’s plenty of valid enough criticisms as is.
By circulating a dumb figure like this, all you’re doing is granting OpenAI the power to come out and say “actually, it only uses X amount of power. We’re so great!”, where X is a figure that on its own would seem bad, but compared to this inflated figure sounds great. Don’t hand these shitty companies a marketing win.
The team measured GPT-5’s power consumption by combining two key factors: how long the model took to respond to a given request, and the estimated average power draw of the hardware [they believe is] running it.
I don’t care how rough the estimate is, LLMs are using insane amounts of power, and the message I’m getting here is that the newest incarnation uses even more.
BTW a lot of it seems to be just inefficient coding as Deepseek has shown.
And water usage which will also increase as fires increase and people have trouble getting access to clean water
https://techhq.com/news/ai-water-footprint-suggests-that-large-language-models-are-thirsty/
It would only take one regulation to fix that:
Datacenters that use liquid cooling must use closed loop systems.
The reason they dont, and why they setup in the desert, is because water is incredibly cheap and energy to cool a closed loop system is expensive. So they use evaporative open loop systems.
Unfortunately I wonder if it’s more expensive to set up a closed loop system that’s really expensive or to buy lawmakers that will vote against bills saying you should do so and it’s a tale old as time
Politicians are cheap
Yeah sorry forgot my /s there
That increases your energy use though, because evaporative cooling is very energy efficient.
We can make energy from renewable sources.
Fresh drinking water is finite, especially in the desert.
BTW a lot of it seems to be just inefficient coding as Deepseek has shown.
Kind of? Inefficient coding is definitely a part of it. But a large part is also just the iterative nature of how these algorithms operate. We might be able to improve that via code optimization a little bit. But without radically changing how these engines operates it won’t make a big difference.
The scope of the data being used and trained on is probably a bigger issue. Which is why there’s been a push by some to move from LLMs to SLMs. We don’t need the model to be cluttered with information on geology, ancient history, cooking, software development, sports trivia, etc if it’s only going to be used for looking up stuff on music and musicians.
But either way, there’s a big ‘diminishing returns’ factor to this right now that isn’t being appreciated. Typical human nature: give me that tiny boost in performance regardless of the cost, because I don’t have to deal with. It’s the same short-sighted shit that got us into this looming environmental crisis.
Coordinated SLM governors that can redirect queries to the appropriate SLM seems like a good solution.
That basically just sounds like Mixture of Experts
Basically, but with MCP and SLMs interacting rather than a singular model, with the coordinator model only doing the work to figure out who to field the question to, and then continuously provide context to other SLMs in the case of more complex queries
Powered by GNU Hurd
Also don’t forget how people like wasting resources by asking questions like “what’s the weather today”.
My guess would be that using a desktop computer to make the queries and read the results consumes more power than the LLM, at least in the case of quickly answering models.
The expensive part is training a model but usage is most likely not sold at a loss, so it can’t use an unreasonable amount of energy.
Instead of this ridiculous energy argument, we should focus on the fact that AI (and other products that money is thrown at) aren’t actually that useful but companies control the narrative. AI is particularly successful here with every CEO wanting in on it and people afraid it is so good it will end the world.
that’s a lot. remember to add “-noai” to your google searches.
Or just use any other better search like Bing or duckduckgo. googol sucks and was never any good. Quit pushing ignorant garbage.
duckduckgo yes, but … bing?
ddg is bing
Bing is for porn.
I’m just going to ignore the AI recommendations, let them burn money.
i don’t judge you for that. honestly it matters fuck all at this point
I think AI power usage has an upside. No amount of hype can pay the light bill.
AI is either going to be the most valuable tech in history, or it’s going to be a giant pile of ash that used to be VC capital.
It will not go away at this point. Too many daily users already, who uses it for study, work, chatting, looking things up.
If not OpenAI, it will be another service.
Those users are not paying a sustainable price, they’re using chatbots because they’re kept artificially cheap to increase use rates.
Force them to pay enough to make these bots profitable and I guarantee they’ll stop.
Those same things were said about hundreds of other technologies that no longer exist in any meaningful sense. Current usage of a technology, which in this specific case I would argue is largely frivolous anyway, is not an accurate indicator of future usage.
Can you give some examples of those technologies? I’d be interested in how many weren’t replaced with something more efficient or convenient.
https://en.wikipedia.org/wiki/Dot-com_bubble
There were certainly companies that survived, because yes, the idea of websites being interactive rather than informational was huge, but everyone jumped on that bandwagon to build useless shit.
As an example, this is today’s ProductHunt
And yesterday’s was AI, and the day before that it was AI, but most of them are demonstrating little value with high valuations.
LLMs will survive, likely improve into coordinator models that request data from SLMs and connect through MCP, but the investment bubble can’t sustain
Technologies come and go, but often when a worldwide popular one vanishes, it’s because it got replaced with something else.
So lets say we need LLM’s to go away. What should that be? Impossible to answer, I know, but that’s what it would take.
We cant even get rid of Facebook and Twitter.
BUT that being said. LLMs will be 100x more efficient at some point - like any other new technology. We are just not there yet.
@themurphy @rigatti There is one difference … LLM’s can’t be more efficient there is an inherent limitation to the technology.
https://blog.dshr.org/2021/03/internet-archive-storage.html
In 2021 they used 200PB and they for sure didn’t make a copy of the complete internet. Now ask yourself if all this information without loosing informations can fit into a 1TB Model ?? ( Sidenote deepseek r1 is 404GB so not even 1TB ) … local llm’s usually < 16GB …
This technology has been and will be never able to 100% replicate the original informations.
It has a certain use ( Machine Learning has been used much longer already ) but not what people want it to be (imho).
And most importantly the Pandora box has been opened for deep perfect scams and illegal usage. Nobody will put it in the box again, because even if everyone agreed to make it illegal everywhere it’s already too late.
That capital was ash earlier this year. The latest $40 Billion-with-a-B financing round is just a temporary holdover until they can raise more fuel. And they already burned through Microsoft, who apparently got what they wanted and are all “see ya”.
Bit of a clickbait. We can’t really say it without more info.
But it’s important to point out that the lab’s test methodology is far from ideal.
The team measured GPT-5’s power consumption by combining two key factors: how long the model took to respond to a given request, and the estimated average power draw of the hardware running it.
What we do know is that the price went down. So this could be a strong indication the model is, in fact, more energy efficient. At least a stronger indicator than response time.
That’s a terrible metric. By this providers that maximize hardware (and energy) use by having a queue of requests would be seen as having more energy use.
Isnt it just worse than 4 tho? If they didnt make it cheaper, nobody would pay…
Tech hasn’t improved that much in the last in the last decade. All that’s happened is that more cores have been added. The single-thread speed of a CPU is stagnant.
My home PC consumes more power than my Pentium 3 consumed 25 years ago. All efficiency gains are lost to scaling for more processing power. All improvements in processing power are lost to shitty, bloated code.
We don’t have the tech for AI. We’re just scaling up to the electrical senand demand of a small country and pretending we have the tech for AI.
Not even the ai tech itself is enough for ai
It’s the muscle car era: can’t make things more efficient to compete with Asia? MAKE IT BIGGER AND CONSUME MORE
This is nonsense, an M1 runs many multiples faster and at much lower wattage.
That’s alright. When they’ve got a generation of people who can’t even hold a conversation without it, let alone do a job, that price increase will drop that energy use pretty rapidly.
How the hell are they going to sustain the expense to power that? Setting aside the environmental catastrophe that this kind of “AI” entails, they’re just not very profitable.
Look at all the layoffs they’ve been able to implement with the mere threat that AI has taken their jobs. It’s very profitable, just not in a sustainable way. But sustainability isn’t the goal. Feudal state mindset in the populace is.
Not just”not profitable”, they don’t make any money at all. Loss only.
I don’t buy the research paper at all. Of course we have no idea what OpenAI does because they aren’t open at all, but Deepseek’s publish papers suggest it’s much more complex than 1 model per node… I think they recommended like a 576 GPU cluster, with a scheme to split experts.
That, and going by the really small active parameter count of gpt-oss, I bet the model is sparse as heck.
There’s no way the effective batch size is 8, it has to be waaay higher than that.
And perhaps even more importantly, the per-token cost of GPT-5’s API is less than GPT-4’s. That’s why OpenAI was so eager to move everyone onto it, it means more profit for them.
I don’t believe api costs are tied all that closely to the actual cost to openAI. They seem to be selling at a loss, and they may be selling at an even greater loss to make it look like they are progressing. The second openAI seems like they have plateaued, their stock evaluation will crash and it will be game over for them.
I based my argument on actual numbers that can be looked up and verified. You “believe” that they “seem” to be doing something else. Based on what?
Their point is that those API prices might not match reality, and the prices may be artificially low to build hype and undercut competitors. We don’t know how much it costs OpenAI, however we do know that they’re not making a profit.
Or it might not. It would be a huge short term risk to do so.
As FaceDeer said, that we truly don’t know.
OpenAI are not profitable today, and don’t estimate they’ll be profitable until 2029, so it’s almost guaranteed that they’re selling their services at a loss. Of course, that’s impossible to verify - since they’re a private company, they don’t have to release financial statements.
There’s a difference between selling at a loss, and having a loss.
OpenAI let’s people use models for free with very little limits other than reducing the model quality over time, and they have very generous limits before they limit you at that.
That all costs money and is a loss for them.
If they get someone who’s willing to pay, and they charge $20/m and on average, they net $5 profit per customer, they aren’t selling it at a loss, they just need more customers. It’s possible that a paid customer uses it even more though and it actually does incur a loss per paid customer and they’re doing that to try and gain users while they figure out how to lower their costs, but that seems less likely.
That’s not what I’m saying. They’ve all but outright said they’re unprofitable.
But revenue is increasing. Now, if it stops increasing like they’ve “leveled out”, that is a problem.
Hence it’s a stretch to assume they would decrease costs for a more expensive model since that would basically pop their bubble well before 2029.
Sure, they might not. But he gives no basis for saying that other than what he “believes.”
People in this community, and on the Fediverse in general, seem to be strongly anti-AI and would like to believe things that make it sound bad and unprofitable. So when an article like this comes along and says exactly what you want to believe it’s easy to just nod and go “knew it!” Rather than investigating the reasons for those beliefs and risking finding out something you didn’t want to know.
that make it sound bad and unprofitable
It is unprofitable, though.
OpenAI recently hit $10 billion in ARR and are likely to hit $12.7b by the end of the year, but they’re still losing a lot of money. They don’t think they’ll make a profit until 2029, and only if they hit their target of $125 billion revenue. That’s a huge amount of growth - 10x in 4 years - so I’m interested as to if they’ll actually hit it.
Okay, make it sound worse and even more unprofitable.
Making their AI models cheaper to run (such as by requiring less electricity) is one step along that path to profitability.
To be fair, OpenAI’s negative profitability has been extensively reported on.
Your point stands though; there’s no evidence they’re trying to decrease revenue. On the contrary, that would be a huge red flag to any vested interests.
How does OpenAI getting less money (with a cheaper model) mean more profit? Am I missing something?
Usually, companies will make their product say 25% cheaper to produce, then sell it to the public at a 20% discount (while loudly proclaiming to the world about that 20% price drop) and pocket that 5% increase in profits. So if OpenAI is dropping the price by x, it’s safe to assume that the efficiency gains work out to x+1.
Thanks! This makes sense, however OpenAI are not yet profitable. It’s definitely possible that they’re losing less money with the new models, though.
That “not profitable” label should be taken with a grain of salt. Startups will do all the creative accounting they can in order to maintain that label. After all, don’t have to pay taxes on negative profits.
In the end, it still means their losses are greater than their profits.
They’ve still got taxes they need to pay, too - things like payroll taxes, real estate taxes, etc.
If the model is cheaper to run then they are able to reduce the price without reducing profit, which gives them an advantage over competitors and draws in more customer activity. OpenAI is far from a monopoly.
This bubble needs to pop, the sooner the better.
OpenAI just needs to harness lightning. Incoming weather control tech.
