The AI boom has turned the standard profit margin model on its head, according to Apollo Chief Economist Torsten Slok—and it’s making the industry’s growth unsustainable.
More specifically, autoregressive transformers LLMs are not a path to a component of AGI.
The architecture is absolutely terrible for such a thing, for so many reasons. I don’t know how anyone who’s played with them can say otherwise and believe it; it’s like saying blimps are a viable path to the moon. It has its niches, but AGI is not one of them.
Calling them a stepping stone is a… stretch.
Maybe world models that “train as they go” and have long moved on from transformers are a bit closer, like a few researchers are playing up, but again… that has almost nothing to do with transformers LLMs. Its why researchers distanced themselves from that.
I think your assessment is an extreme oversimplification which naturally looks like it will fail because you left out 90% of what is going on. Are you only following the media meant for the general public, and the PR statements from AI companies? Like most scientific or businesses endeavors, that’s been dumbed down to the point of being useless, just so the average person can understand what’s happening, or is simply advertising.
I dunno why everyone always jumps to accusations like this.
I’ve been hacking/toying with LLMs on my desktop since 2021, and with GANs and other models before that. I’ve done professional work with text models. I keep up with papers, on-and-off, and upload experiments. I’m not a researcher or anything; I’m just a hobbyist.
But I’m certainly not following any AI YouTubers or anything like that. And I wouldn’t trust Sam Altman if he said the sky is blue.
Okay.
More specifically, autoregressive transformers LLMs are not a path to a component of AGI.
The architecture is absolutely terrible for such a thing, for so many reasons. I don’t know how anyone who’s played with them can say otherwise and believe it; it’s like saying blimps are a viable path to the moon. It has its niches, but AGI is not one of them.
Calling them a stepping stone is a… stretch.
Maybe world models that “train as they go” and have long moved on from transformers are a bit closer, like a few researchers are playing up, but again… that has almost nothing to do with transformers LLMs. Its why researchers distanced themselves from that.
I dunno why everyone always jumps to accusations like this.
I’ve been hacking/toying with LLMs on my desktop since 2021, and with GANs and other models before that. I’ve done professional work with text models. I keep up with papers, on-and-off, and upload experiments. I’m not a researcher or anything; I’m just a hobbyist.
But I’m certainly not following any AI YouTubers or anything like that. And I wouldn’t trust Sam Altman if he said the sky is blue.