Yes, because a lot of AI benchmarking at the moment is something that the companies created for themselves to gauge their own definitions of progress. Which is how OpenAI can spend last year releasing what they think is a massively better model of ChatGPT only to be met with an universal ‘meh, I guess’. From their paying users, even.
What we can conclude is that the US evaporating all their venture capital and also the Gulf’s to build language models does not meaningfully outperform the engineering that some chinese firms do on the side.
To what extent this matters is what’s harder to divine because the language model marketing is overwhelming any practical use for this technology outside of very specific companies making their own very specialized research. The idea that this is a future 100 trillion dollar industry of actual AGI distracts us from what could be a respectable 50 billion dollar industry of very specialized uses in mineral prospecting, manufacturing or QoL for coding.
Yes, because a lot of AI benchmarking at the moment is something that the companies created for themselves to gauge their own definitions of progress. Which is how OpenAI can spend last year releasing what they think is a massively better model of ChatGPT only to be met with an universal ‘meh, I guess’. From their paying users, even.
What we can conclude is that the US evaporating all their venture capital and also the Gulf’s to build language models does not meaningfully outperform the engineering that some chinese firms do on the side.
To what extent this matters is what’s harder to divine because the language model marketing is overwhelming any practical use for this technology outside of very specific companies making their own very specialized research. The idea that this is a future 100 trillion dollar industry of actual AGI distracts us from what could be a respectable 50 billion dollar industry of very specialized uses in mineral prospecting, manufacturing or QoL for coding.