You are 100% correct IMO for the AI stuff, just a question, what’s bitnet? Is it some supposed bitcoin-killer (because that’s not going to happen, IMO).
I also think we’re on the cusp of “having enough” personal computing power with the usual exceptions, take a bit, give a bit.
Our mobile phones are close to what’s needed for most people (8GB, 4k, gaming, …). Or so I think!
Bitnet is a catch-all title for ML models that use a specific mathematical trick.
If elements of a matrix are composed of only 1, 0, or -1, multiplying them is the same as as adding them.
That’s huge. LLM computation is basically all matrix multiplication, so if you replace that with simple addition, you reduce the computational requirements by orders of magnitude.
The catch is such models are hard to train effectively; its proven that it works, but research to get the technique usable and practical is still being done.
Personally, I suspect it’s unviable for many “dense” parts of models, but sparse hybrid bitnet models would be really cool.
I mention it because, if it takes off, suddenly the massive matrix multiplier accelerators we have for LLMs aren’t as useful. Chips with simpler architectures could get the job done, at least for parts of models that are bitnet.
You are 100% correct IMO for the AI stuff, just a question, what’s bitnet? Is it some supposed bitcoin-killer (because that’s not going to happen, IMO).
I also think we’re on the cusp of “having enough” personal computing power with the usual exceptions, take a bit, give a bit.
Our mobile phones are close to what’s needed for most people (8GB, 4k, gaming, …). Or so I think!
Bitnet is a catch-all title for ML models that use a specific mathematical trick.
If elements of a matrix are composed of only 1, 0, or -1, multiplying them is the same as as adding them.
That’s huge. LLM computation is basically all matrix multiplication, so if you replace that with simple addition, you reduce the computational requirements by orders of magnitude.
The catch is such models are hard to train effectively; its proven that it works, but research to get the technique usable and practical is still being done.
Personally, I suspect it’s unviable for many “dense” parts of models, but sparse hybrid bitnet models would be really cool.
I mention it because, if it takes off, suddenly the massive matrix multiplier accelerators we have for LLMs aren’t as useful. Chips with simpler architectures could get the job done, at least for parts of models that are bitnet.