But “90% marketing” and AI being useful isn’t mutually exclusive.
I’d say AI is even 95% marketing but I still use it for the other 5%, where it’s actually useful. It’s just waaaaaay too expensive for the 5% use case usually.
I’d go for smaller, more specialised models instead of trying to fit everything in a broadly incompetent jack of all trades, master of none type of LLM like they’re currently doing.
Yes, something like that. If you use a model which is specifically trained on C(++), Assembly and Rust, as well as scientific articles about Linux and the Linux Kernel, has the whole Linux Kernel already baked in, you can probably get faster and more accurate results than if you add the whole Wikipedia in 20 languages.
My LLM doesn’t need to answer all the questions, it’s enough if it answers the ones I want to ask. AI and the (mathematical) algorithms used for it are very old in terms of computer timeframes (think the 70s to 90s). The new part with LLMs is actually in the first “L”: large datasets. I think we have gone too large.
And, with Linux, you have an issue with authorship if you train on copyrighted data. Reproducing code 1:1 from a non-Free textbook might be problematic, so special LLMs trained on Open/Libre texts will be safer in terms of compliance as well.
But “90% marketing” and AI being useful isn’t mutually exclusive.
I’d say AI is even 95% marketing but I still use it for the other 5%, where it’s actually useful. It’s just waaaaaay too expensive for the 5% use case usually.
I’d go for smaller, more specialised models instead of trying to fit everything in a broadly incompetent jack of all trades, master of none type of LLM like they’re currently doing.
like preferably a model only trained on one language kind of idea?
Supposedly that makes them dumber in general
Yes, something like that. If you use a model which is specifically trained on C(++), Assembly and Rust, as well as scientific articles about Linux and the Linux Kernel, has the whole Linux Kernel already baked in, you can probably get faster and more accurate results than if you add the whole Wikipedia in 20 languages.
My LLM doesn’t need to answer all the questions, it’s enough if it answers the ones I want to ask. AI and the (mathematical) algorithms used for it are very old in terms of computer timeframes (think the 70s to 90s). The new part with LLMs is actually in the first “L”: large datasets. I think we have gone too large.
And, with Linux, you have an issue with authorship if you train on copyrighted data. Reproducing code 1:1 from a non-Free textbook might be problematic, so special LLMs trained on Open/Libre texts will be safer in terms of compliance as well.