Microsoft just open-sourced bitnet.cpp, a 1-bit LLM inference framework. It let's you run 100B parameter models on your local CPU without GPUs. 6.17x faster inference and 82.2% less energy on CPUs.

cm0002@ttrpg.network · 2 months ago

Microsoft just open-sourced bitnet.cpp, a 1-bit LLM inference framework. It let's you run 100B parameter models on your local CPU without GPUs. 6.17x faster inference and 82.2% less energy on CPUs.

Arghblarg@lemmy.ca · 2 months ago

Is it still probabilistic slop or does the model understand what it’s doing and verify primary sources? If not, yay for burning the planet more slowly I guess, but still no thanks.

scholar@lemmy.world · 2 months ago

No that would require AI (Actual Intelligence)

Leon@pawb.social · 2 months ago

We already have that. It’s like, you put the AI in your brain. You are the AI.

Microsoft just open-sourced bitnet.cpp, a 1-bit LLM inference framework. It let's you run 100B parameter models on your local CPU without GPUs. 6.17x faster inference and 82.2% less energy on CPUs.

Microsoft just open-sourced bitnet.cpp, a 1-bit LLM inference framework. It let's you run 100B parameter models on your local CPU without GPUs. 6.17x faster inference and 82.2% less energy on CPUs.

GitHub - microsoft/BitNet: Official inference framework for 1-bit LLMs