I have avoided AI (LLMS) because it didn’t seem nearly useful enough to warrant supporting these tech assholes. Is there anyway to use it locally on a normal device (16 Gb ram, 8 year old Geforce graphics, etc?). Can I just slow it down and get it to work locally?
I run a “server” (old 9-year gamer laptop) with an 8 GB Geforce 1070 Max-Q. It’s currently running full GPU-offloaded stock llama.cpp, with Unsloth’s Qwen3.5 9B at UD Q4_K_XL (good enough quantization for coding assistance and short agentic tasks) and a 90k context at around 300 t/s prefill and 25 t/s decode, which are good enough speeds for daily use.
It can’t vibecode GTA6, but it can help with quick scripting, research and things like that.