floofloof@lemmy.ca to Technology@lemmy.worldEnglish · 15 hours agoMotherboard sales are now collapsing amid unprecedented shortages fueled by AIwww.tomshardware.comexternal-linkmessage-square74fedilinkarrow-up1422arrow-down12file-textcross-posted to: technology@hexbear.netpcmasterrace@lemmy.worldtechnology@lemmy.ziphackernews@lemmy.bestiver.se
arrow-up1420arrow-down1external-linkMotherboard sales are now collapsing amid unprecedented shortages fueled by AIwww.tomshardware.comfloofloof@lemmy.ca to Technology@lemmy.worldEnglish · 15 hours agomessage-square74fedilinkfile-textcross-posted to: technology@hexbear.netpcmasterrace@lemmy.worldtechnology@lemmy.ziphackernews@lemmy.bestiver.se
minus-squareSabinStargem@lemmy.todaylinkfedilinkEnglisharrow-up3·5 hours agoYou can use something like KoboldCPP on Linux, which allows both RAM and VRAM combined to run a model. O’course, not as fast when compared to pure VRAM or the Mac approach, but it is an option. I use my 128gb RAM with some GPUs for running models.
You can use something like KoboldCPP on Linux, which allows both RAM and VRAM combined to run a model. O’course, not as fast when compared to pure VRAM or the Mac approach, but it is an option. I use my 128gb RAM with some GPUs for running models.