I consolidated my setup a bit. This is my local LLM hosting server. I took a gamble on a Chinese NVLink SXM2 mezzanine board from ebay, it was surprisingly plug and play for my dual 16GB v100s 😂. I’m also running a Tesla P40 that I repasted with liquid metal that sits at 24C idle 🥶.
It’s been a fascinating experience so far and I feel like I’m only touching the surface. I’m exploring some of the various memory harnesses to hook into Hermes Agent and see how it learns with extended use.