😂 I can see how it looks that way lol

It’s been a fascinating experience so far and I feel like I’m only touching the surface. I’m exploring some of the various memory harnesses to hook into Hermes Agent and see how it learns with extended use.

Currently I’m running a Q6K quant of Hermes 4 14B with a 32K context window via llama.cpp that works pretty well. Generation output is a comfy ~50tok/sec. These v100s are 16GB each, but there are 32GB versions available too.

I’m running everything via NixOS and have to do package overrides to get inference engines to build with the right CUDA versions.

My goal is to get a cohesive environment set up for Hermes Agent to learn my system/lab/network and help my grow it over time.

Overall, I’m happy with them. The mezzanine board is good quality, I’m using PTM sheets under those massive heatsinks and some arctic p9 fans to keep them at around 60C under load.

It’s an older version of a heatset insert fixture. I used it to put threaded brass inserts into my 3d prints instead of trying to capture a nut or screwing directly into plastic.

https://www.printables.com/model/609644-stealth-press-1-heat-set-insert-press-legacy

My favorite smell is also my favorite word; Petrichor. It’s the smell of the earth when it rains after an extended dry period.

From Wikipedia “from Ancient Greek πέτρα (pétra) ‘rock’; or πέτρος (pétros) ‘stone’ and ἰχώρ (ikhṓr) ‘ichor’, the ethereal fluid that is the blood of the gods in Greek mythology.”

https://en.wikipedia.org/wiki/Petrichor

“the front fell off?!”

They’ve been temporarily removed for about a year now lol

I’m actually quite happy with it. It’s a 24gb p40 which runs some of the older LLMs quite well. Repasting with liquid metal helped thermals a ton too

It’s a Tesla P40 24gb I got off eBay a while ago. Just like AllHailTheSheep suggested, it’s a server GPU meant for compute workflows so it relies on forced air from a server rack. I repasted the die with liquid metal and added a 3d printed duct to attach the radial blower to. The thing never cracks 60c under inference which is nice. Idles around 24c

(Edit: I’m mostly using it for LLMs, but it supposedly makes a good CAD card, so I might give that a shot too)

Thank you 🥹 It’s been quite the experience lol

I thankfully snagged this one pretty cheap off eBay. It’s actually an 8-gpu mining rig frame that conveniently fit my CEB motherboard lol

My janky homelab lol. Mostly ebay secondhand Enterprise stuff and a Chinese SXM2 mezzanine board to run dual NVLinked 16gb V100s. I also have a TrueNAS Scale mini itx server running upstairs with my “arr” stack and some other useful tools.