But that aside, I found myself working with huge files that swapped like crazy on my 32GB of RAM, or were completely infeasible to work with. Janky local experiments I wouldn’t want to run on a cloud instance billed by the minute.
128GB has been a godsend, but I could completely fill 192GB and still swap some if I had it.
But I found there are happy side effects to having so much disk cache, too. For instance, game modding/tinkering has sped up immensely over 32GB. So has hashing for transfers, or other scenarios where it’s useful to already have a file cached in RAM.
I can rent cloud stuff, but it’s just not worth it for casual/experimental use.
Like keeping Deepseek V4 loaded and hitting it rarely, but I want it to be in my control… it’s just easier to do it locally.
Or experiments I launch like 30 times before it works. I’d waste hours moving all my stuff to a cloud instance, configuring it, tinkering with the experiment to get it to launch; the thing would be idle the vast majority of the time. And yeah, I know I can containerize stuff, but some projects I can’t even test without a sizable memory pool.
Same. But more than that - I noticed that my 7700x is a huge bottleneck: desktop CPUs limited to just 2 channels, plus controller in 7700 is kinda meh, so I can’t force anything above 5600mt\s with my 4 sticks. Switching to new CPU should allow to increase speed, but will still lock me to 2 channels. So I should’ve bought cheapest threadripper instead, which allows 4 or 8 channels instead.
Depends what you are going for. 7000/9000 series have way more per-thread performance than threadripper.
Another quirk with single CCD 7000 chips (which I have as well) is that the infinity fabric limits effective bandwidth to the CPU cores. Even with two channels, you hit diminishing returns with higher speeds, especially if you aren’t using the IGP.
You can get around it overclocking the IF, but it gets complicated and its much easier to do with 9000 series CPUs.
Only if you actually use that much. You almost certainly don’t.
I do.
I wish I had 192GB, to be honest.
What is the need of 192GB?
I often execute calculations on a cluster where I’ll need some 500-1000 GB of ram. But I have 32 GB in my workstation and I never felt I needed more.
Everything I need to do I can do: I can be having many browser tabs open, several IDEs, compiling and have a game engine open at the same time.
Only times I fill the ram is when I stress test my software for ram usage, but that is to be expected.
Machine learning/data science stuff.
Local LLMs is half of it.
But that aside, I found myself working with huge files that swapped like crazy on my 32GB of RAM, or were completely infeasible to work with. Janky local experiments I wouldn’t want to run on a cloud instance billed by the minute.
128GB has been a godsend, but I could completely fill 192GB and still swap some if I had it.
But I found there are happy side effects to having so much disk cache, too. For instance, game modding/tinkering has sped up immensely over 32GB. So has hashing for transfers, or other scenarios where it’s useful to already have a file cached in RAM.
Fair, I guess I’d also want some ram if I did not have access to a compute cluster.
But to be fair, for what I do I don’t really think you can fit that much ram on a consumer motherboard.
In machine learning I don’t generally find memory to be the bottleneck, as long as the GPU can handle it.
But sure, opening huge files can be big trouble.
I can rent cloud stuff, but it’s just not worth it for casual/experimental use.
Like keeping Deepseek V4 loaded and hitting it rarely, but I want it to be in my control… it’s just easier to do it locally.
Or experiments I launch like 30 times before it works. I’d waste hours moving all my stuff to a cloud instance, configuring it, tinkering with the experiment to get it to launch; the thing would be idle the vast majority of the time. And yeah, I know I can containerize stuff, but some projects I can’t even test without a sizable memory pool.
Same. But more than that - I noticed that my 7700x is a huge bottleneck: desktop CPUs limited to just 2 channels, plus controller in 7700 is kinda meh, so I can’t force anything above 5600mt\s with my 4 sticks. Switching to new CPU should allow to increase speed, but will still lock me to 2 channels. So I should’ve bought cheapest threadripper instead, which allows 4 or 8 channels instead.
Depends what you are going for. 7000/9000 series have way more per-thread performance than threadripper.
Another quirk with single CCD 7000 chips (which I have as well) is that the infinity fabric limits effective bandwidth to the CPU cores. Even with two channels, you hit diminishing returns with higher speeds, especially if you aren’t using the IGP.
You can get around it overclocking the IF, but it gets complicated and its much easier to do with 9000 series CPUs.
introducing: zram
toss 32 or 64gb at that and call it a day
YMMV.
Big ZRAM absolutely tanks performance for me because it “steals” space that could be used for normal memory.
But if its for squirrling away a bunch of idle processes when multitasking like crazy, yeah, it can be wonderful. Just depends.
If you’re able to have swap, Zswap is supposed to be better.
Zram is intended for devices with no swap
…This appears to be amazing advice.
https://linuxblog.io/zswap-better-than-zram/
That’s exactly what I was looking for. Thanks!
ah fair enough, worst case scenario if they dont know what to do with that much ram (I would build a homelab) they could also have a ramdisk