One of the biggest bottlenecks in many workloads is latency. Cache miss and the CPU stalls waiting for main memory. Flash storage, even on an nvme bus is two orders of magnitude slower than ram.
For example L3 cache takes approximately 10-20 nano seconds, ram takes closer to 100 nano seconds, nvme flash is more than 10,000 nano seconds (>10 microseconds).
Depending on your age you may remember the transition from hard drives to ssds. They could make a machine feel much snappier. Early PC ssds weren’t significantly faster throughput than hard drives (many now are even slower writing when they run out of SLC cache), what they were is significantly lower latency.
As an aside, Intel and Microns 3d xpoint was super interesting technically. It was capable of < 5000 nano seconds in early generation parts, meaning it sat in between DDR ram and flash.
One of the biggest bottlenecks in many workloads is latency. Cache miss and the CPU stalls waiting for main memory. Flash storage, even on an nvme bus is two orders of magnitude slower than ram.
For example L3 cache takes approximately 10-20 nano seconds, ram takes closer to 100 nano seconds, nvme flash is more than 10,000 nano seconds (>10 microseconds).
Depending on your age you may remember the transition from hard drives to ssds. They could make a machine feel much snappier. Early PC ssds weren’t significantly faster throughput than hard drives (many now are even slower writing when they run out of SLC cache), what they were is significantly lower latency.
As an aside, Intel and Microns 3d xpoint was super interesting technically. It was capable of < 5000 nano seconds in early generation parts, meaning it sat in between DDR ram and flash.
Arent there nv-ram dimms using a sort of hybrid?