CPU performance increases have slowed to a crawl. NVM promises - and early benchmarks are remarkable - to turbocharge system performance. This is revolutionary tech.
In January, Neal Christiansen, Microsoft's Principle Development Lead for Storage Class Memory, gave a talk at the SNIA NVM Summit. I'm distilling his comments for brevity.
NVM goes by a host of names, including Storage Class Memory (SCM), Direct Access Storage (DAX), Byte Addressable Storage, and Persistent Memory. Ideally, NVM is a non-volatile storage medium with the low latency and the high bandwidth of DRAM. We haven't reached that ideal yet, but NVM such as 3D XPoint is a massive improvement on SSDs.
System software gets in the way
NVM needs OS support because today's software stacks were designed for disks, despite some updating for SSDs. Disks, for example, require 4k blocks, whereas NVM is byte-addressable, like DRAM.
Another factor is efficiency. No one cared about making I/O stacks efficient when disks took 10 milliseconds to return data. But NVM promises I/O in the sub-microsecond range, and now I/O stack efficiency is vital.
The goals for Microsoft's NVM support can be summed up as don't break anything that doesn't have to be broken. More formally:
- Most existing user-mode applications will run without modification
- Provide an option to support 100% backward compatibility
- Support zero-copy access to persistent memory
New class of volume
To make this is, first, a new kind of volume: a Direct Access Storage (DAX) volume. Its memory mapped files give applications direct access to byte-addressable SCM and share many characteristics with current memory mapped files in Windows.
DAX mode is formatted at volume creation. You can't translate an existing volume into DAX mode, mainly for compatibility reasons. Nor is all existing volume functionality compatible with DAX, but DAXwill be supported by both NTFS and ReFS.
New storage drivers
The SCM Bus Driver is a management driver that keeps track of the physical and logical SCM devices. It is not in the I/O path.
The SCM Disk Drivers provide a storage abstraction layer to the OS for logical SCM devices. These are for specific hardware devices, so there will be both MS and vendor-specific drivers.
These drivers expose SCM byte-addressibility to applications and enable SCM device management.
Memory mapped files on DAX
Memory mapped files (MMF) have been in Windows for more than 20 years. Good news: there are no changes to current memory map APIs. MMFs map directly to SCM hardware on DAX volumes.
This gives true zero-copy access to SCM. Applications have direct access to persistent memory.
However, unlike MMFs on DRAM, which periodically updates pages on the backing disk or SSD, DAX volumes don't generate paging reads or writes because - hey! - SCM is storage. But this impacts services that depend on watching paging for their input.
The biggest change is that the file system no longer knows when writeable MMF sections are modified, so file metadata is only accurate at the time of MMF section creation. Other driver services that can't be supported include:
- NTFS encryption
- NTFS compression
- ReFS integrity stream
- ReFS block cloning
Does this mean you can't encrypt data on DAX? Nope. It just can't be done automagically in the OS I/O stack; external encryption still works.
Block mode volumes
For compatibility, Windows will also offer block mode volumes that maintain current storage semantics. Translation: full compatability with today's applications. All I/Os traverse the storage stack to the SCM driver. Not as fast as DAX, but faster than the current stack.
I/O stacks compared
Micron/Intel are spending billions on 3D XPoint. Microsoft is putting real resources into ensuring NVM works well with Windows.
A block diagram from Mr. Christiansen's deck shows how NVM simplifies the Windows I/O stack:
Combined with NVM's high performance and low latency, the combination gives users good reasons to upgrade hundreds of millions of systems. That's the payoff for vendors.
The Storage Bits take
There's way more to this presentation than I could cover here. If you're a developer - especially of high performance software - it is well worth a half-hour of your time to watch the presentation video. Also, don't freak if you see something you don't like: this is Microsoft's first pass at NVM support, not their last. The engineering team has a wish list they're working on.
If you're a Windows user, watch this space. NVM will be impacting systems much more than SSDs have, and they've been huge.
And what about Mac OS? No hints from Apple, but I suspect that they've so deeply baked the obsolete HFS+ into system services that they will be very late to the NVM party. In less than 2 years Mac power users may finally have a strong reason to move to Windows. Time to surprise the Mac faithful in a good way Apple!
Courteous comments welcome, of course. Update: I reached out to Mr. Christiansen to ensure accuracy, and it turns out there is one major update:
Since that presentation we have renamed "DAS" mode volumes to "DAX" mode volumes. This avoids some ambiguity that people were having with the DAS acronym (people kept thinking of "direct attached storage"). It turns out the Linux is also using the term DAX for their support so this should avoid terminology confusion going forward.
Glad to see the major players converging on terminology. I've corrected all examples above. Also, my fat fingers typed SDM when I meant SCM, which I've also corrected. End update.