Microsoft goes public with plans for its new Windows 8 file system

Microsoft goes public with plans for its new Windows 8 file system

Summary: Microsoft officials are sharing some details about the new ReFS file system that will debut first as part of Windows Server 8.

SHARE:

Microsoft officials are finally sharing publicly details about "Protogon," the new file system that the company is developing as part of Windows 8.

Officially named ReFS -- for Resilient File System -- the new file system will be made available via a staged "evolution," according to a January 16 post on the "Building Windows 8" blog.

ReFS will begin life as a storage system for Windows Server only. Then -- some time post Windows 8 -- it will become a storage system for Windows clients, and then ultimately "as a boot volume," said author of the post, Surendra Verma, a development manager on the Windows Storage and File System team. (Interestingly, when the first leaks about ReFS, codenamed Protogon, occurred last year, those who discovered the new file system found it in leaked Windows 8 client builds.)

NTFS, the New Technology File System, has been part of Windows since Windows XP and Windows NT 3.1 were introduced in 2001 and 1993, respectively. (Thanks for the date corrections, readers.)

There are some NTFS features for which Microsoft plans to drop support with ReFS, specifically named streams, object IDs, short names, compression, file level encryption (EFS), user data transactions, sparse, hard-links, extended attributes, and quotas, Verma blogged. That said, one of Microsoft's goals with ReFS is to "maintain a high degree of compatibility with a subset of NTFS features that are widely adopted while deprecating others that provide limited value at the cost of system complexity and footprint," Verma said.

ReFS is designed to complement the Storage Spaces feature in Windows 8 and Windows Server 8. It will help with the verification and auto-correction of data and optimize for scale, according to the post. Here's Verma's explanation of the internals:

"Underneath this reused portion (the code responsible for implementing the Windows file system semantics), the NTFS version of the code-base uses a newly architected engine that implements on-disk structures such as the Master File Table (MFT) to represent files and directories. ReFS combines this reused code with a brand-new engine, where a significant portion of the innovation behind ReFS lies."

It's interesting to hear that Microsoft is doing what very early rumors about Windows 8 claimed it would: Introduce a new file system. (Though to be fair, some early tipsters claimed the Softies planned to do away with the core file system all together in Windows 8....)

Update: As reader Mahesh Sreekandath (@msreekan) noted in the comments below, NTFS, despite its longevity, hasn't always been architected to handle tasks in the most elegant manner. (See Softie Raymond Chen for an example.) Microsoft execs don't want to come out and diss NTFS, since it's the file system inside Windows 7 and Windows Server 2008 R2. But if you're wondering why Microsoft is thinking about reengineering the file system, just know it seemingly could stand some improvement.

Topics: Operating Systems, Microsoft, Software, Windows

About

Mary Jo has covered the tech industry for 30 years for a variety of publications and Web sites, and is a frequent guest on radio, TV and podcasts, speaking about all things Microsoft-related. She is the author of Microsoft 2.0: How Microsoft plans to stay relevant in the post-Gates era (John Wiley & Sons, 2008).

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

122 comments
Log in or register to join the discussion
  • RE: Microsoft goes public with plans for its new Windows 8 file system

    Fascinating overview of the new file system storage mechanisms coming in ReFS.

    Key takeaways:
    * Data integrity is far and away the key priority
    * Ability to recover from many forms of corruption seamlessly and without user intervention
    * Works on partnership with StorageSpaces to provide a holistic reliable, scalable file storage system
    * ReFS comprises the lowest layers of the filesystem so most existing apps will continue to work as expected
    * Pretty much every "limit" (directory size, file size, filename length, etc) are essentially "gone"

    Can't wait to get my hands on this ;)
    bitcrazed
    • RE: Microsoft goes public with plans for its new Windows 8 file system

      @bitcrazed I know what you mean! I'm annoyed we're apparently not getting it in Win8 D=

      Thanks for the list by the way. That's the kind of data I was hoping for, in just the format I prefer. Hah!
      Imrhien
      • RE: Microsoft goes public with plans for its new Windows 8 file system

        @Imrhien - don't be annoyed. Be glad that MS takes the time to bake a change this substantial. The client desktop is a very hostile environment filled with any number of flaky, shaky apps that could trip up if MS changes the on-disk file format.

        Better they bake the server which is generally a much better-managed, cleaner, more demanding environment and then apply any learning & fixes to the client port.

        This is one area where MS should, and will, take a very conservative approach.
        bitcrazed
      • RE: Microsoft goes public with plans for its new Windows 8 file system

        @bitcrazed<br><br>Good point about the apps having to be rewritten to not call for some of those things that Microsoft is going to remove. If they aren't? Flaky and shaky will be the LEAST of the names given to Windows * with ReFS.
        Lerianis10
    • would be interesting

      @bitcrazed .. to have ZDNet or Tech Republic do an in depth side-by-side break-down / comparison between NTFS and ReFS - maybe even a comparison / contrast between ReFS and other common file systems in use: like, say, HFS+, XFS or even ext3. (Possibly Harris or Bott to do so .. maybe both).<br><br>As to your comments, i couldn't agree more: ReFS looks like a great evolutionary step forward in MS file system development. (Echoing the sentiment about removing many hard & soft limits / impediments that previously existed with prior file system architectures).
      thx-1138_
      • RE: Microsoft goes public with plans for its new Windows 8 file system

        @thx-1138_@...

        I think they should compare it more with Btrfs. It's not fully out yet (the fsck utility in particular has yet to surface), but it seems to have a lot of the same underlying technology as ReFS - while at the same time providing a few features that ReFS does not... For example, copy-on-write snapshots.

        I'd say, overall, the five filesystems to be compared are:

        Commonly used already:
        ZFS
        Ext4
        NTFS

        In Development:
        ReFS
        Btrfs

        The reason why I include Ext4 in there, is because it's the de-facto filesystem on Linux at the moment - hence it's good to compare things to how things "Really are," and not just how things "Will be."
        Tynach
    • RE: Microsoft goes public with plans for its new Windows 8 file system

      @bitcrazed Sure hope they re-worked the filter driver hooks, maybe firewall them a bit and make them not so darn difficult to write for the filesystem so that 3rd parties can't bluescreen the box. Seeing that about 99% of the bluescreens I've seen that come from NTFS.sys are all related to some filter driver issue I would think that would be on their radar.
      TGGR
    • Half-baked on rev.v.1.0

      @bitcrazed
      So, let's see, ZFS is on what, version 29 . . . version 30 . . . what ever.... It takes a bit of time to get those types of features baked in and working. Not sure how they can get a lot of them working on v.1.0. Just separating the disk engine from NTFS layer with no features adds is going to be tough enough. It would be like separating the Disk Mangement interface into two separate pieces. The part that is "online", "initialize" and "partition would have to be all in some sort of ZFS like construct with pools, virtual devices (vdev), vdev redundancy (mirror, raid, raw) and replacemnt/repair/scrub tools. And then the NTFS layer with format, drive letter, mount point stuff. Also, for server side, no mention of ZFS style enhancements like SSD log or SSD cache adjunct features.
      eddoaloha
      • It's not ZFS doh doh ...

        @eddoaloha reply by @cracoscosmos -- First of all, it's more like AZFS and second of all I commend your references, though ... !
        crcgraphix
    • RE: Microsoft goes public with plans for its new Windows 8 file system

      @bitcrazed
      You can, it's all been in ZFS since 2005.
      914four
    • RE: Microsoft goes public with plans for its new Windows 8 file system

      @bitcrazed About damn time! I'm tired of only been able to move or copy files that have a certain amounts of characters in them/total path length.
      guiri
    • RE: Microsoft goes public with plans for its new Windows 8 file system

      @bitcrazed One of the most annoying "limits" is the lack of case sensitivity. Fix that and we'll have better interoperability with Linux.
      davidr69
      • RE: Microsoft goes public with plans for its new Windows 8 file system

        @davidr69 NTFS actually has that feature. If the filename is in the POSIX namespace it will not only be case sensitive compared but it will allow many characters not usually legal in Windows filenames.
        Win8AnUglyDisaster
      • RE: Microsoft goes public with plans for its new Windows 8 file system

        @davidr69 Fixing Case Sensitivity will Break most of the Existing Software, All the developers and user's have been ignoring case. Most utilities wont work. Utter Kiosk, unless they can switch it off.
        narenhacker@...
      • RE: Microsoft goes public with plans for its new Windows 8 file system

        @davidr69
        One of the most annoying features of Linux et al is the case sensitivity. It's a real pain.
        Rather than fixing the biggest operating system to work better with something that only geeks use, why not start a movement to make Linux have a non-case-sensitive mode of operation?
        JoCaBa
      • RE: Microsoft goes public with plans for its new Windows 8 file system

        @JoCaBa - Ha ha, only from a Windows zealot. All other operating systems have had case sensitivity from their beginning EXCEPT Windows (Unix, OS X, Mac 6-9, Posix, Vax, OS/2, AIX, even Commodore). MS and Windows developers need to get their act together.
        The Danger is Microsoft
      • RE: Microsoft goes public with plans for its new Windows 8 file system

        IMHO, I don't see any value in having case-sensitive names for file system paths. Can you name some good reasons to provide this feature? Especially on the consumer side, this could irritate many users since they usually don't have a high level of technical understanding and almost take it for granted that writing "C:\Temp" is the same as "c:\temp". It's not very practical to make a difference in casing because it creates a complexity that is simply not useful in many cases.
        sevenacids
      • RE: Microsoft goes public with plans for its new Windows 8 file system

        @davidr69,
        Oh...I hate case sensitivity on file systems. Do you really need a folder name 'Foo' and another name 'foo'?
        bmonsterman
    • Why all the gushing?

      @bitcrazed<br><br>There's arguably as many takeaways and potential pitfalls to consider as there are upsides, especially this early on. As it is, you're simply bargaining to be is a guinea pig for the round one release. A word of caution: Walk, don't run.<br><br>Real-world testing is done initially on better managed server environments to provide more reliable baseline data from which to build on, beyond other pragmatic considerations. It also speaks to the prioritization of their targets.<br><br>But servers as a "more demanding environment" than the aggregate maelstrom client side, workstation and desktop pinata are subjected to with their relentless 360 degree, 20 fingers 'n' all thumbs, dive bombing + flagellation assaults? You may need to rethink that through, now that we've reached multicore mayhem.
      klumper
      • RE: Microsoft goes public with plans for its new Windows 8 file system

        @klumper
        Dude, you should write a book. I felt like I was reading an extremely short novel. :-)

        Well said...
        rpollard@...