Are you ready for 4k sector drives?

Are you ready for 4k sector drives?

Summary: The industry is shipping new 4k drives that drop the ancient 512 byte disk sector for a 4096 byte - 4k - sector. What's in it for you? And what will it do to you? Here's what you need to know.

SHARE:

WD has started shipping drives that drop the ancient 512 byte disk sector for a 4096 byte - 4k - sector. What's in it for you? And what will it do to you?

Disks write your data in fixed chunks. For several decades those chunks have been almost always been 512 byte sectors (some vendors have played with other sizes - such as 520 bytes - and irritated their customers no end). But the industry is beginning to phase in a 4k block. Here's what you need to know.

Today's sectors look like this:

courtesy Western Digital

courtesy Western Digital

Why? Rising bit density means smaller magnetic areas and more noise. The underlying or raw disk media error rate is approaching 1 error in every thousand bits on average - while tiny media defects can lose hundreds of bytes in a row. The larger sectors make it easier to fix those gaps.

The raw error rate is cleaned up by the sector error correcting code (ECC) and sophisticated signal processing to reach a SATA drive's specified error rate of 1 in 1014. The magnetic spots are smaller than 45nm transistors and they're spinning 120 times a second. Without robust ECC disk storage wouldn't work.

Why now? A 512 byte sector can't support enough ECC to correct for increasing raw error rates. Thus bigger sectors with stronger ECC capable of detecting and correcting much larger errors - up to 400 bytes on a 4k sector.

Here's a diagram of the 4k sector. Less protocol overhead and better ECC.

courtesy Western Digital

courtesy Western Digital

Note: the longer ECC doesn't change the drive level read error rate. It remains at 1 unrecoverable read error about every 12.5 TB.

Is this new? 4k sectors have been cooking for over a decade. Drive vendors started by convincing Microsoft and other OS and BIOS vendors of the problem years ago.

The late adopters are the cloning software vendors. More on that in a moment.

Why should I care? Would you like a 4 TB disk? 8 TB?

The 4k sector enables disk manufacturers to keep cramming more bits on a disk. Without them the annual 40% capacity increases we've come to expect would stop.

What about performance? Marginal, invisible-to-the-naked-eye improvements. But it won't be worse, either.

Will 4k sectors use capacity faster? If you write 500 bytes and the minimum sector is 4k, will that write take up the full 4k, wasting 3.5 KB? No.

The initial WD drives - and I assume other vendors as well - will operate in a 512 byte emulation mode. Eventually new disks will operate in native 4k mode, and then you might have a concern. But many operating systems already do 4k IO, and at a couple of cents per future GB, who cares?

The emulation mode puts 8 512 byte writes into a single 4k sector. There is no loss at all. Here's a picture:

courtesy Western Digital

courtesy Western Digital

Gotchas? If you are in either of these 2 groups:

  1. Windows XP users
  2. Windows users who clone disks with software like Norton Ghost

there are a couple of gotchas if you want to use a 4k drive. Since most drives aren't 4k and won't be for another year or more, this may not affect you either. Vista and W7 users are cool except for cloning.

1) Windows XP does not automatically align writes on 4k boundaries, which hurts performance. WD has software - the Advanced Format Align Utility for their drives. I assume other vendors will too when they start shipping.

XP users need to run this utility once to use a 4k drive with a clean install, cloning software or a do-it-yourself USB drive. WD-branded 4k USB drives are already aligned so it isn't needed for those drives.

2) Windows clone software vendors have yet to implement 4k support. If you clone an XP, Vista or W7 drive you should run the align utility. The cloning vendors need to get on board Real Soon Now. Vendors are welcome to comment on their plans.

What about Macs? No worries: Mac OS just works with 4k drives - including cloning.

How can I recognize a 4k drive? WD is labeling theirs and I assume other vendors will follow suit. Here's WD's label:

The Storage Bits take There's been a lot of heavy lifting behind the scenes to make this transition as smooth as possible. With Vista, W7, Mac OS and Linux support well in hand most users won't notice any change.

XP users have a bit more to be aware of and some will get bit by performance issues. The easiest solution for XP users: avoid installing 4k drives. Factory installed XP will be fine.

My question: why not a better read-error spec? Today's large SATA drives shouldn't be used in 4 drive RAID 5 arrays due to the high likelihood of a read error after a drive failure, which will abort the RAID rebuild. A better error spec would fix this.

Comments welcome, of course. WD's dynamic Heather Skinner arranged my briefing. No sectors, old or new, changed hands.

Topics: Software, Hardware, Microsoft, Operating Systems, Windows

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

40 comments
Log in or register to join the discussion
  • Good Blog

    The hard disk makers have been crawling in this direction for a few years, looks like we finally got there.
    On the error correction side, this is alway a 3 way performance trade-off, speed, computational complexity, and power.

    More info on WD Hdrive at
    http://www.wdc.com/wdproducts/library/whitepapers/en/2579-771430.pdf
    Agnostic_OS
  • Its not just about the ECC

    I think the biggest reason to make the move to 4k sectors
    isn't just the ECC. Its also about data density on the
    disk itself. If you add up all the overhead around the
    512 sector sizes (Sync/DAM, sector gap, etc) compared to
    that around 4k sectors there is less "wasted" disk and so
    inevitably means you can squeeze more writeable space out
    of a disk.
    garethmcc
    • ECC contributes significantly to that "wasted" disk space.

      Your argument and that about ECC are basically one and the same.
      ye
      • Well based on todays drives...

        ECC is not wasted space, if you were to watch the amount of ECC corrections being done on your drive hourly, you would change your mind about wasted space.
        mrlinux
        • Perhaps you missed the quotations around the word wasted.

          They were put there for a reason.
          ye
  • Give this guy a payraise

    An excellent technical story and a serious problem plaguing the industry.
    someitguy79
  • Nicely done Robin!

    and Merry XMAS!
    D T Schmitz
  • The last comment is the most important of the article

    The fact is that the error rate specification has not significantly tightened even as the volumes of data has risen by orders of magnitude. This has already created some big problems for SAN designers but at least they are aware of it. Small business and home users are not aware of it, and don't even know that its causing them problems.

    The specification needs to be tightened, and new operating systems and file systems need to take into account that these errors must be dealt with in a better fashion than what we see today.
    terry flores
    • There is at least one file system out there that can help.

      ZFS, used in OpenSolaris as well as Sun Solaris 10 has error correcting at the byte level. This allows it to avoid issues around silent corruption, ghost writes and other basic forms of data loss, unlike UFS, JFS (all variations), NTFS, HFS/HFS+ and most other common file systems in use today. If you want to protect your Windows Server environments from silent corruption (2 bits flipping in a byte because of crosstalk on the magnetic medium) you can run your existing servers in xVM on top of OpenSolaris on just about any Intel box and get the best of both worlds, Microsoft compatibility on top of a rock solid file system designed this century. Or, as Robin suggested once, you can take a look at Sun's 7000 series storage.
      914four
      • asus and opensolaris

        Be certain to make sure that opensolaris supports your hardware. Asus says that they do not support opensolaris.
        rws0205@...
        • Check out:

          http://www.sun.com/bigadmin/hcl/data/os/
          for the OpenSolaris HCL.
          914four
  • Cluster size is already 4k for most folks

    In NTFS any way clusters are already 4k for most modern computers (with 200+ GByte drives) so the smallest allocation is already 4k. Having a 4k sector without any shenanigans just makes it impossible to go below a 4k cluster. Alignment would certainly be an issue but with alignment on 4k boundaries most people would not notice any increase in slack space as since the file system already typically allocates a minimum of 4k. You can do less (on 512K sectors) but then more memory is required to cache disk info.
    DevGuy_z
  • RE: Are you ready for 4k sector drives?

    The size of a Windows virus, which hides in the unused portions of the final disk sector (cavitation virus) will go up immensely!
    tburzio
  • An example of what Zdnet should be

    Great write up. I will coward this link to me friends.
    Bruizer
  • Robin is the reason I still read ZDNet, well done yet again (nt)

    (nt)
    pgit
  • I am ready.

    I use Linux. Linux already supports it.
    Subsentient
  • RE: Are you ready for 4k sector drives?

    Brilliant!
    This is what ZDNet should be.
    Robin has written an excellent overview.
    Lets have more of this standard!
    Bob Harris UK
  • RE: Are you ready for 4k sector drives?

    Great piece! I have questions not answered:

    First, I was not remotely aware that "Today?s large SATA drives shouldn?t be used in 4 drive RAID 5 arrays..."

    How large is "large"? Where can I fine more info on this subject?

    Second, what are the implications for 64bit OSs and Apps computing?

    For apps like video editing (an a 64 bit OS and 64 bit Adobe video apps), is 4k sectoring going to increase performance?

    Is this bettter for the anticipated new 6.0 Gbps SATA drives?
    wsmith@...
    • Some answers . . .

      Large SATA drive issue: due to the error rate you have about a 40%
      chance of an aborted rebuild with a RAID5 4 2TB drive config. See
      http://blogs.zdnet.com/storage/?p=483 for more info.

      "Large" isn't black & white, but 1 TB is my limit.

      No implications for 64 bit OS that I see. Performance may be a tiny bit
      better, but nothing you'll notice.

      It will be years before data rates off the head exceeds 3Gbps. The
      advantage to 6Gbps is that the larger 64MB buffers many disks are
      sporting can be emptied faster.

      Robin
      R Harris
  • RE: Are you ready for 4k sector drives?

    Now all they have to do is get people to buy them....
    not for me for some time yet (12months +)
    ronangel