Are you ready for 4k sector drives?

By | December 20, 2009, 6:53pm PST

Summary: The industry is shipping new 4k drives that drop the ancient 512 byte disk sector for a 4096 byte - 4k - sector. What’s in it for you? And what will it do to you? Here’s what you need to know.

WD has started shipping drives that drop the ancient 512 byte disk sector for a 4096 byte - 4k - sector. What’s in it for you? And what will it do to you?

Disks write your data in fixed chunks. For several decades those chunks have been almost always been 512 byte sectors (some vendors have played with other sizes - such as 520 bytes - and irritated their customers no end). But the industry is beginning to phase in a 4k block. Here’s what you need to know.

Today’s sectors look like this:

courtesy Western Digital

courtesy Western Digital

Why?
Rising bit density means smaller magnetic areas and more noise. The underlying or raw disk media error rate is approaching 1 error in every thousand bits on average - while tiny media defects can lose hundreds of bytes in a row. The larger sectors make it easier to fix those gaps.

The raw error rate is cleaned up by the sector error correcting code (ECC) and sophisticated signal processing to reach a SATA drive’s specified error rate of 1 in 1014. The magnetic spots are smaller than 45nm transistors and they’re spinning 120 times a second. Without robust ECC disk storage wouldn’t work.

Why now?
A 512 byte sector can’t support enough ECC to correct for increasing raw error rates. Thus bigger sectors with stronger ECC capable of detecting and correcting much larger errors - up to 400 bytes on a 4k sector.

Here’s a diagram of the 4k sector. Less protocol overhead and better ECC.

courtesy Western Digital

courtesy Western Digital

Note: the longer ECC doesn’t change the drive level read error rate. It remains at 1 unrecoverable read error about every 12.5 TB.

Is this new?
4k sectors have been cooking for over a decade. Drive vendors started by convincing Microsoft and other OS and BIOS vendors of the problem years ago.

The late adopters are the cloning software vendors. More on that in a moment.

Why should I care?
Would you like a 4 TB disk? 8 TB?

The 4k sector enables disk manufacturers to keep cramming more bits on a disk. Without them the annual 40% capacity increases we’ve come to expect would stop.

What about performance?
Marginal, invisible-to-the-naked-eye improvements. But it won’t be worse, either.

Will 4k sectors use capacity faster?
If you write 500 bytes and the minimum sector is 4k, will that write take up the full 4k, wasting 3.5 KB? No.

The initial WD drives - and I assume other vendors as well - will operate in a 512 byte emulation mode. Eventually new disks will operate in native 4k mode, and then you might have a concern. But many operating systems already do 4k IO, and at a couple of cents per future GB, who cares?

The emulation mode puts 8 512 byte writes into a single 4k sector. There is no loss at all. Here’s a picture:

courtesy Western Digital

courtesy Western Digital

Gotchas?
If you are in either of these 2 groups:

  1. Windows XP users
  2. Windows users who clone disks with software like Norton Ghost

there are a couple of gotchas if you want to use a 4k drive. Since most drives aren’t 4k and won’t be for another year or more, this may not affect you either. Vista and W7 users are cool except for cloning.

1) Windows XP does not automatically align writes on 4k boundaries, which hurts performance. WD has software - the Advanced Format Align Utility for their drives. I assume other vendors will too when they start shipping.

XP users need to run this utility once to use a 4k drive with a clean install, cloning software or a do-it-yourself USB drive. WD-branded 4k USB drives are already aligned so it isn’t needed for those drives.

2) Windows clone software vendors have yet to implement 4k support. If you clone an XP, Vista or W7 drive you should run the align utility. The cloning vendors need to get on board Real Soon Now. Vendors are welcome to comment on their plans.

What about Macs?
No worries: Mac OS just works with 4k drives - including cloning.

How can I recognize a 4k drive?
WD is labeling theirs and I assume other vendors will follow suit. Here’s WD’s label:

The Storage Bits take
There’s been a lot of heavy lifting behind the scenes to make this transition as smooth as possible. With Vista, W7, Mac OS and Linux support well in hand most users won’t notice any change.

XP users have a bit more to be aware of and some will get bit by performance issues. The easiest solution for XP users: avoid installing 4k drives. Factory installed XP will be fine.

My question: why not a better read-error spec? Today’s large SATA drives shouldn’t be used in 4 drive RAID 5 arrays due to the high likelihood of a read error after a drive failure, which will abort the RAID rebuild. A better error spec would fix this.

Comments welcome, of course. WD’s dynamic Heather Skinner arranged my briefing. No sectors, old or new, changed hands.

Kick off your day with ZDNet's daily e-mail newsletter. It's the freshest tech news and opinion, served hot. Get it.

Topics

Robin Harris has been messing with computers for over 30 years and selling and marketing data storage for over 20 in companies large and small.

Disclosure

Robin Harris

Robin Harris is a president of TechnoQWAN, a consulting and analyst firm in northern Arizona. He also writes StorageMojo.com, a blog which accepts advertising from companies in the storage industry, and has a 25 year history with IT vendors. He has many industry contacts, many of whom are friends and all of whom he has opinions about. Robin has relationships with many companies in the technology industry. Every company he writes about may have sought to influence his opinion through carefully-crafted marketing messages and self-serving white papers, gifts ranging from desk calendars, t-shirts, lunches and trips as well as analyst or consulting assignments. He also invests in some technology companies. He may accept payment for services in stock as well. Robin discloses financial investments in or client relationships with companies named in Storage Bits. To help readers sort out the gold from the dross in his writings, Robin tries to communicate his reasons as clearly as he can. If you agree, you are intelligent and discerning. If you disagree, well, you disagree. In all cases, Robin encourages readers to subject everything they read, see or hear on the internet or from politicians to some simple questions: * What assumptions are implicit in the world view and judgments of the author? * What, if any, is the factual basis for the opinions the author expresses? * Is it reasonable, logical and clear? Your critical faculties: use ‘em or lose ‘em!

Biography

Robin Harris

Harris has been messing with computers for over 30 years and selling and marketing data storage for over 20 in companies large and small. He introduced a couple of multi-billion dollar storage products (DLT, the first Fibre Channel array) to market, as well as a many smaller ones. Earlier he spent 10 years marketing servers and networks. After leaving corporate life he founded TechnoQWAN, a consulting and analyst firm. He also developed StorageMojo into one of the top storage industry blogs.

Robin writes, consults, coaches and lives among the mountains of northern Arizona.

Talkback Most Recent of 40 Talkback(s)

  • Good Blog
    The hard disk makers have been crawling in this direction for a few years, looks like we finally got there.
    On the error correction side, this is alway a 3 way performance trade-off, speed, computational complexity, and power.

    More info on WD Hdrive at
    http://www.wdc.com/wdproducts/library/whitepapers/en/2579-771430.pdf
    ZDNet Gravatar
    Agnostic_OS
    20th Dec 2009
  • Its not just about the ECC
    I think the biggest reason to make the move to 4k sectors
    isn't just the ECC. Its also about data density on the
    disk itself. If you add up all the overhead around the
    512 sector sizes (Sync/DAM, sector gap, etc) compared to
    that around 4k sectors there is less "wasted" disk and so
    inevitably means you can squeeze more writeable space out
    of a disk.
    ZDNet Gravatar
    garethmcc
    21st Dec 2009
  • ECC contributes significantly to that "wasted" disk space.
    Your argument and that about ECC are basically one and the same.
    ZDNet Gravatar
    ye
    21st Dec 2009
  • Well based on todays drives...
    ECC is not wasted space, if you were to watch the amount of ECC corrections being done on your drive hourly, you would change your mind about wasted space.
    ZDNet Gravatar
    mrlinux
    21st Dec 2009
  • ZDNet Gravatar
    ye
    21st Dec 2009
  • Give this guy a payraise
    An excellent technical story and a serious problem plaguing the industry.
    ZDNet Gravatar
    someitguy79
    21st Dec 2009
  • Nicely done Robin!
    and Merry XMAS!
    ZDNet Gravatar
    D T Schmitz
    21st Dec 2009
  • The last comment is the most important of the article
    The fact is that the error rate specification has not significantly tightened even as the volumes of data has risen by orders of magnitude. This has already created some big problems for SAN designers but at least they are aware of it. Small business and home users are not aware of it, and don't even know that its causing them problems.

    The specification needs to be tightened, and new operating systems and file systems need to take into account that these errors must be dealt with in a better fashion than what we see today.
    ZDNet Gravatar
    terry flores
    21st Dec 2009
  • There is at least one file system out there that can help.
    ZFS, used in OpenSolaris as well as Sun Solaris 10 has error correcting at the byte level. This allows it to avoid issues around silent corruption, ghost writes and other basic forms of data loss, unlike UFS, JFS (all variations), NTFS, HFS/HFS+ and most other common file systems in use today. If you want to protect your Windows Server environments from silent corruption (2 bits flipping in a byte because of crosstalk on the magnetic medium) you can run your existing servers in xVM on top of OpenSolaris on just about any Intel box and get the best of both worlds, Microsoft compatibility on top of a rock solid file system designed this century. Or, as Robin suggested once, you can take a look at Sun's 7000 series storage.
    ZDNet Gravatar
    914four
    21st Dec 2009
  • asus and opensolaris
    Be certain to make sure that opensolaris supports your hardware. Asus says that they do not support opensolaris.
    ZDNet Gravatar
    rws0205@...
    21st Dec 2009
  • ZDNet Gravatar
    914four
    23rd Dec 2009
  • Cluster size is already 4k for most folks
    In NTFS any way clusters are already 4k for most modern computers (with 200+ GByte drives) so the smallest allocation is already 4k. Having a 4k sector without any shenanigans just makes it impossible to go below a 4k cluster. Alignment would certainly be an issue but with alignment on 4k boundaries most people would not notice any increase in slack space as since the file system already typically allocates a minimum of 4k. You can do less (on 512K sectors) but then more memory is required to cache disk info.
    ZDNet Gravatar
    DevGuy_z
    21st Dec 2009
  • RE: Are you ready for 4k sector drives?
    The size of a Windows virus, which hides in the unused portions of the final disk sector (cavitation virus) will go up immensely!
    ZDNet Gravatar
    tburzio
    21st Dec 2009
  • An example of what Zdnet should be
    Great write up. I will coward this link to me friends.
    ZDNet Gravatar
    Bruizer
    21st Dec 2009
  • ZDNet Gravatar
    pgit
    21st Dec 2009

Talkback - Tell Us What You Think

Formatting +
BB Codes - Note: HTML is not supported in forums
  • [b] Bold [/b]
  • [i] Italic [/i]
  • [u] Underline [/u]
  • [s] Strikethrough [/s]
  • [q] "Quote" [/q]
  • [ol][*] 1. Ordered List [/ol]
  • [ul][*] · Unordered List [/ul]
  • [pre] Preformat [/pre]
  • [quote] "Blockquote" [/quote]

The best of ZDNet, delivered

ZDNet Newsletters

Get the best of ZDNet delivered straight to your inbox

Facebook Activity

White Papers, Webcasts, & Resources