USB drive life: fact or fiction?

USB drive life: fact or fiction?

Summary: Flash storage wears out after too many writes, but vendors claim wear-leveling and other tools fix the problem. Can vendor claims be trusted? Researchers at Northeastern University decided to find out. Here's what they found.

SHARE:
TOPICS: Hardware
37

We know flash storage wears out after too many writes. But are vendor claims to be trusted? Researchers at Northeastern University decided to find out. Here's what they found.

Simona Boboila and Peter Desnoyers' Write Endurance in Flash Drives: Measurements and Analysis (pdf) used chip-level testing, reverse engineering, whole device testing and analytic approaches to test 3 different USB thumb drives to see how they hold up.

The problem Flash storage isn't like disk storage, so flash controllers have to go through some intricate contortions to make it look like a disk. The firmware that does that magic is called the Flash Translation Layer or FTL.

USB drives typically use Multi-level cell (MLC) flash that crams 2 or more bits into each cell - at the cost of much lower write specs: 1,000 to 10,000 writes. Generally writing to flash requires that an entire block of flash - 128 KB to 256 KB - be erased and the entire block written again.

You may only want to write a 6 KB text file, but the flash drive may have to write an entire 256 KB block. On a disk you'd just write a few blocks - and besides, magnetism doesn't wear out.

Managing writes or wear-leveling on flash is very important to keep the devices entire capacity available for as long as possible. And since erasing and rewriting flash takes dozens of milliseconds, it is important to maintain a supply of empty blocks to handle incoming writes.

The FTL handles virtualizing the flash capacity, garbage collection, wear-leveling and block management - and there are a lot of tradeoffs involved (see Design tradeoffs for SSD performance for a deep dive).

The life of a flash drive is approximately equal to the chip-level endurance (# of writes) + the intelligence of the FTL.

The tests The team investigated 3 USB drives: a generic 64 MB; a house brand 2 GB drive; and a Memorex branded 512 MB TravelDrive. They wrote all logical blocks - an empty drive doesn't need the erase/write cycle - and then started overwriting pages.

They hooked the flash drives up to a logic analyzer to see what happened between the read/write commands they issued and the actual reading, erasing and writing on the flash chip.

Here's the output from the logic analyzer on a test of the Memorex device:

The results The good news: all 3 drives handled more writes than they were spec'd for. For example, the cheap generic drive handled 77 million writes before failing.

Even better news: the premium Memorex drive did not break! FWIW they predicted failure after 60,000,000,000 writes - they just didn't have time to do that many.

The researchers estimated that Memorex drive's endurance is about 1,000x that of the generic device. Cool!

The bad news: there were several failure modes observed. The most troubling is the silent error where the write operation reports success, but a read finds that the data was not written. Trust, but verify.

Some generic drive writes exhibited blocking, where the write operation hangs indefinitely. Not seen on the Memorex drive.

The Storage Bits take After the Nightmare on DIMM street and Everything you know about disks is wrong, I was expecting to be disappointed by the USB thumb drives. But no: they do what they claim!

Given the astronomical write numbers achieved by even the cheap generic drive, it is clear that 3 and 4 bit MLC flash - whose writes will be as low as 1,000 - will meet consumer expectations. Since they will be much cheaper as well, expect to see flash in even lower-cost devices.

Instead, FTL engineers should look at data integrity and write completion issues, the one area where some of the drives fell short.

But all in all, this study is good news for consumers and merits kudos for USB drive developers and vendors.

Comments welcome, of course.

Topic: Hardware

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

37 comments
Log in or register to join the discussion
  • Your comments got clipped off

    Robin, in the results, your comments on the generic drive seem to have been dropped off. (I'm willing to credit cosmic rays.) Thanks for the good information as always.

    Where it got cut:
    The results
    The good news: all 3 drives handled more writes than they were spec?d for. For example, the cheap generic drive handled over-----???????
    Bill4
    • Yeah...

      I really appreciated the topic, but that is kind of a key number.
      Economister
    • It was a write failure???

      ;-)
      No_Ax_to_Grind
      • Moderator getting carried away?

        I posted a single word: "Funny", because I found this post quite good and amusing. Somehow the moderator did not think it should stay. Would the moderator care to tell us which rule my post violated?
        Economister
    • Fixed!

      It was a PEBCAK* bug.

      *Problem Exists Between Chair And Keyboard

      Cheers,

      Robin
      R Harris
      • We call that a Code 12 error ...

        Problem is 12 inches away from the keyboard ...

        Kudos for admitting ...

        Ludo
        Ludovit
  • One thing I still would like to know

    I kind of do from experiencing it...

    Do the higher end drives (OCZ, Corsair,...) hold up better than the generic or cheaper drives?

    .
    rhonin
    • Generalizations are tricky

      but I would expect the higher end, more expensive drives to have the
      better controllers and more bandwidth that allows them to do what the
      Memorex drive does: last 1000x longer than a generic drive.

      Robin
      R Harris
    • Reliability is important but..

      how do you exceed 77 million writes on a USB thumb drive? Or cut that by a factor of 10 just to be on the safe side: how do you exceed about 8 million writes?

      If you do 100 writes per day, that is about 200 years. I think I will stop worrying about USB thumb drive reliability, although I always use duplicate drives for important data backup.
      Economister
      • They had a program that drove the writes

        as fast as possible. They describe it in the paper.

        Robin
        R Harris
      • Easy to do in the right circumstances..

        Such as Windows Vista/7's Readyboost feature. It might still take a year or two - but it can still happen.
        Wolfie2K3
      • It goes a LOT quicker if you have the drive to . . .

        hold your /tmp or Windows swap-file.

        Still a very good figure.
        hkommedal
    • OCZ 120GB failed....

      Bought OCZ 120GB SSD in July '09, it woke up completely dead Feb '10. So a 7-month lifespan for a ?300 drive.... Unfortunately, the suppliers sent it back to OCZ because it was past the 28-day replacement window - and I have been told I will NOT get a report as to what failed or why, so I'm looking to put it into a non-critical environment (the Guest-Room PC). Shame, as it revitalised my 3-yr old laptop!
      Katsu999
  • RE: USB drive life: fact or fiction?

    I would be interested in more information on the current differences between SLC and MLC USB drives ... I have an SLC that seems to perform much faster ...

    Ludo
    Ludovit
    • SLC is faster

      The fastest USB drive I've used is the Corsair Flash Voyager GT. It was
      about 5x faster than generic USB drives. Don't know that it has SLC, but if
      you have more chips you can have more parallel bandwidth to the chips.

      That is the key to all the fast flash drives: parallel writes to multiple chip
      dies.

      Robin
      R Harris
  • Do SD cards have wear levelling?

    Consider three broad categories of device: SD or other memory cards, USB thumb drives, hard-disk-shaped SSD. I'd speculate that these are successively better engineered, for wear levelling for instance. Furthermore, cells of memory in the same device may not be equal. Designers can anticipate that the file allocation table on a FAT or FAT32 device will be heavily used.

    By the way, I can put an SD card into a USB adapter and see the same files as when I use it directly. I'm assuming that there isn't clever translation going on in the adapter, but I may be wrong.

    So, if I video THE DAILY SHOW onto one card or USB stick each day to watch on my portable device, how many shows can I expect to enjoy? ("About half of them?") And is it worse if I keep one show that I didn't get around to watching in the same storage for a while, so that that memory isn't getting recycled and the rest is working harder?
    Robert Carnegie 2009
    • Some do, but vendors seem reluctant to call it out

      probably to avoid raising questions among the happily unknowing.

      For most people even 1,000 full writes is more than they will ever use.

      For your USB drive, you'll likely see several years worth of service before
      it fails.

      Robin
      R Harris
  • OCZ Rally 32meg failed!

    MY OCZ Rally 32 meg flash drive has failed after about a year of use. First it had some bad sectors, detected by chkdsk after some strange write failures. Now I can't access it all. Windows sees the drive, but reports it has no media in it.
    bmeacham98@...
    • Sometimes....

      drive corruption has caused "failures" for me. I have had success using Active Killdisk to bring them back to life. The fact that Windows can see the drive is somewhat encouraging.

      If the electronics are bad, no SW will be able to revive the drive. If you can lay your hands on a copy of Active Killdisk, it may be worth a try, at least for a 32GB drive. You may be able to wipe everything off the drive (including partitions) and start over. Good luck.

      Edit: Was that 32MB? If so, chuck it. :-)
      Economister
  • I've a 128MB

    Kingston that's been through an industrial washer and dryer - and still works! For sheer robustness and durability you can't beat that.

    Incidentally R.H, have you written [i]any[/i] articles about utilizing flash memory as a bootable device? Formatting flash memory, USB devices in NTFS? Would be interested in any industry developments / breakthroughs in these fields.

    Regards.
    thx-1138_