USB drive life: fact or fiction?

We know flash storage wears out after too many writes. But are vendor claims to be trusted? Researchers at Northeastern University decided to find out. Here's what they found.

Simona Boboila and Peter Desnoyers' Write Endurance in Flash Drives: Measurements and Analysis (pdf) used chip-level testing, reverse engineering, whole device testing and analytic approaches to test 3 different USB thumb drives to see how they hold up.

The problem Flash storage isn't like disk storage, so flash controllers have to go through some intricate contortions to make it look like a disk. The firmware that does that magic is called the Flash Translation Layer or FTL.

USB drives typically use Multi-level cell (MLC) flash that crams 2 or more bits into each cell - at the cost of much lower write specs: 1,000 to 10,000 writes. Generally writing to flash requires that an entire block of flash - 128 KB to 256 KB - be erased and the entire block written again.

You may only want to write a 6 KB text file, but the flash drive may have to write an entire 256 KB block. On a disk you'd just write a few blocks - and besides, magnetism doesn't wear out.

Managing writes or wear-leveling on flash is very important to keep the devices entire capacity available for as long as possible. And since erasing and rewriting flash takes dozens of milliseconds, it is important to maintain a supply of empty blocks to handle incoming writes.

The FTL handles virtualizing the flash capacity, garbage collection, wear-leveling and block management - and there are a lot of tradeoffs involved (see Design tradeoffs for SSD performance for a deep dive).

The life of a flash drive is approximately equal to the chip-level endurance (# of writes) + the intelligence of the FTL.

The tests The team investigated 3 USB drives: a generic 64 MB; a house brand 2 GB drive; and a Memorex branded 512 MB TravelDrive. They wrote all logical blocks - an empty drive doesn't need the erase/write cycle - and then started overwriting pages.

They hooked the flash drives up to a logic analyzer to see what happened between the read/write commands they issued and the actual reading, erasing and writing on the flash chip.

Here's the output from the logic analyzer on a test of the Memorex device:

Even better news: the premium Memorex drive did not break! FWIW they predicted failure after 60,000,000,000 writes - they just didn't have time to do that many.

The researchers estimated that Memorex drive's endurance is about 1,000x that of the generic device. Cool!

The bad news: there were several failure modes observed. The most troubling is the silent error where the write operation reports success, but a read finds that the data was not written. Trust, but verify.

Some generic drive writes exhibited blocking, where the write operation hangs indefinitely. Not seen on the Memorex drive.

The Storage Bits take After the Nightmare on DIMM street and Everything you know about disks is wrong, I was expecting to be disappointed by the USB thumb drives. But no: they do what they claim!

Given the astronomical write numbers achieved by even the cheap generic drive, it is clear that 3 and 4 bit MLC flash - whose writes will be as low as 1,000 - will meet consumer expectations. Since they will be much cheaper as well, expect to see flash in even lower-cost devices.

Instead, FTL engineers should look at data integrity and write completion issues, the one area where some of the drives fell short.

But all in all, this study is good news for consumers and merits kudos for USB drive developers and vendors.

Comments welcome, of course.