The big marketing lie on flash drive performance
When it comes to false advertising I used to think the Wireless LAN industry were the worst offenders, but the flash storage industry seems to be vying for this dubious distinction. With the launch of Windows Vista at the end of this month, flash media performance will become a big issue for ordinary consumers because of Vista's ReadyBoost feature which will boost the performance of Vista. But consumers are in for a rude awakening when they find out they have a flash drive that is incompatible with Vista ReadyBoost because of poor performance. Making matters worse are the ridiculous speed ratings that companies claim on their flash products which really don't translate to actual performance.
Take the following two compact flash cards for example from A-Data and PQI.
I bought these cards looking at the "120x" or "Hi-Speed 100" rating in the advertisement thinking that these must perform much better than normal flash cards that use to have the 40x and 80x on them. I took them home and both of them failed the ReadyBoost performance test which means they can't be used for Vista ReadyBoost.
I am aware of the fact that the 120x rating may be referring to large file transfers used in digital photography applications, but even that theory collapsed when I performed my own throughput and I/O tests showing bottom of the barrel performance. Rob Galbraith runs a photography websiteand he has one of the most complete large-file sequential transfer rate databases on the Internet. When we look at his chart for CF (Compact Flash) cards, we see that the ratings have little to do with actual performance. Even the card reader has a lot of impact on the final performance. My two particular cards were not in these databases but it's safe to say that they would place near the bottom of the pack based on my testing with the card reader I'm using. I even tried an IDE to CF adapter and it was still terribly slow though the results were much better than going through the USB card reader (note that IDE to CF adapters are not supported in Vista though I really wish they would be).
Windows Vista ReadyBoost requires a different kind of performance that requires high IOPS (Input Output Per Second) on small file transfers. If large file sequential transfer can be thought of as drag racing, IOPS can be thought of as street racing in the city with lots of tight turns. AnandTech has this USB flash drive roundup that's gives you an idea how much of a difference there is in flash drive perform. Vista ReadyBoost requires 3.5 MB/s (megabytes per second) on 4 KB random reads and 2.5 MB/s on 512 KB random writes. That translates to a blazing 896 IOPS which is about ten times faster than a typical hard drive. Both of my compact flash cards scored around 2 MB/s for the 4 KB random read test which means they're not even close to being fast enough for Vista ReadyBoost.
From a consumer standpoint it's almost impossible to determine what kind of flash drive they need until they plug it in to their Vista computer and wait for Vista to report ReadyBoost capable or not. This website has a user-submitted database of ReadyBoost compatible devices which may help users decide what kind of flash memory to buy. According to AnandTech's roundup, the Lexar JumpDrive Lighting and Kingston DataTraveler Elite USB drives seem to lead the pack for both read and write performance so it's probably a safe bet to go with those models if you're looking for a ReadyBoost flash drive though they are typically twice as expensive than the cheaper flash cards. USB memory sticks are the safest bet because you don't need to worry about the performance of the card interface. But a memory card isn't the best solution from a form factor perspective because it sticks out of the computer whereas a flash card can be hidden inside the PCMCIA slot.
It would really help if flash drives had a ReadyBoost logo on it guaranteeing ReadyBoost capable performance. Ideally the device would have a ReadyBoost multiplier rating with a 1.0x exactly meeting the 3.5/2.5 read/write MB/s specification and a 2.0x representing double the 3.5/2.5 specification. This would be a great way for flash manufacturers to differentiate themselves if they would advertise these specifications. Unfortunately Vista doesn't report the actual raw data for its ReadyBoost test since it only gives you a pass/fail score making it as useless and confusing as the Vista performance index that's based on a scale of 0 to 6. While it's not Microsoft's job to benchmark the hardware market, it's certainly a missed opportunity to make the user's life easier. [Update 3:30 PM - Ed Bott has a nice follow up to this blog and he reveals how to get Vista to tell you the flash performance details]