Disk performance hits the wall

Disk performance hits the wall

Summary: Disk drives have come a long way from the massive rotating plates of the 1950s and 1960s, when 200MB was a huge volume, to today when disk drives containing 2TB and up are considered commonplace.But this massive improvement has a little-considered dark side.

SHARE:
TOPICS: Networking
0

Disk drives have come a long way from the massive rotating plates of the 1950s and 1960s, when 200MB was a huge volume, to today when disk drives containing 2TB and up are considered commonplace.

But this massive improvement has a little-considered dark side. It's not so much a problem for desktops but for an enterprise, the exponential growth in capacity generates huge performance problems. Speed simply hasn't increased at anything like the same rate, as this fascinating paper from IBM points out.

The reason this is a problem is that doing anything with the huge amounts of data that enterprises are now storing takes longer and longer. We're talking about tasks such as database indexing, backing up, data migrations -- anything in fact that involves large numbers of transactions.

That's because a disk that contains 2x as much data as that of three years ago is in no way close to twice as quick. In fact, IBM's paper, GPFS Scans 10 Billion Files in 43 Minutes, published by its advanced storage lab in San Jose, California, shows that latency has remained unchanged for at least five years, while the rest of the performance metrics are improving at modest rates of between five and 20 per cent annually. They're not keeping pace with Moore's Law.

The traditional way of fixing that problem is to add more spindles, which boosts throughput but also adds capacity, often where it's not needed and not required -- and adding spindles also adds complexity, cost, heat and likelihood of failure.

SSDs would seem to be the way out of this as they clearly don't suffer from the same problems but the file system is also a crucial component. With the growth of object storage and the like, this is just the start of a very interesting technological voyage. With the availability now of high-end file systems such as ZFS for free, it might not be too long before end-users can all enjoy the benefits too.

Topic: Networking

Manek Dubash

About Manek Dubash

Editor, journalist, analyst, presenter and blogger.


As well as blogging and writing news & features here on ZDNet, I work as a cloud analyst with STL Partners, and write for a number of other news and feature sites.


I also provide research and analysis services, video and audio production, white papers, event photography, voiceovers, event moderation, you name it...


Back story
An IT journalist for 25+ years, I worked for Ziff-Davis UK for almost 10 years on PC Magazine, reaching editor-in-chief. Before that, I worked for a number of other business & technology publications and was published in national and international titles.

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

0 comments
Log in or register to start the discussion