Is it time for ATA-over-Ethernet?

Is it time for ATA-over-Ethernet?

Summary: When it comes to pushing storage packets around the datacentre, there's one protocol that's king: Fibre Channel (FC -- and they even spell it properly). But everyone is predicting that FC will eventually wither away, for a number of reasons.

TOPICS: Networking

When it comes to pushing storage packets around the datacentre, there's one protocol that's king: Fibre Channel (FC -- and they even spell it properly). But everyone is predicting that FC will eventually wither away, for a number of reasons. Is ATA-over-Ethernet going to be a replacement?

Although FC is a high performance technology, ideally suited in many ways for storage-intensive applications such as virtualisation and database access and so is used primarily for connecting servers to storage area networks, it's also expensive and complex. Expensive because the kit that it runs on isn't cheap, of course, but also because it's a separate network that -- because it's complex -- needs quite a bit of careful setting up if you're to get the best from it.

There's a couple of alternatives in the wings, one of which we've heard very little about. The most argued-about is probably Fibre Channel over Ethernet (FCoE), which uses the same protocols and so needs less reconfiguring and re-learning, only the access medium is different. Being Ethernet, it's cheap, ubiquitous, and everyone knows how it works. What's more you can speed it up easily by aggregating multiple channels (which of course you can also do with FC).

But the fastest growing segment of storage in the datacentre is in network attached storage, and a lot of those boxes connect using iSCSI, which wraps SCSI block disk commands in TCP packets and squirts them down the network -- a processor-intensive job.

Recently however, I've been hearing more about ATA over Ethernet (AoE), which has now made it into the Linux kernel. This technology, invented in 2003 by a Californian company Coraid which then open sourced the technology, dispenses with the TCP wrapping, and uses ATA disks instead of SCSI. Proponents say this makes it much much faster and much much cheaper -- and on the face of it this does seem to be true.

But of course there are drawbacks. The first one you hear from enterprise storage folk is that ATA disks aren't as robust as SCSI devices. Maybe so, maybe not -- but since the mechanisms are very similar and ATA disks rotate slower and so are cooler and less stressed, it's hard to tell. And if Google can manage with cheap kit and work its way around the fact there will be failures -- since everything fails eventually, and ATA disks are seriously cheaper than SCSI ones -- maybe it's cheaper to plan for hardware failure rather than try to prevent it. But I digress.

One other issue with AoE, which runs at layer 2 in the network. is that it doesn't route -- although as I understand it, neither does Fibre Channel. Being unroutable makes AoE more secure but also means it's harder to use in distributed environments -- but then, as many in discussion forums have pointed out, how often do you really have to route your storage? I don't know the answer to this -- if you do, it would be good to hear.

Perhaps the biggest problem though is that iSCSI is pretty cheap anyway -- in enterprise storage terms that is, and that the disadvantage of wrapping data in TCP packets is small, given the massive raw processing power of today's servers. And further, AoE technology is only made by a single company which makes many storage managers nervous.

But could there be a future for it? Some customer wins have been announced in Europe, at least one of them in the UK, where Coraid is beefing up its sales and marketing operations. One to keep an eye on -- especially if EMC, HP or Dell buy it...

Topic: Networking

Manek Dubash

About Manek Dubash

Editor, journalist, analyst, presenter and blogger.

As well as blogging and writing news & features here on ZDNet, I work as a cloud analyst with STL Partners, and write for a number of other news and feature sites.

I also provide research and analysis services, video and audio production, white papers, event photography, voiceovers, event moderation, you name it...

Back story
An IT journalist for 25+ years, I worked for Ziff-Davis UK for almost 10 years on PC Magazine, reaching editor-in-chief. Before that, I worked for a number of other business & technology publications and was published in national and international titles.

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.


Log in or register to join the discussion
  • While there is a theoretical performance advantage over iSCSI the main reason I stick with using AoE for the last 6 years is the simplicity: It is absolutely trivial to set up. Load the AoE kernel module on one system, start up vblade on another, they automatically find each other and you've got a SAN. While Coraid invented AoE it has made absolutely all of the necessary technology available as Free Software under the GPL license. Alternative implementations of the AoE target exist. And if you aren't happy with that, the RFC is only 8 pages long: The protocol is simple, roll your own. Unlikely, sure, but certainly doable. But simplicity is definitely the reason I chose AoE over iSCSI.
  • Hi Manek, nice article. I do want to clarify one item: Coraid arrays do not use ATA disks. Rather, the arrays provide the flexibility to run SSD, SAS, or SATA drives, even mixed into the same array.

    In other words, the AoE protocol in no way requires that you use ATA drives. Rather, it provides very simple, low overhead data transport between the server and the array.
  • Aoe isn't routable but you can encapsulate in a routable protocol (MPLS for instance). Coraid even sells appliance that adds the capacity to do asynchronous replication to a remote site.
    99% of the time you don't root you SAN traffic so IMHO it's better to have a non-routable protocol (without the overhead of a routable one), that you can encapsulate in a routable one if needed.
    Jim Profit