Is it time for ATA-over-Ethernet?

When it comes to pushing storage packets around the datacentre, there's one protocol that's king: Fibre Channel (FC -- and they even spell it properly). But everyone is predicting that FC will eventually wither away, for a number of reasons. Is ATA-over-Ethernet going to be a replacement?

Although FC is a high performance technology, ideally suited in many ways for storage-intensive applications such as virtualisation and database access and so is used primarily for connecting servers to storage area networks, it's also expensive and complex. Expensive because the kit that it runs on isn't cheap, of course, but also because it's a separate network that -- because it's complex -- needs quite a bit of careful setting up if you're to get the best from it.

There's a couple of alternatives in the wings, one of which we've heard very little about. The most argued-about is probably Fibre Channel over Ethernet (FCoE), which uses the same protocols and so needs less reconfiguring and re-learning, only the access medium is different. Being Ethernet, it's cheap, ubiquitous, and everyone knows how it works. What's more you can speed it up easily by aggregating multiple channels (which of course you can also do with FC).

But the fastest growing segment of storage in the datacentre is in network attached storage, and a lot of those boxes connect using iSCSI, which wraps SCSI block disk commands in TCP packets and squirts them down the network -- a processor-intensive job.

Recently however, I've been hearing more about ATA over Ethernet (AoE), which has now made it into the Linux kernel. This technology, invented in 2003 by a Californian company Coraid which then open sourced the technology, dispenses with the TCP wrapping, and uses ATA disks instead of SCSI. Proponents say this makes it much much faster and much much cheaper -- and on the face of it this does seem to be true.

But of course there are drawbacks. The first one you hear from enterprise storage folk is that ATA disks aren't as robust as SCSI devices. Maybe so, maybe not -- but since the mechanisms are very similar and ATA disks rotate slower and so are cooler and less stressed, it's hard to tell. And if Google can manage with cheap kit and work its way around the fact there will be failures -- since everything fails eventually, and ATA disks are seriously cheaper than SCSI ones -- maybe it's cheaper to plan for hardware failure rather than try to prevent it. But I digress.

One other issue with AoE, which runs at layer 2 in the network. is that it doesn't route -- although as I understand it, neither does Fibre Channel. Being unroutable makes AoE more secure but also means it's harder to use in distributed environments -- but then, as many in discussion forums have pointed out, how often do you really have to route your storage? I don't know the answer to this -- if you do, it would be good to hear.

Perhaps the biggest problem though is that iSCSI is pretty cheap anyway -- in enterprise storage terms that is, and that the disadvantage of wrapping data in TCP packets is small, given the massive raw processing power of today's servers. And further, AoE technology is only made by a single company which makes many storage managers nervous.

But could there be a future for it? Some customer wins have been announced in Europe, at least one of them in the UK, where Coraid is beefing up its sales and marketing operations. One to keep an eye on -- especially if EMC, HP or Dell buy it...