A SAN for the rest of us

A SAN for the rest of us

Summary: Fibre Channel to costly? iSCSI too slow?In a past life I was the product manager for the industry's first full Fibre Channel array.

TOPICS: Storage, Hardware

Fibre Channel to costly? iSCSI too slow? In a past life I was the product manager for the industry's first full Fibre Channel array. We had great hopes for FC as a storage-optimized network that would bring the power of network economics to drive down storage costs. My bad.

Hijacked by vendors, FC has evolved into a costly and fragile mess. It is the fastest block-based storage interconnect, so it isn't going to disappear tomorrow, but the FC market has peaked. iSCSI is coming on strong, but for smaller SANs that need performance the IP overhead can be a killer. TCP Offload Engines (TOEs) help, and add to the cost.

Wouldn't it be nice if there were an open-source, low-cost, high-performance SAN?

Actually, there is Far away from the big storage centers of Silicon Valley, Boston and Denver, a little company in Athens, Georgia named Coraid has seen the need and responded.

They've developed open protocol called ATA over Ethernet (AoE) that is simple and fast. And cheap.

Coraid sells a chassis that accepts standard SATA disks. Go out and make your best deal and populate the chassis. No paying several hundred percent markups for "qualified" disks. Attach it to standard gigabit ethernet. Mount it using the appropriate AoE driver - Windows, OS X, Linux, FreeBSD and OpenSolaris - and it looks like a local disk.

Since AoE is a low-overhead protocol, it performs much like a local disk as well.

Some things you Don't Get and some things you Don't Need The big Don’t Get is that the protocol isn’t routable - it is strictly local - no IP involved. The Don’t Needs include no TCP/IP overhead, no TCP/IP offload engines, no CPU-cycle sucking and latency-inducing TCP/IP stacks. AoE sits right on the data link layer - level two - of the ISO network model, so with a switched LAN you get very low latency and full network bandwidth across a low-cost, industry standard LAN.

How fast? With a 15 disk RAID 0 - a configuration suitable only for data you can afford to lose - they report 145 MB/sec write speed, using jumbo frames.

The Storage Bits take When RAID was invented, almost 20 years ago, disks were frail, expensive and small. Ethernet was still 10 Mbit/sec, NFS still young, and the name "open source" hadn't been coined. Today, the storage component economics are totally different, as are the trade-offs. Thanks to Moore's Law, we can now build powerful storage systems out of commodity components. Coraid's creative AoE protocol is just one such example.

Comments welcome, of course.

Topics: Storage, Hardware

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.


Log in or register to join the discussion
  • AoE is a very very good solution.


    And I think the "limitation" of being LAN only does not limit anyway the application purpose of this sort of storage solutions.
    For home use (not very likely) there is no problem, in the enterprise scenario almost 99.9% of this type of storage application is made in a Server LAN, so no problem either.
    I have deployed a Coraid solution a couple of years ago and it was indeed very very interesting in all points of view.
    Reliable, fast, very cost effective!
    I even mentioned it in Murphy's Blog as being a great Linux product, and I think AoE is only going to grow at a very fast pace as demand for storage is always groing and 750GB SATA drives are becoming very cheap, very soon we will have 1TB drives!

  • RE: A SAN for the rest of us

    Another AoE disadvantage I see is lack of support from VMWare, Citrix Xen, Virtual Iron, and Microsoft 2008 HV.

    Citrix Xen can be tweaked with kernel modules compiled for AoE but isnt officially "supported".

    Virtual Iron can be tweaked as well to the detriment of lack of "support" from the company.

    VMWare I'm told uses too old a kernel to compile the AoE modules for. Not sure about 3.5, but apparently 3.0 had this limitation.

    Microsoft doesnt seem to have word one typed anywhere about possible support for AoE initiator being built in or added on to Windows 2008 for its Hypervisor also built on Xen.

    So, for the virtualization craze, the "SAN" portion of AoE doesnt quite fit the bill yet. FC or iSCSI are still the preferred choice.

    I also heard a colleague tell me a story of VMWare on NFS being better than any SAN with VMFS for performance and cost control... What he didnt tell me was whether or not VMWare's Vmotion work work with that.