A SAN for the rest of us
Fibre Channel to costly? iSCSI too slow? In a past life I was the product manager for the industry's first full Fibre Channel array. We had great hopes for FC as a storage-optimized network that would bring the power of network economics to drive down storage costs. My bad.
Hijacked by vendors, FC has evolved into a costly and fragile mess. It is the fastest block-based storage interconnect, so it isn't going to disappear tomorrow, but the FC market has peaked. iSCSI is coming on strong, but for smaller SANs that need performance the IP overhead can be a killer. TCP Offload Engines (TOEs) help, and add to the cost.
Wouldn't it be nice if there were an open-source, low-cost, high-performance SAN?
Actually, there is Far away from the big storage centers of Silicon Valley, Boston and Denver, a little company in Athens, Georgia named Coraid has seen the need and responded.
They've developed open protocol called ATA over Ethernet (AoE) that is simple and fast. And cheap.
Coraid sells a chassis that accepts standard SATA disks. Go out and make your best deal and populate the chassis. No paying several hundred percent markups for "qualified" disks. Attach it to standard gigabit ethernet. Mount it using the appropriate AoE driver - Windows, OS X, Linux, FreeBSD and OpenSolaris - and it looks like a local disk.
Since AoE is a low-overhead protocol, it performs much like a local disk as well.
Some things you Don't Get and some things you Don't Need The big Don’t Get is that the protocol isn’t routable - it is strictly local - no IP involved. The Don’t Needs include no TCP/IP overhead, no TCP/IP offload engines, no CPU-cycle sucking and latency-inducing TCP/IP stacks. AoE sits right on the data link layer - level two - of the ISO network model, so with a switched LAN you get very low latency and full network bandwidth across a low-cost, industry standard LAN.
How fast? With a 15 disk RAID 0 - a configuration suitable only for data you can afford to lose - they report 145 MB/sec write speed, using jumbo frames.
The Storage Bits take When RAID was invented, almost 20 years ago, disks were frail, expensive and small. Ethernet was still 10 Mbit/sec, NFS still young, and the name "open source" hadn't been coined. Today, the storage component economics are totally different, as are the trade-offs. Thanks to Moore's Law, we can now build powerful storage systems out of commodity components. Coraid's creative AoE protocol is just one such example.
Comments welcome, of course.