Get to know iSCSI SAN components

With iSCSI making major inroads into the enterprise environment, more CIOs and storage administrators are considering this lower-cost technology to help facilitate a move to a centralized storage environment, or to supplement their existing storage systems. While there is great debate over how and where iSCSI should be used, the future is clear: iSCSI is here to stay. Further, with pricing that is often a fraction of Fibre Channel and generally very good performance, iSCSI's future is bright.

In previous articles, I've discussed some of iSCSI's higher-level functionality (many can be found in Fibre Channel as well), such as snapshots, replication and clustering. In this article, I'll go over the basic building blocks of an iSCSI system, discuss some common iSCSI terminology, and explain iSCSI best practices.

iSCSI building blocks & terminology
The simplest iSCSI system has at least the following four components:

• iSCSI initiator or iSCSI HBA
• iSCSI target
• One or more servers
• Ethernet network

An "iSCSI initiator" is a hardware device or a piece of software residing on a computer that handles communications with an iSCSI array. There are two common ways to connect a server or other computer (even desktops and other workstations can directly connect to an iSCSI array) to an iSCSI array. The first is to use a software-based iSCSI initiator, such as Microsoft's free iSCSI initiator or the iSCSI initiator included with many Linux distributions and other operating systems (including AIX, NetWare, Solaris, and HP-UX). Second, you can skip the software and go right to hardware. Known as an iSCSI HBA (Host Bus Adapter), a hardware-based iSCSI initiator offers some advantages over software-based counterparts. First, a hardware-based device does not consume CPU resources to handle iSCSI commands, as does a software-based initiator. Second, hardware devices can be used to boot the system from an iSCSI SAN, which is not possible with software-based initiators. Third, since you have a device absolutely dedicated to a single task, a hardware-based iSCSI initiator may provide better overall storage performance.

My recommendations: First, start with gigabit Ethernet adapters and only use an iSCSI HBA if you need to boot from SAN, or if your server's CPU is getting killed due to iSCSI overhead. Gigabit Ethernet is very, very cheap these days and in almost all cases, the iSCSI initiator on a gigabit Ethernet link is more than sufficient. Second, use MPIO (Multi Path I/O) on two separate NICs whenever possible. In short: dedicate two gigabit Ethernet NICs in each of your servers to iSCSI connectivity and make sure you read up on your operating system's iSCSI MPIO implementation. At my workplace, we are moving to Dell blade servers and have four gigabit Ethernet NICs in each blade. Two NICs are used for front-end connectivity, with two used for connections to our EqualLogic PS200E array. We are using MPIO wherever possible.

The difference between "array" and "target"
Until this point, I've been calling the storage device an "iSCSI array". In iSCSI-speak, an iSCSI array, or the iSCSI-enabled device to which you will be storing data, is called an "iSCSI target". I make the differentiation between iSCSI array and iSCSI-enabled device for a reason: For most operating systems, there is software available that allows you to convert that system into an iSCSI target and use that system's disks for your storage needs. In these instances, while you still have an iSCSI implementation, you're not really using an array, per se. So, to avoid confusion, use the term "target" instead.

The two terms initiator and target form two endpoints for your iSCSI SAN. In many cases, if you're using enterprise-grade iSCSI targets (like the EqualLogic PS series array I've written about previously), you'll have a single target even if you have multiple arrays, since each individual array is joined to a single cluster that each initiator then addresses as the overall target. If you do have more than one array, I always recommend that you join the two into a single cluster, as you'll enjoy greater performance (depending on the manufacturer) and much easier management since all administrative tasks can then be performed from a single console.

Of course, that leaves the big expanse between the initiator and the target. As you probably know, iSCSI uses TCP/IP for its communication, so a typical Ethernet network is all that's needed to join the two sides of the iSCSI equation into a working system. Always use gigabit Ethernet switches on your iSCSI network. Anything less would severely degrade performance to unacceptable levels. Further, whenever possible, mesh the network between your initiators and your target. That is, use two good quality gigabit Ethernet switches and multihome two connections (one from each switch) to two separate NICs in each server. To your iSCSI targets, multihome each switch to two separate ports on each individual array. Finally, connect each switch to the other. Using this configuration, you can lose a NIC, one of the two switches, and a cable, and remain operational. On the switches, enable Jumbo Frames and flow control to achieve the best and most reliable performance.

iSCSI will continue to evolve over the coming years, with support for more and better drives and support for the next generation of Ethernet running at 10Gbps.