Storage APIs show signs of permanence

The great promise of storage area networks is consolidation -- everything working together as one for better performance, management and reliability. It's a compelling argument that looks good in PowerPoint. Yet the real world demands that the details are right before the big picture works, and in storage the most important detail is the application program interface or API.

An API is the language a storage component speaks. It doesn't matter how many features or what sort of performance a component has, if it's not available through the API it might as well not be there. Exactly the same argument works for the software that manages the components: if it can't control a feature through the API it can't manage it. Vendors know that cost-effectiveness is a prime selling point: a single management system that can be handled by a minimum number of staff without the need for complex, cross-platform training is an attractive option. Therefore, management systems are trying to be as capable as possible, able to handle fibre channel adaptors and switches, disks, databases, tapes and so on. The glue that makes it all possible is a rich set of APIs.

This makes the API of prime importance, and highlights a tension between the commercial wishes of storage companies and those of their customers. In an ideal world for customers, all APIs would be public and would work across all hardware, software, networking and management systems. A vendor, however, knows that if it can lock in features to its private or controlled APIs then customers will have to continue to purchase its products if they want to keep up with innovation.

In practice, this tension leads to an uneasy mix of public, semi-public and private APIs. Companies form alliances and share APIs between themselves, industry bodies try to get as many names on board for high profile standards while the big guns of the industry try to present 'all in one' packages where interoperability with competitive products is downplayed. With storage networks though, systems that don't work in heterogeneous environments are automatically at a disadvantage, and to date companies have tried to square the circle by selling compatibility options.

One notable case of this is -- or was -- EMC's WideSky storage management platform. Launched in 2001, it soon included an API swap with Compaq, so that management software from either company could handle each other's storage components. Over time, EMC's WideSky net got cast wider to include BMC, Computer Associates, Microsoft, and Oracle, among others. Intended to be a universal translator matrix that converted each organisation's APIs, WideSky would even include APIs that weren't from members of EMC's club of friends -- EMC said that it would reverse engineer those that weren't public and include them anyway.

But at the same time as EMC was pursuing this strategy, a Hitachi-originated project called Bluefin was also maturing under the auspices of the Storage Networking Industry Association. The standard is now officially known as the Storage Management Interface Specification, SMIS, as well as sometimes just SMI or SMI-S. SMIS takes some existing standards such as CIM -- the Common Information Model, which describes the management requirements and capabilities of systems -- and the Web-Based Enterprise Management, WBEM, and specifies how they should be used. SMIS has attracted the support of just about every company working in storage management, and EMC has recently announced that it is moving from WideSky to SMIS, as the standard matures. Currently, SMIS 1.0.1 is being voted on for approval as a full standard, which is expected to happen at the beginning of November this year. SMIS includes common interoperable and extensible management transport; automated discovery; and resource locking. This all means that when a new SMIS component is introduced to a SAN, it will announce who it is and what it can do, and will then be able to properly share its resources with multiple entities.

Just because a company claims support for an API doesn't mean that everything will work. There are plenty of areas where the published API may be ambiguous or incomplete, or where one company's interpretation differs from another's. Test and verification is essential, and this always takes longer than seems likely in the first flush of a new standard. Part of the SNIA's job is to arrange 'plugfests' -- get-togethers where vendors mix and match their products and check that they do play well together after all. Invariably, they won't at first -- it's not until a large number of products are commercially available that properly correspond to the standard that companies can get the experience necessary to make sure their internal testing is accurate and a consensus about design and implementation is reached.

This delay in rolling out a new standard is one of the reasons that non-conformist companies quote when offering their alternatives. Customers want solutions now, they say, and it's no great drawback to have a working system today that can be upgraded to the new standard when it's ripe. This can even be true, but it is important that the migration process from proprietary to standard APIs is known and properly planned, together with deadlines for the appearance of a conformant product.

Although the SAN API world has been complex and difficult to navigate, there is now little or no excuse for companies to ignore the burgeoning standards movement. Just as no applications software company would dare insist on having a non-standard operating system for its wares -- leaving aside those with monopolistic powers -- we are moving to a situation where any SAN provider without standard APIs will fall at the first. But people who plan and implement SANs will still have to watch the small print and check first. We're not at the stage where it's safe to assume that nominally compliant devices and software will actually work together out of the box: make sure the vendors take that risk.