What to look for in a SaaS vendor

SaaS and on-demand are relatively new categories of application provision and it's often difficult for customers to know how to evaluate whether a prospective supplier is any good. Here's my eight-point checklist:

• Try before buy. If a SaaS vendor can't offer you some kind of pre-purchase trial, it's a bad sign. The whole point of the on-demand model is that the application is already running on the provider's servers. So there should be no big deal with giving prospective customers a test run. Most SaaS vendors embrace this as a fantastic opportunity to win new customers. Any vendor that avoids doing so either has to offer a very good explanation or has something to hide.
• Top-to-bottom configuration. The multi-tenant architecture of the on-demand model means that everyone runs on the same code base, and often the same instance. That forces vendors to offer a huge variety of configuration options so that customers can adapt their view of the application to their own individual business needs. You should expect a plethora of configuration choices that allow you to rename data fields and add your own custom fields; assign policies by user or department or any other context; and reconfigure processes and workflow at will. Preferably via point-and-click checkboxes and menu options that any business manager can understand.
• Service delivery management infrastructure. Several of the remaining checkpoints follow on from this. Neophytes often overlook the necessity for a robust, mature SDM setup. But if a supplier doesn't have sophisticated software in place to manage user provisioning, service level monitoring and reporting, trouble ticket tracking, versioning, server load balancing and a whole host of other infrastructure concerns, then it's simply not serious about software-as-a-service. Nor is it really in a position to guarantee quality of service, security and the like.

• Service level agreement. Some very well-known SaaS vendors don't offer any kind of service guarantees. Amazon Web Services, for example. It doesn't necessarily mean you shouldn't use that provider. After all, the cost of monitoring and administering an SLA puts up the cost of running the service, which means you might have to pay more. But it does mean that if an outage occurs, you'll have no benchmarks to tell you how hard the provider is trying to restore service, and no recourse.
• Status visibility. Very few providers go as far as Salesforce.com, which publishes the status of its servers at trust.salesforce.com. It's another cost of doing business as an on-demand vendor that many choose not to incur. To my mind, they're making a very short-sighted error of judgement. It should be the industry default.
• Business services API. Some kind of API is an absolute must, otherwise you'll never be able to exchange data with this application or link it into larger processes. The best APIs comply with web services standards, are properly documented and — most important — have been carefully designed to make sense from a business point of view: 'GetCustomer' not 'OpenRecord'.
• Paying customers. This is the best indication of whether an on-demand vendor will still be around in a years' time and beyond. It's still an emerging model with few established best practices, so it can be difficult for a vendor to work out the right pricing model and proposition to start getting market traction. If it has referenceable customers on its books, that's a good sign that it's found its feet. And once the money is coming in, it has a steady, predictable income stream that's much more stable than a conventional software vendor's.
• Finance. It can cost two to three times as much to get established as an on-demand vendor compared to a conventional licensed-software vendor. That's because pay-as-you-go revenues don't come in big hits, they build up over time. So an on-demand startup needs to be well funded, either by incrementally building up a paying customer base, or by venture capital of some kind. Rapid growth especially won't be sustained without some form of financing to build out the infrastructure and operations.

How did your vendor score?
6-8: Best in class.
3-5: Promising, but room for improvement.
0-2: Needs to shape up. OK for non-critical apps, but have a back-up plan handy just in case.