Salesforce.com executives have been giving more details of their plans to prevent repeats of the embarrassing outages earlier this year, according to a report of a user group meeting posted to SalesForceWatch.com, an independent user's blog.
The company itself is neither confirming nor denying the accuracy of the report — most of its key executives are in Hawaii today for its annual sales junket and thus unavailable for measured comment on the topic.
Bonnie Crater, VP and general manager, is reported to have set out a two-stage project plan to resolve the database problems that led to serious outages before and after Christmas:
- Upgrade to Oracle 18.104.22.168
- Replaced defective components in Sun hardware
- Increased Oracle database cluster processing capacity by 50%
- Tune Oracle and the Sun cluster
- Enhanced search architecture (20% - 50% faster)
- Enhanced API cursor table architecture
120-day plan (Approx. June 2006):
- NA1 instance will be split into multiple instances: NA1 - NA4 and NA5 for new customers.
- Each instance will have anchor (read: large) customer with the rest of us peppered throughout.
What's particularly interesting here is the apparent admission that Salesforce.com has been too exposed to suffering outages across its entire customer base because everyone in North America has been on a single instance — which is a feature of the architecture that Zach Nelson, CEO of rival provider NetSuite, had singled out as an important weakness back in January.
There was more, too, about MirrorForce, the Salesforce.com's mirrored data centers, which it seems will not be as all-singing, all-dancing as had originally been indicated:
It may have been "overmarketed" — since true high-availability failover will not occur until Phase 2 of Mirrorforce which is currently scheduled for sometime late 2006 or Q1 2007.
Phase I is complete which mirrors data "at the block level" to an East Coast data center. It would take two days to bring up data center on East Coast as primary.
All I can get out of Salesforce.com officially today is this:
"The field of green that you see on 'trust.salesforce.com' is testament to the great improvements we have seen in system performance. This is a process of continuous improvement, and the steps that we discussed with customers are consistent with that. The customer response has been fantastic so far, and we look forward to delivering even greater levels of service in the future. Just look at the stats— we are delivering more pages faster than ever before."
As soon as I hear anything substantive I'll let you know.