Riding the hockey stick: Scaling Web 2.0 software
With all the talk last week about MySpace becoming the #1 most visited site on the Web, there's also been a lot of talk about how Web 2.0 sites like MySpace handle their sharp growth rates. Because Web 2.0 sites explicitly leverage network effects, when they hit their inflection point there can be a simply unmanageable amount of new traffic. Think the Slashdot-effect squared. Discussion about this recently has ranged from Tim O'Reilly's observations about operations becoming the new secret sauce to Jon Udell exploring the burgeoning market for hosting services that specialize in making scaling a problem that you simply outsource.
O'Reilly spoke with Microsoft's Debra Chrapaty, VP of operations for Windows Live about this, noting that:
Her belief is that there's going to be a tipping point in Web 2.0 where the operational environment will be a key differentiator. I mentioned the idea that Web 2.0 has been summed up as "Fail Fast, Scale Fast," and she completely agreed. When it hit its growth inflection point, MySpace was adding a million users every four days -- not at all an easy feat. As these massive apps become the norm, unless you can play in a game where services can be highly stable, geodistributed, etc., you won't be in the game. And that's where she came to the idea that being a developer "on someone's platform" may ultimately mean running your app in their data center.
As I wrote recently, it's the viral feedback loops that Web 2.0 sites use to grow and grow and grow that makes them so successful. That's not to say that the understanding how to do this repeatebly is fully understood yet. But it's an going to be a subject of increasing interest as the disruptive potential of these methods become increasingly proven out and well-known:
Of course, in a few years the exact design patterns for triggering a new MySpace, Facebook, or similar social juggernaut will become common. Then most likely balance will be reachieved in the industry and there will be less disruption. But for now the secret balance of Web 2.0 techniques that powers growth through efficient access to network effects is still an art.
Now, most startups or enterprises attempting to do this in the consumer space or with their customers face the same trouble: The infrastructure challenges can be an enormous and expensive effort that can be an serious distraction and even become the weakest link. After all, any serious success can immediately redline the hosting infrastructure of all but the most competent organizations. This is what O'Reilly means by "Fail Fast, Scale Fast."
To address this, I'm starting to see a significant number of serious new entires aimed specifically at the next generation hosting space. Companies like 3TERA are bringing Web-scale grid computing to the Web 2.0 market that is on-demand, plug-and-play, scale-ready, and supports the Web 2.0 development style. Says 3TERA about their Web 2.0 hosting services:
Web 2.0 operations embrace a dizzying pace of change. Two-week release cycles are common and daily releases aren't unheard of. This pace invalidates traditional release planning. Deployments must be immediate, and roll-backs must be automatic. Amidst the rapid product changes, Web 2.0 firms simultaneously face an unprecedented demand for scale. MySpace, for example, grew from a simple two-tier deployment three years ago to operating 3 data centers, 2,682 web servers, 90 Cache servers, 450 Dart Servers, 60 database servers, 150 media servers, and a 1,000 disk SAN. Growth on this scale, requires being able to add resources to the application without downtime and preferably without operator assistance. Achieving this, means the deployment mechanism must either be an integral part of the infrastructure or be built directly into the application.
Another player in this space is ActiveGrid; they are solving the deployment problems of Enterprise Web 2.0 applications in a slightly different way. ActiveGrid has developed a solution called ActiveGrid Server 2.0 for organizations that are willing to do the hardware hosting themselves:
"There's been a ton of hype around Web 2.0, 'mash-ups,' and AJAX for a couple of years now, but ActiveGrid has become the first vendor to actually release the type of development and deployment tools that make these cool new web apps accessible to developers at Global 2000 companies," said Peter Yared, CEO at ActiveGrid. "We're about to see Web 2.0 make a big jump from its consumer web site origins into primetime enterprise application development."
Finally, there is the OpenFount project, which uses an application model based on Google's Web Toolkit and Amazon's S3 storage server. OpenFount makes it possible to externally host and manage Ajax applications. (See link from Jon Udell above for more details.)
In the end however, many of these products currently constrain developers to use their coding models, technologies, and application structure. While this standardization isn't always bad, it does mean that there will be the inevitable push towards a standard application deployment model so that hosting Web 2.0 applications are portable between hosting services. Otherwise, when your hoster goes out of business, so do you.
The bottom line however, is that hosting and operations will become increasingly strategic as software moves online. Application architecture that thinks about about deployment as separate from the software stack will become outmoded thinking. Thus, the deployment model will increasingly become a central element of application design.
Interestingly, however, this doesn't answer the question of how to scale up applications that are built from the fabric of the Global SOA that the Web has become. This is where I've discussed how service-level agreements and other means of negotiating clear expectations from external suppliers of services will be an essential part of capacity planning in the era of Web sites fueled by network effects. Ultimately, scalability, like success, is never really easy.
Have you been plagued by online software outages? Tell us your story.