Applying Web 2.0 in the enterprise: News from the trenches

Over the Thanksgiving holiday weekend I finally had time to compile my notes on a significant new Web 2.0 in the enterprise story that emerged earlier this month from the Web 2.0 Summit.  At the same time, one of the inevitable recurring debates about Web 2.0 made its rounds on the blogosphere.  Both pieces of news seem to be hallmarks of important and connected emerging trends on the Internet and are worth a closer look.

Taking the second one first, the term Web 2.0 itself took its latest bashing from Bill Thompson in a new article at The Register that subsequently got some good play in the blogosphere, with probably the most level-headed by our very own Dan Farber.  Bill laments what he thinks Web 2.0 is putting excessive focus on: Ajax and open APIs, to the exclusion of what he thinks is more important, namely to "look beyond the [user] interface to distributed systems, scalable solutions and a network architecture that will support the needs and aspirations of the next five billion users."

Interestingly, while rich user experiences are indeed an important facet of the Web as it has become a true platform in its own right, Web 2.0 has fundamentally been about those exact things that Bill Thompson recommends focusing on.  It is true however, as I wrote up in my Gartner Web 2.0 Hype Cycle analysis, most organizations can probably get nearer-term, less disruptive ROI faster by merely putting dynamic user interfaces on their Web pages. 

But the deeper and more important opportunities have always been about directly exploiting the properties of the Web as platform including user generated content, subsequent potentials to harness collective intelligence, network effects, and creating organic, federated software ecosystems that get better the more people use them (i.e. the blogosphere, SourceForge, Flickr, iTunes, etc.)

Unfortunately, though there have been increasingly rigorous definitions about what Web 2.0 is, many people still focus exclusively on only the more popular new surface elements of Web sites: the evolution of the visual Web user experience, and as Bill Thompson points out, the nascent trend of (often quasi) open Web APIs.  But while the latter story will likely only grow stronger, Ajax and RIAs are almost of peripheral interest when it comes to trends on the Web that appear to be genuine game-changers in business, the enterprise, or even the consumer space.

The fact is, as we look at the unique properties intrinsic to the Web itself -- as well as its scale and pervasiveness -- we can see how the rules of online software aren't being so much rewritten as fundamentally discovered anew.  And some industry watchers are starting to see major opportunities -- even a mandate -- to reinvent products and services to embrace the potentials of this new platform.

And this line of thought this takes us to the first part of our story: How major corporations are starting to seriously explore the use of Web 2.0 techniques in various aspects of their business.   At the IBM session I wrote about recently at the Web 2.0 Summit, Bob Morgan, Vice President of the Chief Technology Office, American Express Technology gave a fascinating presentation on how the financial services giant has become "an enterprise user of Web 2.0 technology" in a number of rather significant ways.

As Bob noted and I'm often witnessing, there appears to be happening in a variety of organizations a distinct trend towards integrating the initiatives around service-oriented architecture (SOA) and Web 2.0 since they overlap extensively and are often handled by the same internal groups.  Indeed, they are almost two sides of the same coin.

Thus, Bob's presentation specifically focused on how these two conceptual approaches seem to be converging for them as well how their product development models are also changing to a more Web 2.0 methodology with things like leveraging customers as testers and co-creators.  But the best story is told by Bob himself and so I took detailed notes at his presentation and here they are with minor edits. 

Bob started off with the three "buckets" that American Express is currently dividing its Web 2.0 and SOA technologies into:

Web 2.0 and SOA Combined: Three Resulting Aspects 

1) Improving the Customer Experience ("this is top for us")

• Interactive, dynamic compelling user experience
• Convenience and reach
• Harnessing user generated content in recent promotions they've been doing
• Experimenting with rich Internet applications
  
• RSS used on customers sites and partner sites
• Looking at RSS technologies, subscribe to the events that are interest to them, and get notified when they want.

2) Community and Collaboration

• Experimenting and piloting wiki technologies
• "Pretty good uptake" with tools by internal teams to collaborate and generate content
• AmexLabs, an online area where select customers come into a site and preview and provide feedback of new interactive capabilities (these customers are individuals, not businesses, cardholders)
• Gives us the opportunity to better understand how products are being developed)
• Pioneering and piloting widget technologies

3) Simplicity and approachability

• How do we make integration a whole lot easier (REST)?
• Leveraging the ease of use of the consumer Web

This is a pretty interesting list in and of itself but Bob went on to keep explore how Web 2.0 techniques, when combined with SOA, can help deliver returns that SOA by itself generally hasn't been able to.

Notes on Using Web 2.0 to Leverage SOA (Source: Bob Morgan, American Express) 

• SOA succeeds in making data more readily available in general
• Integration and security requirements, combined with Web 2.0 creates brand-new challenges (compounds integration and security challenges)
• The power combination: Web 2.0 (mashups and RIAs) can bring the data in a SOA to life
• Allows one to take the volumes of data on hand and makes it accessible
• Because it's embedded in place, mashups can put corporate data into the right context
• But putting so many pieces together creates problem in the environment
• Enterprise-specific Challenges

• Security  - If we can't secure it, we won't use it (hit with as first issue every time)
• Protecting privacy and including identity in mashups

• Mashups can result in overly complex integration
• Unpredictable throughput, capacity, and difficult to monitor
• Lots of sensitive customer data, big challenges, and not a lot yet done to address this
• When we build these things, there's lots to take into account, how to deploy, monitor, manage end-to-end

Admittedly, this is a long, complicated list and seems to go against one of the principal lessons from the Web: Ease of use is the most important design factor.  Yet this confluence of topics and issues is what enterprise IT staff have to grapple with day after day.  And it's important to note that the free-wheeling Web 2.0 world on the Web -- with gigantic viral communities, sites made of mountains of user generated content -- are far less constrained by regulation, governance, privacy, and trust issues than business sites are.  Thus figuring out how to leverage the positive aspects of the emerging best practices on the Web today, without eliminating the very benefit they provide, is one of the biggest challenges in providing a Web 2.0 "context" in the enterprise.

Finally, Bob explored some of the specific challenges of Web 2.0 in the enterprise, noting that these are "not negative statements", but just challenges that have to be seriously sorted through as companies evolve their products and services to these new models:

Some Challenges in Applying Web 2.0 in the Enterprise

1) Development Challenges

• All of this needs close ties to manageability
• The big design tension: Fast and easy vs. well designed and well engineered
• Lack of standard development methods with Web 2.0: Need proven tools and enterprise design patterns
• We now have tools built to bring applications together very quickly, in contrast with traditional development platforms
• And if you build them this way, how can you make them scale to million of customers?
• Lots of existing platforms with different tools, how to preserve an organization's huge investment in current skills and technologies?

2) Business Value

• Is there broad business value in social networking?
• How to do this dealing with a lot of private customer data?
• It's not likely you'll want to bring people together to share customer data
• Not entirely clear yet how to apply social networking profitably to business models

From user generated content and rich user experiences to users as testers, community, and collaboration, American Express is clearly embracing the new Web.  The lessons learned will be of much value and I'm going to try to keep abreast of what Bob and his co-workers are doing.  I've also been collecting other Web 2.0 in the enterprise stories and I will share as much as I can here soon.

Have a Web 2.0 in the enterprise story to share -- big or small?  Please drop me a line.