Mashups: The next major new software development model?

At last week's Mashup Ecosystem Summit held in San Francisco and sponsored by IBM with an invited assemblage of leading players in this space, I gave an opening talk about the current challenges and opportunities of mashups.  And there I posed the title of this post as a statement instead of a question.   The reason that it's a question here is entirely driven by the context of who is currently creating the majority of mashups these days.  Because even a cursory examination of what people are doing every day on the Web right now tells us that mashups -- also known as ad hoc Web sites created on the fly out of other Web sites -- are indeed happening in a large way, albeit in simple forms, by the tens of thousands online every day. 

The consumerization of the enterprise as younger workers bring their Web 2.0 skills and habits to work has already begun.But inside our organizations, both in the IT department and in business units, mashups are a much rarer phenomenon.  And in fact, this is one of the classic hallmarks of the Web 2.0 era; the much larger community of the Web as a major source of innovation and leading edge behavior that subsequently moves across the firewall and into our workplaces.

However, the topic of this blog is aimed at the application of Web 2.0 to the enterprise and so whether mashups will be a significant new model for application development inside our businesses anytime soon is still somewhat of an open question.  It's worth noting that McKinsey's recent global executive survey of Web 2.0 in business said that a whopping 21% of large businesses across the board are planning investment in mashups in 2007, but a sobering 54% of business executives also said mashups were not even under consideration.  Understanding the timing on mashup adoption therefore is important along with the challenge of communicating their potential.

Since the mashup story is primarily being driven by spontaneous activity at the edge of the Internet, an accurate and updated picture of what's actually happening with them is harder to make out than if it was being driven by a centralized industry effort.  And as it turns out, this makes what's happening richer and more exciting than it would be otherwise while at the same providing significant challenges for those that want to take these compelling ideas and apply them deliberately to solve business problems.

So in the interest of making sure we have the broadest industry discussion we can about mashups -- and to make sure there is some kind of snapshot of what we think we're seeing in this space -- I thought I'd summarize the notes from my talk at the Mashup Ecosystem Summit.

To bring folks that are just joining the mashup conversation up to speed on why mashups are so exciting, I'll start with my take on the key aspects of mashups from a value proposition perspective.

Key Aspects and Benefits of the Mashup Approach

• Effective leverage of Web parts and the Global SOA. Mashups are generally built out of the bits, pieces, and services of other Web applications that already exist, adding code only when it can't be sourced from internal or external suppliers or to provide integration "glue" between the parts.  This reuse can quickly and easily leverage millions of dollars in previous investment and results in a "building on the shoulder's of giants" effect.  Like the visual component marketplace successfully built around ActiveX in the late 90's, mashups are a form of reuse that actually works but on a much larger scale than ever before.  See ProgrammableWeb's open API cloud and WidgetBox for a good example of the vast array Web parts that are now widely available.
• Simple, lightweight software models and services. By focusing on the simplest possible techniques and formats, Web mashups appear to be successful and widespread primarily because just about anyone can and are creating them.  Mashups are typically built using techniques like cutting and pasting snippets of Javascript, using feeds and XML to connect the various parts together, and even one-line Javascript inclusions that can pull in and integrate an powerful external component, such as Google Maps or a YouTube video player, that originally required a massive investment from its creator.  There sometimes seems to be no limit to the effort to lower the barrier to consumption of these Web parts.  Both Google and YouTube are poster children for easy Web part consumption and have reaped corresponding rewards.  Google makes its AdWords and Maps widgets incredibly easy to install and deploy while YouTube even puts the hosting code next to each and every video on its site.  Finally, mashups are 100% pure Software as a Service (SaaS) and require no installation, updates, plug-ins, admin rights, or anything but a garden variety Web browser and the mashup's URL to run.
• A focus on self-service and DIY.  As I alluded to above, mashup development can be just as much for everyday Web users as it is for professional software developers.  Like so many things on the Web that put the power of publishing and participation into everyone's hands, mashups have the potential to give all of us the ability to create real, useful software.  And while end-user mashups will remain in the bottom half or one quarter of the software complexity spectrum, it means that applications that could never have been justified on a build-vs.-buy perspective (and took too long to acquire to help) are now possible  These apps can now just be created by users -- and groups of collaborating users -- on the fly as they need them.  This has the potential to further enable the productivity of knowledge workers as well as release The Long Tail of IT demand.  It could also reduce the application backlogs that continue to bedevil IT departments and their customers everywhere.  And to re-emphasize, it's because mashups use such simple techniques that jsut about anyone can now create the views, dashboards, and even real software apps that let them get their work done better and faster without unnecessary bureaucracy (some bureaucracy is required as I indicate below).  I've been collecting real-world examples of this in the workplace and will share them in an upcoming post.

There are numerous smaller, ancillary benefits of mashups including the fact they are Web-oriented and 1) can leverage link structure, 2) tend to be more open and visible which results in more transparency and information sharing, and 3) their content can even be discoverable by search if some care is taken.  In this way, they become part of the Enterprise 2.0 story as well, particularly if they are socially enabled.  Fortunately, a good number of widgets, such as those from txtDrop and MyBlogLog, make it easy to add simple social aspects to mashups to enable their full power.

Not convinced anyone can build mashups?  View a screencast of an entire application being built out of Web parts in 15 minutes using IBM's QEDWiki platform (link near bottom)

This also doesn't mean there aren't challenges and a few drawbacks to mashups which I'll highlight below.  For now, it's enough to keep in mind that the three major benefits including high-levels of reuse and leverage, models that support rapid, easy development and integration, and a focus on DIY by anyone from expert developers to newbie Web users.

The Challenges and Opportunities of Mashups

Mashup Challenges

• Deconflicting the two major mashup models.  Right now mashups are happening mostly "in the wild" with users taking their blogs, wiki pages, MySpace profiles, and even ordinary Web pages and covering them with badges, widgets, and gadgets from elsewhere on the Web.  On the high end of the complexity spectrum, there are about 2,000 developer created mashups presently available (Source: ProgrammableWeb.) There is also now a growing body of tools from commercial software vendors that also try to bring the apparent advantages of mashups to the business world, both on the Web and in our intranets.  In general, these two mashup models are quite different from each other.  On the consumer Web we have the natural, emergent mashup phenomenon as numerous Web parts suppliers and their consumers try different strategies out and sometimes hit upon the models that work best (which appears to be things like one-line includes, cut-and-paste, and smart widgets that make integration easy, with Google Maps being another great example.)  On the vendor side, enterprise mashup tools try to add missing enterprise context like security, support for local SOAs, and so on, as well as a mashup development model, usually by choosing a particular Web component standard and providing a visual IDE that makes it easier for the non-HTML savvy to create mashups.  These commercial mashup application models try to a priori figure out what will work best and in this they may be making the mistake of ignoring the vast laboratory of the Web that is already proving out highly effective models for mashup parts and integration strategies on a large scale.  There is also a skill and open/closed barrier between these two approaches that tends to prevent the movement or migration from model to the other.  My best guess is that the commercial mashup tools that will succeed the best will eliminate this barrier or at least greatly reduce it.
• Too many widget formats.  Both Microsoft and Google have their own gadget models, NetVibes has the compelling Universal Widget Architecture (UWA), and OpenAjax has no component model per se but vital strategies for making Web parts work together in the same mashup.  Even the venerable and respected W3C has gotten in the act with a first draft of the Widgets 1.0 specification. Visual tool support is important to fully enable mashup development and realize the productivity potential and this support requires a consistent widget format.  But the proliferation of widget models (both ad hoc and formal specs) makes visual tooling expensive and time-consuming to implement.  And though the greater Web has been relatively successful so far without them, enterprises are not yet anxious to start figuring out which widget models are the most important.  Unfortunately, no obvious solution is on the horizon though some early techniques have formed including wrapping the different component models into a standard widget wrapper.
• Not enough Web services exist in our enterprises or on the Web.  While the number of open APIs on the Internet continues to grow quickly (431 by last count according to Programmable Web), there just aren't enough Web services available to supply the data and back-end functionality to mashups.  Most data on the Internet and our intranets are still in static Web pages in the form of HTML.  While the growing adoption of XHTML helps a little, the greater service-enablement of the Web will take years and years.  In the meantime we need to quickly service-enable our silos of Web-based information if we are to fully exploit it.  Fortunately, this challenge can now be partially mitigated with companies like Kapow and Yahoo with Pipes, which are providing just such tools to make this possible.  I recently did a screencast using Kapows' RoboMaker showing how Web 1.0 content can be transformed into a Web 2.0 friendly service format like RSS in a few minutes using visual tools.  (Disclaimer: I have a client relationship with Kapow.)  Also note that this challenge is one reason why widgets have grown so popular, since they offer both a data connection back to the server they came from as well as a visual aspect that allows the information to be seen and integrate with.  Thus widgets are ideal for consumption by providing a sort of easy-to-use "visual SOA."
• Security and identity need to be sorted out.  The most useful mashups will involve Web-based creations that are powered with our personal and business information.  For now, most mashups don't require (or support) logins that allow it to collect information from your private repositories of information.  While initiatives like OpenID have the potential to resolve some of these issues, there is a lot of work to be done before the average user will trust a mashup with access to their private information.  This shortcoming fundamentally limits the real value that mashups have the potential to provide.  While mashup security is actually a long topic byself, I'll save a deeper discussion for another post.
• No common creation metaphor.  Other major Web 2.0 platforms such as blog and wikis have very simple, well-known usage models.  There is a save button, an edit button, and either a reverse chronology of posts (blogs) or a series of page revisions (wikis).  However, other than the aforementioned simple cut-and-paste model, nothing has really emerged in a similar vein for the creation of mashups.  This will resolve itself slowly as some widgets now have configuration popups or easy code generators to get what you need done without needing to be a Javascript expert.  However this is a far cry from having a development model that is generally well understood by most people and well documented.  I often say that spreadsheets and Microsoft Access are the end-user development tools most of us have today and they offer some insight, as well as blogs and wikis, into what a workable model might look like.  Offering some hope, mashup vendors are exploiting the near ubiquity of the blog and wiki model and IBM has merged mashup development with wikis with QEDWiki and Dan Bricklin has done something similar with wikis and spreadsheets with WikiCalc.

Mashup Opportunities

• Defining the essential ingredients of a successful mashup ecosystem.  This is about conveying a clear conception to the marketplace that we really are moving more from green field development to a world where we assemble our software out of the rich content and functionality we now have ready access to on the Web and our SOAs.  Other ecosystem questions: What kind of Web services do you need?  How about adapters to legacy systems and content?  Should you encourage mashups to be the foundation for other mashups? How do we guarantee compatibility and interoperability?  These and a lot more issues about how to create a successful mashup ecosystem need to be better articulated than they have been to date.
• Addressing the tension between the two major styles of integration.  Most integration today is done up front with lots of testing and configuration control and baselined code from your external suppliers.  On the other hand, mashups rely on live pulls of code from your supplier and are a much more extreme form of combining our systems together.  The very word "mashup" conveys how ad hoc this really is.  This new live model of integration has security, testing, and version control issues written all over it.  Google and some of the larger suppliers are getting a handle on some of these but market leadership on this could go a long way.
• Providing effective "enterprise context."  Mashups are a creation of the consumer Web and are not always ready to "play" in the enterprise space.  To even get a foot in the door, enterprise mashup tools need to have solid stories around single sign-on (SSO), LDAP, JSR168 (portals/portlets), legacy integration, management, monitoring, RSS strategy, etc.  Most enterprise tools are still falling short in these categories and will likely not get broad adoption until they address them.
• Distribution and consumption.  Many of the ideas that the consumer Web has come up with for Web parts to be highly viral, easily distributable, and eminently consumable are also important strategies that we must think seriously about moving into our SOA initiatives.  That's because our internal SOAs are smaller versions of the very same ecosystem that we have seen form on the Web.  Getting services adopted and used in the enterprise has been entirely too hard up until now and even many companies out on the Web are falling short of in terms of applying the latest low-barrier, viral distribution techniques for success and uptake.
• SEO, analytics, page views are all challenged by the mashup model.  Just like Ajax and Flash, mashups turn single Web pages into entire applications and all three of these Web application models have been slow to address some of the more important models and monetization strategies that power business on the Web.  There is ample room for companies that let mashup creators address the loss of these important aspects of Web usage. 
  

As I read through this list, it's a clear that I've tried to address both consumer and enterprise mashups, two very different beasts with a somewhat different audience.  I say somewhat different since the consumerization of the enterprise as younger workers bring their Web 2.0 skills and habits to work has already begun.  For now however, it's seems clear that whatever the adoption speed, mashups are here to stay as one of the most compelling and efficient ways to turn time and money into working software and solve business problems in new and innovative ways.

Mashups are so easy to create that Janko Roettgers at GigaOM says they should be used as a preliminary litmus test for a startup's business model. 

Read other excellent coverage of the mashup summit from Ross Mayfield, Jeff Nolan, SnapLogic, and Deepak Alur.

No list of mashups challenges and opportunities could be complete.  What did I miss?  Add them below in TalkBack and I'll update this post with the best submissions in a week or so.