Mashups: The next major new software development model?

Mashups: The next major new software development model?

Summary: At last week's Mashup Ecosystem Summit held in San Francisco and sponsored by IBM with an invited assemblage of leading players in this space, I gave an opening talk about the current challenges and opportunities of mashups. And there I posed the title of this post as a statement instead of a question. The reason that it's a question here is entirely driven by the context of who is currently creating the majority of mashups these days. Because even a cursory examination of what people are doing every day on the Web right now tells us that mashups -- also known as ad hoc Web sites created on the fly out of other Web sites -- are indeed happening in a large way, albeit in simple forms, by the tens of thousands online every day.

SHARE:

At last week's Mashup Ecosystem Summit held in San Francisco and sponsored by IBM with an invited assemblage of leading players in this space, I gave an opening talk about the current challenges and opportunities of mashups.  And there I posed the title of this post as a statement instead of a question.   The reason that it's a question here is entirely driven by the context of who is currently creating the majority of mashups these days.  Because even a cursory examination of what people are doing every day on the Web right now tells us that mashups -- also known as ad hoc Web sites created on the fly out of other Web sites -- are indeed happening in a large way, albeit in simple forms, by the tens of thousands online every day. 

The consumerization of the enterprise as younger workers bring their Web 2.0 skills and habits to work has already begun.But inside our organizations, both in the IT department and in business units, mashups are a much rarer phenomenon.  And in fact, this is one of the classic hallmarks of the Web 2.0 era; the much larger community of the Web as a major source of innovation and leading edge behavior that subsequently moves across the firewall and into our workplaces.

However, the topic of this blog is aimed at the application of Web 2.0 to the enterprise and so whether mashups will be a significant new model for application development inside our businesses anytime soon is still somewhat of an open question.  It's worth noting that McKinsey's recent global executive survey of Web 2.0 in business said that a whopping 21% of large businesses across the board are planning investment in mashups in 2007, but a sobering 54% of business executives also said mashups were not even under consideration.  Understanding the timing on mashup adoption therefore is important along with the challenge of communicating their potential.

Since the mashup story is primarily being driven by spontaneous activity at the edge of the Internet, an accurate and updated picture of what's actually happening with them is harder to make out than if it was being driven by a centralized industry effort.  And as it turns out, this makes what's happening richer and more exciting than it would be otherwise while at the same providing significant challenges for those that want to take these compelling ideas and apply them deliberately to solve business problems.

So in the interest of making sure we have the broadest industry discussion we can about mashups -- and to make sure there is some kind of snapshot of what we think we're seeing in this space -- I thought I'd summarize the notes from my talk at the Mashup Ecosystem Summit.

The Opposite Ends of the Mashup Spectrum

To bring folks that are just joining the mashup conversation up to speed on why mashups are so exciting, I'll start with my take on the key aspects of mashups from a value proposition perspective.

Key Aspects and Benefits of the Mashup Approach

  • Effective leverage of Web parts and the Global SOA. Mashups are generally built out of the bits, pieces, and services of other Web applications that already exist, adding code only when it can't be sourced from internal or external suppliers or to provide integration "glue" between the parts.  This reuse can quickly and easily leverage millions of dollars in previous investment and results in a "building on the shoulder's of giants" effect.  Like the visual component marketplace successfully built around ActiveX in the late 90's, mashups are a form of reuse that actually works but on a much larger scale than ever before.  See ProgrammableWeb's open API cloud and WidgetBox for a good example of the vast array Web parts that are now widely available.
  • Simple, lightweight software models and services. By focusing on the simplest possible techniques and formats, Web mashups appear to be successful and widespread primarily because just about anyone can and are creating them.  Mashups are typically built using techniques like cutting and pasting snippets of Javascript, using feeds and XML to connect the various parts together, and even one-line Javascript inclusions that can pull in and integrate an powerful external component, such as Google Maps or a YouTube video player, that originally required a massive investment from its creator.  There sometimes seems to be no limit to the effort to lower the barrier to consumption of these Web parts.  Both Google and YouTube are poster children for easy Web part consumption and have reaped corresponding rewards.  Google makes its AdWords and Maps widgets incredibly easy to install and deploy while YouTube even puts the hosting code next to each and every video on its site.  Finally, mashups are 100% pure Software as a Service (SaaS) and require no installation, updates, plug-ins, admin rights, or anything but a garden variety Web browser and the mashup's URL to run.
  • A focus on self-service and DIY.  As I alluded to above, mashup development can be just as much for everyday Web users as it is for professional software developers.  Like so many things on the Web that put the power of publishing and participation into everyone's hands, mashups have the potential to give all of us the ability to create real, useful software.  And while end-user mashups will remain in the bottom half or one quarter of the software complexity spectrum, it means that applications that could never have been justified on a build-vs.-buy perspective (and took too long to acquire to help) are now possible  These apps can now just be created by users -- and groups of collaborating users -- on the fly as they need them.  This has the potential to further enable the productivity of knowledge workers as well as release The Long Tail of IT demand.  It could also reduce the application backlogs that continue to bedevil IT departments and their customers everywhere.  And to re-emphasize, it's because mashups use such simple techniques that jsut about anyone can now create the views, dashboards, and even real software apps that let them get their work done better and faster without unnecessary bureaucracy (some bureaucracy is required as I indicate below).  I've been collecting real-world examples of this in the workplace and will share them in an upcoming post.

There are numerous smaller, ancillary benefits of mashups including the fact they are Web-oriented and 1) can leverage link structure, 2) tend to be more open and visible which results in more transparency and information sharing, and 3) their content can even be discoverable by search if some care is taken.  In this way, they become part of the Enterprise 2.0 story as well, particularly if they are socially enabled.  Fortunately, a good number of widgets, such as those from txtDrop and MyBlogLog, make it easy to add simple social aspects to mashups to enable their full power.

Not convinced anyone can build mashups?  View a screencast of an entire application being built out of Web parts in 15 minutes using IBM's QEDWiki platform (link near bottom)

This also doesn't mean there aren't challenges and a few drawbacks to mashups which I'll highlight below.  For now, it's enough to keep in mind that the three major benefits including high-levels of reuse and leverage, models that support rapid, easy development and integration, and a focus on DIY by anyone from expert developers to newbie Web users.

The Challenges and Opportunities of Mashups

Mashup Challenges

  • Deconflicting the two major mashup models.  Right now mashups are happening mostly "in the wild" with users taking their blogs, wiki pages, MySpace profiles, and even ordinary Web pages and covering them with badges, widgets, and gadgets from elsewhere on the Web.  On the high end of the complexity spectrum, there are about 2,000 developer created mashups presently available (Source: ProgrammableWeb.) There is also now a growing body of tools from commercial software vendors that also try to bring the apparent advantages of mashups to the business world, both on the Web and in our intranets.  In general, these two mashup models are quite different from each other.  On the consumer Web we have the natural, emergent mashup phenomenon as numerous Web parts suppliers and their consumers try different strategies out and sometimes hit upon the models that work best (which appears to be things like one-line includes, cut-and-paste, and smart widgets that make integration easy, with Google Maps being another great example.)  On the vendor side, enterprise mashup tools try to add missing enterprise context like security, support for local SOAs, and so on, as well as a mashup development model, usually by choosing a particular Web component standard and providing a visual IDE that makes it easier for the non-HTML savvy to create mashups.  These commercial mashup application models try to a priori figure out what will work best and in this they may be making the mistake of ignoring the vast laboratory of the Web that is already proving out highly effective models for mashup parts and integration strategies on a large scale.  There is also a skill and open/closed barrier between these two approaches that tends to prevent the movement or migration from model to the other.  My best guess is that the commercial mashup tools that will succeed the best will eliminate this barrier or at least greatly reduce it.
  • Too many widget formats.  Both Microsoft and Google have their own gadget models, NetVibes has the compelling Universal Widget Architecture (UWA), and OpenAjax has no component model per se but vital strategies for making Web parts work together in the same mashup.  Even the venerable and respected W3C has gotten in the act with a first draft of the Widgets 1.0 specification. Visual tool support is important to fully enable mashup development and realize the productivity potential and this support requires a consistent widget format.  But the proliferation of widget models (both ad hoc and formal specs) makes visual tooling expensive and time-consuming to implement.  And though the greater Web has been relatively successful so far without them, enterprises are not yet anxious to start figuring out which widget models are the most important.  Unfortunately, no obvious solution is on the horizon though some early techniques have formed including wrapping the different component models into a standard widget wrapper.
  • Not enough Web services exist in our enterprises or on the Web.  While the number of open APIs on the Internet continues to grow quickly (431 by last count according to Programmable Web), there just aren't enough Web services available to supply the data and back-end functionality to mashups.  Most data on the Internet and our intranets are still in static Web pages in the form of HTML.  While the growing adoption of XHTML helps a little, the greater service-enablement of the Web will take years and years.  In the meantime we need to quickly service-enable our silos of Web-based information if we are to fully exploit it.  Fortunately, this challenge can now be partially mitigated with companies like Kapow and Yahoo with Pipes, which are providing just such tools to make this possible.  I recently did a screencast using Kapows' RoboMaker showing how Web 1.0 content can be transformed into a Web 2.0 friendly service format like RSS in a few minutes using visual tools.  (Disclaimer: I have a client relationship with Kapow.)  Also note that this challenge is one reason why widgets have grown so popular, since they offer both a data connection back to the server they came from as well as a visual aspect that allows the information to be seen and integrate with.  Thus widgets are ideal for consumption by providing a sort of easy-to-use "visual SOA."
  • Security and identity need to be sorted out.  The most useful mashups will involve Web-based creations that are powered with our personal and business information.  For now, most mashups don't require (or support) logins that allow it to collect information from your private repositories of information.  While initiatives like OpenID have the potential to resolve some of these issues, there is a lot of work to be done before the average user will trust a mashup with access to their private information.  This shortcoming fundamentally limits the real value that mashups have the potential to provide.  While mashup security is actually a long topic byself, I'll save a deeper discussion for another post.
  • No common creation metaphor.  Other major Web 2.0 platforms such as blog and wikis have very simple, well-known usage models.  There is a save button, an edit button, and either a reverse chronology of posts (blogs) or a series of page revisions (wikis).  However, other than the aforementioned simple cut-and-paste model, nothing has really emerged in a similar vein for the creation of mashups.  This will resolve itself slowly as some widgets now have configuration popups or easy code generators to get what you need done without needing to be a Javascript expert.  However this is a far cry from having a development model that is generally well understood by most people and well documented.  I often say that spreadsheets and Microsoft Access are the end-user development tools most of us have today and they offer some insight, as well as blogs and wikis, into what a workable model might look like.  Offering some hope, mashup vendors are exploiting the near ubiquity of the blog and wiki model and IBM has merged mashup development with wikis with QEDWiki and Dan Bricklin has done something similar with wikis and spreadsheets with WikiCalc.

Mashup Opportunities

  • Defining the essential ingredients of a successful mashup ecosystem.  This is about conveying a clear conception to the marketplace that we really are moving more from green field development to a world where we assemble our software out of the rich content and functionality we now have ready access to on the Web and our SOAs.  Other ecosystem questions: What kind of Web services do you need?  How about adapters to legacy systems and content?  Should you encourage mashups to be the foundation for other mashups? How do we guarantee compatibility and interoperability?  These and a lot more issues about how to create a successful mashup ecosystem need to be better articulated than they have been to date.
  • Addressing the tension between the two major styles of integration.  Most integration today is done up front with lots of testing and configuration control and baselined code from your external suppliers.  On the other hand, mashups rely on live pulls of code from your supplier and are a much more extreme form of combining our systems together.  The very word "mashup" conveys how ad hoc this really is.  This new live model of integration has security, testing, and version control issues written all over it.  Google and some of the larger suppliers are getting a handle on some of these but market leadership on this could go a long way.
  • Providing effective "enterprise context."  Mashups are a creation of the consumer Web and are not always ready to "play" in the enterprise space.  To even get a foot in the door, enterprise mashup tools need to have solid stories around single sign-on (SSO), LDAP, JSR168 (portals/portlets), legacy integration, management, monitoring, RSS strategy, etc.  Most enterprise tools are still falling short in these categories and will likely not get broad adoption until they address them.
  • Distribution and consumption.  Many of the ideas that the consumer Web has come up with for Web parts to be highly viral, easily distributable, and eminently consumable are also important strategies that we must think seriously about moving into our SOA initiatives.  That's because our internal SOAs are smaller versions of the very same ecosystem that we have seen form on the Web.  Getting services adopted and used in the enterprise has been entirely too hard up until now and even many companies out on the Web are falling short of in terms of applying the latest low-barrier, viral distribution techniques for success and uptake.
  • SEO, analytics, page views are all challenged by the mashup model.  Just like Ajax and Flash, mashups turn single Web pages into entire applications and all three of these Web application models have been slow to address some of the more important models and monetization strategies that power business on the Web.  There is ample room for companies that let mashup creators address the loss of these important aspects of Web usage. 

As I read through this list, it's a clear that I've tried to address both consumer and enterprise mashups, two very different beasts with a somewhat different audience.  I say somewhat different since the consumerization of the enterprise as younger workers bring their Web 2.0 skills and habits to work has already begun.  For now however, it's seems clear that whatever the adoption speed, mashups are here to stay as one of the most compelling and efficient ways to turn time and money into working software and solve business problems in new and innovative ways.

Mashups are so easy to create that Janko Roettgers at GigaOM says they should be used as a preliminary litmus test for a startup's business model. 

Read other excellent coverage of the mashup summit from Ross Mayfield, Jeff Nolan, SnapLogic, and Deepak Alur.

No list of mashups challenges and opportunities could be complete.  What did I miss?  Add them below in TalkBack and I'll update this post with the best submissions in a week or so.

Topics: Software, Browser, Software Development

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

26 comments
Log in or register to join the discussion
  • Open Standards for Online Office Mashups

    A set of open standards have been developed to support interoperability across Office 2.0 applications for document sharing, single sign-on, and linking/embedding of live office content in a web context. This set of standards is called "Open Simple Ajax Mashup" - OpenSAM for short - with a community web site at www.opensam.org. The OpenSAM standard builds on existing open Internet standards such as webDAV, CGI, AJAX, ALE (Ajax Linking/Embedding), and SSO (via HTTP authentication, OpenID, etc.).

    This standards effort is intended to (1) make it easy for end-users to share documents across Office 2.0 applications from different vendors (2) easily access multiple Office 2.0 applications with a single login; and (3) embed a live spreadsheet in a document for example where each app comes from a different vendor.

    The first commercial product built on the OpenSAM standard, ShareOffice, is comprised of five different web services (EditGrid, iNetWord, Preezo, ShareMethods, and salesforce.com) and this mash-up has just launched as the first online office in the salesforce.com AppExchange. It is hoped that the OpenSAM standards will lead to greater interoperability and choice for end-users when they seek to mash-up applications that they need for best of breed online office services.
    Eric Hoffert
    • RE: Mashups: The next major new software development model?

      @Eric Hoffert This will be a classic and highly recommended article for consultants to point clients toward! Congratulations. <a href="http://www.arabaoyunlarimiz.gen.tr/zeka-oyunlari/">zeka oyunlari</a>
      AlexSerdar
    • RE: Mashups: The next major new software development model?

      @Eric Hoffert Think of a real estate company using Google Maps to show where the houses in their listings are.
      AlexSerdar
  • What's a mashup?

    and why is it important?
    Badge3832
    • The author gave a pretty good definition of mashup:

      "mashups ? also known as ad hoc Web sites created on the fly out of other Web sites"

      Think of a real estate company using Google Maps to show where the houses in their listings are.

      Think of about any company using YouTube to store promotional videos, and embedding a YouTube player on their website so customers can view the video right there.

      These are examples that would have required a ton of work for something that would not have been near as good. Imagine a real estate company creating or buying its own mapping software and and gathering the satellite data and street information. Here, for FREE, they use Google Maps. Sure, Google Maps gets a little publicity out of the deal, but, who cares?

      And, it looks like Google and others are more worried about market share right now, and will figure out how to monetize the components they provide later. Well, they already get publicity out of it.
      DonnieBoy
  • Why mashups won't matter until..

    they can quantitatively demonstrate a higher productivity level throughout their lifespan. This includes support costs, training, etc. Imagine if you could forget document sharing and develope a mashup that can extract that data out of the systems in a way that a person would mine the data.

    Removing each of the obstacles to decisions is where mashups need to go, otherwise they will just be another cool buzzword that retains its place on the fringe of business.
    THEE WOLF
    • Mashups do need to provide clear value...

      And while we see interesting, even very compelling, results from the consumer space, we'll have to see what happens as hacker activity matures into business activity that provide more value.

      Excellent points however and I agree that these obstacles need to come down for mashups to stay on the radar...

      Dion
      dionhinchcliffe
      • Clear value??? Huh??? You mean like having maps instanly working without

        paying a dime???
        DonnieBoy
    • Mashups already matter. There are a ton of companies using Google Maps, and

      YouTube video service AS WE SPEAK. This is just the tip of the iceberg.
      DonnieBoy
  • excellent summary

    Looked a bit daunting at first, and I'm wary of mashups that consist of a speech that the writer did for for someone else, and the need to get something in their blog, so they throw the speech into the blog, but hey, this really is a valuable piece of work.
    degrootp
  • Mashups must deliver business value

    Ultimately for mashups and SaaS ecosystems to succeed, the participating vendors and customers have to find speed to value. It's important that development of mashups and integrated solutions can happen quickly or be available in a marketplace in which pre-existing applications can be added a la carte. Of equal importance:

    Customization: services/applications need to allow customers to integrate with existing applications/solutions. The platform must be open, not proprietary.

    Collaboration: need to offer integrated collaborative solution, whether real-time via IM, web conferencing, presence or asynchronous via email, shared databases and documents

    Control: needs to work with enterprise policies and directory services.
    michaelcaton
    • Open Standards for Online Office Mashups

      Hi Michael,

      Please have a look at OpenSAM (Open Simple Ajax Mashup) - www.opensam.org. This is a new SaaS ecosystem based on open Internet standards with goals to (1) make it easy for end-users to share documents/content for mash-ups across a variety of online office applications with a single login; and (2) make it easy for app developers to achieve "speed to value" that you are referring to - i.e., make it quick (i.e., days or weeks vs. months or years) to combine one or more online office apps into a "sum greater than its parts". The focus is on open mash-ups standards to connect collaboration, document management, CRM, office productivity apps, etc.

      Best, Eric
      Eric Hoffert
    • By the time some idiot determines it has "business value" your competitors

      will already be up and running. You post really makes me laugh.
      DonnieBoy
  • SharePoint as Mashup Platform?

    Excellent stuff, Dion.

    I've been looking at a number of emerging mashup platforms, most of them startup companies taking turns telling the composite application story. While a lot of them have promise, I find myself wondering how they will compete with the big boys (IBM, Microsoft, Google, Yahoo, etc.) beyond the time horizon of the current hype cycle.

    In particular, I'd be interested in your thoughts about Microsoft SharePoint in the mashup context. While it's probably one major release cycle from enabling this at a high level, doesn't SharePoint have (or promise) everything the enterprise needs in a mashup platform?

    DR
    spindruiz@...
  • bad cites

    Why cite such disreputable outfits as Ill-Begotten Monstrosities and McKinsey if
    you want any of what you write to be believed?
    Professor8
  • Programming with gaffer tape, spaghettti and string

    or Web 2.0 as it has become known. Strangely anough we left this model behind us and only the limitations of HTML and the flexibility of the browser (essentially a picture frame) allows it to persist.

    It's a very old software development model, it's called kludgeing and no amount of scholarly critique or cute diagrams is going to change that.
    TonyMcS
    • Criticisms of Mashups as Gaffer Tape

      Thanks for the comment Tony. I would just point out that not every (or event most) pieces of software needs to be ruggedized and milspec to survive the rigors of millions of users and six sigma.

      In fact, like spreadsheets, the majority of these "applications" will be created and used by a small number of people. If they are successful, and used by more and more people over time, the platform they are based on should scale, or if necessary the IT department can decide to reinforce it or rewrite it. The key is to let the users on the ground create the solutions they need, throw them away if they are bad, or evolve them if they are good.

      And yes, it's an very old software development model, maybe even the oldest, and embracing it, supporting it, and making sure it goes in the right direction is likely the best solution, and not discouraging it and the mass innovation it represents. Just my two cents. :-)

      Dion
      dionhinchcliffe
      • Glass full/half empty

        It's a question of perspective. To this day IT database programmers all over the world shudder when they think of all the code written in spreadsheets, that works some of the time.

        You're also right, not all code needs to be well designed and elegant and debugged to a critical degree - just the stuff I want to use ;-)

        My objection was seeing this as a new software model and a rejection of the amount of hype that's been attached to it. Nor do I see this as a mass innovation, rather a fall back to work arounds due to the lack of a stable and powerful Web development platform. It also comes from using most of the sophisticated Visual IDEs (usually MS) compared to what passes as a Visual IDE for Web development. When I sit down to write in VB or C#, I have sophisticated tools. If I want to develop some web services in PHP or fiddle with Javascript or start making use of the ubiquitous plug-ins, I feel like I'm back in the 1970's ;-)
        TonyMcS
  • Another nail in the coffin of corporate IT

    This article highlights another reason why corporate IT is on its way out. The ongoing excessive and wasteful corporate spend on IT will decrease when business executives begin to understand the power the of Web 2.0 and the limitations of traditional corporate computing models. As corporate IT organisations try to figure out how mashups work, information is being disseminated and consumed at extraordinary speed on the Internet. Yet, many (if not most) large corporations cannot share information even internally (let alone software components) because of poor architecture, poor planning, bureaucracy, etc - but mostly lack vision.

    When small organisations, with little to no budget, can publish high quality information quickly and dynamically on the Internet; and large organisations trying to do the same thing, take months and spend millions of dollars - something is fundamentally wrong.

    Many professionals are more productive on their home computers connected to the Internet than on their company machines "locked down" by outdated SOE standards. Large organisations need to improve the value of their IT spend by pulling back the firewalls to protect only core applications and unleashing their users to the same open environment that we deal with outside of work. Corporations should develop strategies that harness and expose corporate content and capabilities as widgets for use in mashups through their web sites for consumers, partners, suppliers, etc. Surely, this would allow them to add value to their relationships.

    Rather than waiting for vendors to deliver some COTS mash-up tool that is "Enterprise Ready" why not just begin to use what's already available to gain some real experience. Also, focus on exposing existing information locked in legacy systems.

    In order to remain relevant, reduce costs, increase productivity, etc. pull back the firewalls, remove the "lock down" of PCs, expose content and services and set your knowledge workers free.
    mr_tony_c
  • Enterprise Mashup Tools

    IMO, most web applications in the enterprise essentially manipulate (view, edit,
    create ?) Customers, Orders, Accounts, Bills, Receipts ? typically using HTML
    forms. I?ve built many applications over time where the front end (forms)
    essentially perform CRUD operations on documents and over time I?ve found REST
    to be the ideal approach for creating these applications (see http://
    www.frevvo.com/blog/?p=26).

    What?s missing is an easy way to mash up these operations on documents to
    create Rich Internet Applications - in the enterprise, this mostly means rich,
    dynamic forms. This is a powerful software development model but for it to
    succeed, the enterprise needs Tools. Throwing together a bunch of web APIs using
    bits of Javascript just doesn't work for most corporate applications.

    Thx,
    -Ashish
    (http://www.frevvo.com/blog)
    A_D