The 10 top challenges facing enterprise mashups

The promise of remixing existing online services and data into entirely new online applications in a rapid, inexpensive manner, often referred to as mashups, has captured the software industry's imagination since the release of first major example, HousingMaps.com, in early 2005. Since then, mashups have offered the potential to finally make widespread software reuse a reality, enable SOA initiatives to achieve positive ROI, and radically drive down the cost of application development while satisfying large applications backlogs that plague organizations almost everywhere.
Written by Dion Hinchcliffe, Contributor

The promise of remixing existing online services and data into entirely new online applications in a rapid, inexpensive manner, often referred to as mashups, has captured the software industry's imagination since the release of first major example, HousingMaps.com, in early 2005. Since then, mashups have offered the potential to finally make widespread software reuse a reality, enable SOA initiatives to achieve positive ROI, and radically drive down the cost of application development while satisfying large applications backlogs that plague organizations almost everywhere.

Applying mashups in a business settings is often referred to as "enterprise mashups" and recently we've finally begun to see the tools emerging to bring real mashup capabilities to consumers, business users, and IT professionals.

However, though anecdotal evidence seem to abound -- there are a good number of stories about businesses creating isolated mashups here and there -- and mashups are again getting placed on hot tech trends lists for 2008, we're clearly still not yet seeing the flood of mashup-based apps inside of organizations despite their consistent and steadfast growth on the consumer Web.

ProgrammableWeb's mashup graphs (left of page) currently reports that over 2,400 mashup-based apps currently exist.

The public Web of course has been a global laboratory for innovation for 15 years and it's not surprising that experimentation and creativity in such a large pool of resources of people and services would generate some interesting outcomes like the several thousand mashup applications currently available. But the question has been: Where is the same result inside our organizations? Those same organizations that often desperately need software to solve a business problem for which software simply isn't available -- at least without extensive customization -- because the typical business problem's unique, situational nature. In previous posts I've discussed how spreadsheets are often the only end-user development tool available to the average person to meet this need today.

So what exactly is holding back enterprise mashups from becoming a more popular phenomena inside our organizations? This has been in contrast to many other aspects of Web 2.0 inside the enterprise, where openness, network effects, and radical power and simply are often driving extremely fast uptake and adoption of new apps and technologies. By many indications, mashups -- particularly in the enterprise -- have so far fallen short of their potential and the question is why?

I've discussed this with a various people in the mashup community and analyzed a number of the leading mashup platforms and have boiled the outstanding challenges down to 10 points I believe are the biggest obstacles. Some of these challenges are counterintuitive and some of them might be self-evident to those that are struggling to enable them but nevertheless are still major issues. But given that SOA, composite apps, Web services & open APIs, and widgets are all receiving significant venture capital and corporate investment in 2007 and heading into 2008, the enterprise mashup space is poised for tremendous growth by all other indications. If only we could discover and resolve with the remaining barriers for their creation and use. Here are what seem to be the biggest issues with widespread adoption of enterprise mashups:

The top 10 challenges facing enterprise mashups in late 2007

  1. No commonly accepted assembly model. The reason that spreadsheets are still so enormously successful is that they are really a highly capable functional programming environment. A spreadsheet has an easily understood visual grid model that can hold data of many different kinds that can then be processed by cell references and user written formulae turning hours of previously tedious work into automatic processes. Spreadsheets are largely intuitive, easy to learn, and used by millions of people on a daily basis. It's been so successful that some mashup tools actually use them as their preferred model, most notably the pretty interesting Instacalc and BEA's AquaLogic Data Services Platform that offer good consumer and enterprise example respectively, though there are other players as well. Spreadsheets offer a mashup fabric in a model that is already proven and generally accepted by consumers and business users virtually everywhere.But most mashup tools take a different tack and either reinvent the wheel by offering completely new and unfamiliar types of models, some simple and some quite sophisticated but virtually all of them foreign to anyone who is not a developer. And of course the issue is if mashups remain a developer only phenomenon the larger bulk of their business value will be lost. If only developers could process data with computers, we'd still be in the era where personal computers were primarily used by the few people with coding skills. Spreadsheets liberated computers to be used for number crunching by the masses and mashups have the potential to liberate the browser for software creation by the masses. The big question is who will hit upon the right model that will provide both an incredibly easy on-ramp that also supports the creation of useful business applications but one that can support sophisticated applications and be easily maintained and supported over time.Is the spreadsheet model for mashups the answer? It's too early to tell, but with IBM's QEDWiki and Serena's functional flow model in their business mashup product for end-users, we've seen compelling alternative models. So the jury is out on the best mashup model, and when that jury finally comes in -- in the form of the mashup platform that gets fast uptake -- we'll have a much better sense of which direction is best.
  2. An immature services landscape. Mashups are predicated upon the ready preexistence of ready-to-use Web services and network APIs which are ready to be used to build on top of. Though many of the leading Internet firms have extensive Web services divisions that offer much of their data and products up through feeds that can be plugged into browser-based mashups, the reality is that the rest of the Web has been somewhat slow to follow. It's just now becoming well-known that APIs can greatly reinforce the value of a Web application and allow it to be reused hundreds or thousands of times in other products, leveraging a form of Jakob's Law.But for now, even in organizations that have SOA initiatives in advanced stages, not nearly enough information and functionality is currently "Web-enabled" via feeds and services and hence can't be leveraged inside a mashup application canvas. Products such as Kapow and other Web-clipping tools have put a decent dent in this problem recently, but we have a way to go before we have enough ready Web services to feed our mashups. The good news: There are a lot of high value APIs that do exist on the open Web today and increasingly inside our organizations particularly in the form of RSS that can unleash the mountains of data and functionality lying largely fallow inside most businesses today.
  3. The splintering of widgets. One of the most exciting phenomenons on the Web today has been the advent of widgets; mobile Web parts that can be used by anyone with a cut and paste. Web widgets actually form the basis of a primitive yet highly end-user friendly mashup model and have been gaining enormous popularity as a quick look at the widget gallery on WidgetBox shows us. It's the user remixable Web at its best and widgets are distributed and used by millions of people a day.It also turns out they are the other key ingredient to mashups along with Web services/APIs but they are uniquely significant because they are entirely end-user friendly whereas most APIs require much more technical savvy to use. Consequently, widgets make a compelling and ubquitious packaging system for mashup functionality, particularly since most widgets also provide automatic API connectivity back to its originating site (for example an eBay listing widget connects back to eBay from a mashup or Web page live to pull auctions from the API.) However, the widget world is very informal and there are no official standards (yet) and widgets have been co-opted by countless companies and named badges, gadgets, modules, and other things. Some of these companies, particularly Microsoft, Google, and a few others, have been imposing their own gadget models that allow businesses to plug into their ecosystems of users but largely keep their gadgets captive.The real issue is that mashup creators need to have access to the Web's full inventory of functionality and the different widget models are fostering silos of widgets that it's not likely most mashup creators will have full access to with their tools. The solution? A simple, open standard for widgets. Are we likely to get it? Not soon unless someone with authority, in a manner similar to the way RSS 2.0 was created and spread, steps up to the plate.
  4. Management and support of end-user mashup apps. If enterprise mashups unleash hundreds of new applications inside an organization, then who will catalog them, support them, maintain them, and fix them when they break? The IT department? The business units? Using what tools? This is an objection I frequently get from enterprise IT about their fears that mashups will bring back the horrors of having unsupported Microsoft Access-based apps running loose in their organization, though much more widespread because of their collaborative, open, network-distributed nature. This is a genuine challenge and the best mashup platforms have some answers to this baked in but there is lots of room for improvement.
  5. Deep support for security and identity. The most useful enterprise mashups will have access to a user's private individual data and other corporate information protected by security and identity systems. Consumer mashups are challenged in a similar way since it's hard to provide vital function if a user isn't willing to trust a mashup with the user IDs and passwords necessary to obtain the data from the back-end services guarding the information the mashup needs to function.The common enterprise solution to this, Single Sign-On (SSO), has not made it fully to the mashup world and the problem is only exacerbated if a mashup is also meant to be used by many types of users including consumers, partners, and suppliers which will all have their own identity requirements and infrastructures. Initiatives like openid are helping but are not a complete solution. Like widget models, this too needs a leadership figure to definitely resolve for the industry or many types of useful mashups are not and will not be possible in the near future.
  6. Data quality and accuracy. This is the classic "truthiness" problem Nick Carr brought up way back when mashups first began to get attention and are still a critical issue as far as enterprise mashups are concerned. How do I know if the data displayed in a mashup is correct I'm often asked? Particularly in many types of applications, the veracity of data is paramount yet in a browser-based mashup, who is to say where the data really came from and if it's fresh. Formal IT systems have an advantage in this regard, at least for now, because they have controls to ensure they are the system of record. However, the recent advent of reputation systems and provenance indicators can help but there's lots more work to be done here so that users can be sure of what they are looking at, where the data came form, how it processed, and how recent is it?
  7. Version management. When users can tweak, recombine, share, adjust, and otherwise change a mashup on-the-fly in "edit mode", how does a business deal with issues of consistency and change management when so many more versions of applications exist than before. Fast rates of change are increasingly becoming the norm both on the Web and in our organizations and mashup platforms that don't directly have features to support and enable rapid changes in mashup applications will be bringing to their customers as many issues as they solve. IBM's QEDWiki stands out again as a model where mashups are automatically versioned, just like a wiki page, so that even the mashup can be mashed up again and its clients pinned to particular version. Several of the most recent mashup platforms are making this kind of capability a priority and by the end of 2008, mashup platforms that don't support version management robustly will likely have serious issues gaining an audience.
  8. Awareness and realization of the potential of mashups by the businesses community. One other problem, probably not helped by the term 'mashup' itself, is the general awareness of the business value of mashups and their potential to solve tough business problems by providing faster, cheaper access to the right information and IT capabilities than ever before. The fairly immature state of tooling doesn't help but a lot more has to be done to educate and bring awareness to the general business community and educating them that an important new model for software is emerging. Businesses should be aware of the potential that smart, appropriate application of mashups in many business problems can bring by unleashing productivity, increasing institutional knowledge, and creating new business opportunities and outcomes that simply weren't possible before. Prior to mashups, there was generally few ways to buy or create a custom software application in a timely, inexpensive manner.
  9. Low levels of support by major software firms. Though open source software tends to power the public Web, a large percentage of the infrastructure running businesses today, especially in medium to large organizations, are still largely based on commercial software. And commercial software vendors have been slow to provide explicit support for an enterprise mashup friendly environment. What's lacking? A number of things including support of Web-Oriented Architecture, offering application functionality in useful API forms, "widgitized" content and functionality, offering rich support for RSS feeds and notification, mashup security solutions, and other related topics. This ensures that enterprises have a lot of work to do on their own before mashups are commonplace in their organizations.The good news, Service-Oriented Architectures are growing more common that understand mashup approaches, which strong prefer standards that support easy consumption in the browser, will help but commercial software needs to be much more mashup friendly. Why are commercial software firms slow to support mashups? Part of it is the typically slow pace of commercial software development as well as a wait and see attitude by many to see how mashups impact their business models, particularly around professional services and integration.
  10. Few killer demo mashups. Because of many of the issues above, particularly around security, clearly useful business mashups can be hard to find. Where is the mashup that shows the average user an easy-to-use combined view of all of their calendars: home, business, and mobile? How about the app that allows someone to wire together data from the data warehouse, local SOA, and enterprise content management system? These are still to hard to create for the reasons in the list above and until compelling stories emerge from businesses that have reached the tipping point, enterprise mashups will, like the SOAs that power them, remain a fascinating idea for most but not a significant business motivation.

You might think I paint a bleak picture of the enterprise mashups world and nothing could be further from the truth. The last couple of years have taught us an enormous amount about what is possible and what needs to be changed. Most of these items above will likely be addressed in some form or another in the next year or two. Even solving a couple of the key issues will change the software development landscape considerably. In the meantime, I'll do my best to provide a front-row seat to the unfolding story of what appears to be the next major new software development model.

Are you building mashups or using widgets and feeds in your business? Tell your story in TalkBack below.

Editorial standards