Mashups: who's really in control?

In the mashup ecosystem, let's get one thing straight. The data owner is ultimately in control, because a mashup developer is reliant on data owners to keep the supply of data flowing.

Sometimes data services that were open to begin with, then have restrictions imposed on them. Or the data services may even be taken away completely, if the owner of the data decides they don't like what the mashup does to their bottom line.

In 2005 there was a graphic illustration of this, Data Owners hold the balance of power in the world of  mashups when vertical employment search engine Oodle found that one of their main sources of data - the online classifieds market leader craigslist - decided to disallow Oodle from scraping their data. In fact there was never an explicit agreement by craigslist to allow services like Oodle to scrape their RSS feed and use that data in a mashup.

There are many similar circumstances in the Web world today, where the co-operation for data services is implicit and not explicitly defined. Indeed mashups poster child HousingMaps is potentially in the same position as Oodle, because they too rely on craigslist's RSS feeds.

But why did Oodle receive a 'please desist' notice from craigslist, yet Housing Maps continues to enjoy the benefits of using craigslist data? In an article in SFGate.com at the time, craigslist chief executive Jim Buckmaster explained that Oodle's automated scraping of listings was slowing down the craigslists website and costing it "thousands of dollars for extra bandwidth and manpower". On the other hand, Buckmaster said that HousingMaps was allowed to continue scraping its RSS because it is "a noncommercial, one-man shop that has generally won praise from Craigslist users."

So it seems that putting a heavy server load on the data source, particularly if you're profiting off it but not giving anything back, is likely to land your mashup in trouble.

To put it bluntly, data owners hold the balance of power in this new world of Web mashups. Some data owners, like Google and Yahoo, provide formal APIs and are careful to explicitly define restrictions on what external developers may do with their data. Some data owners, like craigslist, don't provide APIs and are more implicit and arbitrary about whom they allow to use their data.

Either way the fact remains that data owners can easily block off the air supply for mashups, either with a business decision (as in the craigslist-Oodle case), a policy update for an API, or by simply changing the data or technical rules around it.