Mashups: who's really in control?

By | March 2, 2006, 1:48am PST

Summary: In the mashup ecosystem, let’s get one thing straight. The data owner is ultimately in control, because a mashup developer is reliant on data owners to keep the supply of data flowing. Sometimes data services that were open to begin with, then have restrictions imposed on them. Or the data services may even be taken away [...]

In the mashup ecosystem, let’s get one thing straight. The data owner is ultimately in control, because a mashup developer is reliant on data owners to keep the supply of data flowing.

Sometimes data services that were open to begin with, then have restrictions imposed on them. Or the data services may even be taken away completely, if the owner of the data decides they don’t like what the mashup does to their bottom line.

In 2005 there was a graphic illustration of this, Data Owners hold the balance of power in the world of  mashups when vertical employment search engine Oodle found that one of their main sources of data - the online classifieds market leader craigslist - decided to disallow Oodle from scraping their data. In fact there was never an explicit agreement by craigslist to allow services like Oodle to scrape their RSS feed and use that data in a mashup.

There are many similar circumstances in the Web world today, where the co-operation for data services is implicit and not explicitly defined. Indeed mashups poster child HousingMaps is potentially in the same position as Oodle, because they too rely on craigslist’s RSS feeds.

But why did Oodle receive a ‘please desist’ notice from craigslist, yet Housing Maps continues to enjoy the benefits of using craigslist data? In an article in SFGate.com at the time, craigslist chief executive Jim Buckmaster explained that Oodle’s automated scraping of listings was slowing down the craigslists website and costing it "thousands of dollars for extra bandwidth and manpower". On the other hand, Buckmaster said that HousingMaps was allowed to continue scraping its RSS because it is "a noncommercial, one-man shop that has generally won praise from Craigslist users."

So it seems that putting a heavy server load on the data source, particularly if you’re profiting off it but not giving anything back, is likely to land your mashup in trouble.

To put it bluntly, data owners hold the balance of power in this new world of Web mashups. Some data owners, like Google and Yahoo, provide formal APIs and are careful to explicitly define restrictions on what external developers may do with their data. Some data owners, like craigslist, don’t provide APIs and are more implicit and arbitrary about whom they allow to use their data.

Either way the fact remains that data owners can easily block off the air supply for mashups, either with a business decision (as in the craigslist-Oodle case), a policy update for an API, or by simply changing the data or technical rules around it.

Kick off your day with ZDNet's daily e-mail newsletter. It's the freshest tech news and opinion, served hot. Get it.

Topics

Biography

Richard MacManus, formerly a ZDNet blogger, is a Web consultant and writer based in Wellington, New Zealand. He specialises in next generation web technology and runs a popular weblog called Read/WriteWeb on this topic. He currently does research, analysis and writing for Internet companies in Silicon Valley, the UK and beyond. Prior to that, he worked with some of New Zealand's top commercial companies as a web manager and producer.

9
Comments

Join the conversation!

Just In

0 Votes
+ -
Reason #419 why mashups are a gimmick...
Justin James 2nd Mar 2006
... and not ready (or can ever be ready for) Enterprise use! Are you going to place your entire business at the whim of some third party you have zero control or binding relationship for? If so, you're an idiot, your investors/shareholders are idiots, and you should be put to jail for fraud.

Frankly, I find the idea of thinking that starting a business based on the idea of using someone else's copyrighted information without their express permission highly offensive at best. Who do you think you are? Someone else worked hard to produce that content, and you go ahead and make a buck on their back. Search engines fall under this category. They should not be allowed to spider or index a site unless that site owner had explicitly given them permission, such as entering the URL into a search submission page, double-checked by sending an email to the administrative account for that DNS entry in whois that contains a single-use URL for confirmation.

I could not care less about "innovation", this is unethical. Mashups that don't explicitly have a business relationship, or where the data content creator has created a license for the content that allows "Free For All" access should not happen. And you should never base your business on getting a stream of "free" content from anywhere.

If your business model requires that all you do is spend some time coding then sit back and watch money roll in, you've been reading too many junk emails and actually beleiving their promises.

J.Ja
0 Votes
+ -
Whoa there partner
RStiennon 2nd Mar 2006
Got a spleen to vent today? Sheesh. Go over to Salesforce.com and their app store if you want to see at least the beginnings of mash ups for the enterprise. You are going to be even more bitter than you are today when you realize you missed out on something big.

-Stiennon
0 Votes
+ -
Are they actually up and running today?
Justin James 3rd Mar 2006
Or are they just taking their customers' money while displaying a big fat "we're currently offline" message? Because Salesforce.com has been doing a lot of that lately. They are the last company I would give money to, certainly not for a piece of mission critical software that resides on their systems within their network.

J.Ja
0 Votes
+ -
Or Trillian and AOL?
RStiennon 2nd Mar 2006
Predates "mashups" but remember AOL and MSN trying to block Trilian by frequent changes to their protocols?

-Stiennon
0 Votes
+ -
Strong Warning for Startups
Jhaus 2nd Mar 2006
Excellent peice! A very under-reported aspect that deserves more attention...thank you.

This has been a major concern of mine as I've planned out my own mashup. The data owners truly do have significant power over this new trend. Clarity lacks in this area when looking to the law; hence, we are at the mercy of the data owners. Although, it's always possible to purchase data - it's proven difficult (at least for me) to find a reliable, timely & accurate provider for a reasonable fee...Hasn't Alexa recently come out with some sort of offering in this respect? Does anyone have experience with this?
0 Votes
+ -
Lack of clarity from the law?
serpentmage 3rd Mar 2006
Sorry, but there is no lack of clarity from the law. The law states very clearly that a company can sue you for copyright violations. Where there is an exemption is for the purpose of search engines which are references to the data.

Way back in 1999 when I was involved in a dot com we had to deal with this situation. What is different now is that "mashups" have become popular and trendy.
0 Votes
+ -
www.HomePriceMaps.com compliments thie HousingMaps site quite well.

while HousingMaps integrates Craigs list homes currently for sale and rent with Google Maps, www.HomePriceMaps.com integrates how much homes SOLD for with the google mapping technology
www.HomePriceMaps.com integrates
how much homes SOLD for nationwide using the google mapping technology.

Simply select city and state from the city menu and click search.

In addition, we are now offering a real-estate mapping package for real estate professionals which allow you to map and display your real estate listings on your website.

Join the conversation!

Formatting +
BB Codes - Note: HTML is not supported in forums
  • [b] Bold [/b]
  • [i] Italic [/i]
  • [u] Underline [/u]
  • [s] Strikethrough [/s]
  • [q] "Quote" [/q]
  • [ol][*] 1. Ordered List [/ol]
  • [ul][*] · Unordered List [/ul]
  • [pre] Preformat [/pre]
  • [quote] "Blockquote" [/quote]
ie8 fix

The best of ZDNet, delivered

ZDNet Newsletters

Get the best of ZDNet delivered straight to your inbox

Facebook Activity

White Papers, Webcasts, & Resources
ie8 fix