If RSS ain't broke...

The Microsoft evangelist sounded an alarm about RSS polling characteristics, as exemplified by a Microsoft Developer Network group blog that aggregates some 1,000 Microsoft bloggers in a single full-text feed. The MSDN brain trust, apparently caught up in Steve Ballmer's program to trim billions out of Microsoft operating budgets, trimmed the feed down to links to the full text of posts.

To Scoble, this was an indication that RSS (Really Simple Syndication) is broken. Specifically, the default setting of one-hour polling in RSS aggregators is broken. In the MSDN case, every time one of the 1,000 bloggers posted an entry, the full feed would need to be downloaded all over again to thousands of subscribers.

First of all, as blog and RSS pioneer Dave Winer and others have noted, the MSDN problem is one of bad design, not specification. Aggregating all those feeds virtually mandates that the entire feed must change. Yet, RSS aggregators, whether on the client or the server, can manage each of the individual feeds at much less overhead, triggering an update only when each individual feed is changed. So the prescription is the same as the one given by the doctor when told by the patient: "It hurts when I do this."--"Don't do that."

But Scoble extrapolated from that initial problem to the more pervasive issue of polling frequency. Here, he repeated the common wisdom about RSS poll intervals--they should be no more frequent than once an hour. In fact, Scoble, who scans more than a thousand feeds every night after work, would be just as happy with one update a day.

So I called Scoble up and asked him this question: "How frequently does your e-mail client update?" "Every few seconds," he said. OK, so let's say we use a less efficient protocol like POP3. I set mine to poll once a minute. Outlook comes set to a 10-minute default, which is just long enough for a mail thread to spin dangerously out of control if I'm caught in the middle of a time-sensitive discussion, such as arranging a group meeting or covering my deadline butt with a trigger-happy editor. Instant messaging rode into the enterprise on this horse.

But wait, you say. What about all these subscribers hitting at the same time, every hour, or 'Grid' forbid every 30 minutes like NetNewsWire lets you do. Even if the server returns a code that indicates there is no new data, that conversation can get expensive. And, what happens if Microsoft finally gets it (or some version of it) and puts RSS in Internet Explorer. Suddenly there are millions of browsers set to the same dumb default on the hour.

Well, if Microsoft wants to release a denial of service attack of such egregious proportions on its customers, it will be a great boon to FireFox marketing, not to mention aggregation services such as Bloglines and Feedster that are already growing virally as software-as-a-service plays. As they say, bring it on.

Then there are the real fanatics like me, who don't want no stinkin' hourly intervals, thank you. I want my POP3 intervals, or even better, event-driven intervals powered by peer-to-peer. And before you get too irate, I want more data, not less, via full-text-and-graphics feeds where appropriate, and RSS enclosures via BitTorrent, Bram Cohen's open source peer-to-peer file distribution software.

You see, there's this new platform -- the iPod Platform -- that's emerging to compete directly against the broadcast channels of television and radio, and against the page view portal models of the Web. Already today there's a loosely-coupled software-hardware bridge that's delivering radio-on-demand to information-starved audiences trapped in their drive-time and exercise routines.

We've all heard the Newton Minnow quote about the vast wasteland; that was when there were only three networks and 10 radio stations per market. Then came the 500 channels model of cable, satellite, and HBO. Now it's Wi-Fi, TiVo, and RSS, where we jump from thousands to millions of channels. And we have Adam Curry assembling his Daily Source Code program with a single PowerBook and one outboard sound processor that sounds as good as or better than any program on the dial. In fact, developers on Curry's Yahoo mail list report Apple's next version of the operating system, code-named Tiger, will let audio producers produce broadcast-quality shows without external devices.

That's the creation side. On the delivery side, Curry and a group of open source developers are bootstrapping scripting languages, iTunes, RSS, and BitTorrent to automate the capture, meta-data tagging, and transfer of MP3 files to the iPod from both the Mac and Windows machines. Server-based aggregators such as Feedster are getting into the act with RSS enclosure services. Third-party adapters such as Belkin's TuneCast II Mobile FM transmitter pushes content to car or stereo speakers. Or skip the iPod completely at home with Airport Express and AirTunes.

Apple is looking over its shoulder at phones such as the new Nokia 6620, which supports at least 75K downstream -- plenty of bandwidth to enjoy the Gillmor Gang at AT&T Wireless' $29 all-you-can-eat flat fee data rate. It's a small step from there to a hybrid iPod that caches Wi-Fi-enabled BitTorrent feeds with news, event notifications, and traffic data.

Nowhere in this vision -- or reality, if you're already on board -- is the notion that notification and delivery of updates needs to be constrained. If anything, the needle is swinging hard in the opposite direction -- toward a fabric of an always-on, always-connected information stream. In that world, the gating factor is the user's ability to efficiently consume the information.

RSS bottlenecks are the early indicators of business models for the new architecture. Witness Slashdot, which constrains the number of RSS requests per hour. Alternatively, services that offer more frequent rates can charge for the privilege. A multi-tiered cost structure can offer variable rates based on the relationships between customers, suppliers, and partners along the supply chain. High-value customers might pay nothing for rich feeds, just as Amazon throws in shipping above a certain order amount.

The rewards for adopting the RSS model are greater for those who lag in the current online economy. By contrast, Microsoft has little apparent incentive to destabilize Office by extending the free browser to support not just content aggregation but creation. Yet that is exactly what the competition is moving toward: an RSS console that automates the capture, consumption, and routing of strategic information.

Just as blogging drove the initial adoption of RSS aggregators, so too will moblogging (short for mobile blogging) drive the addition of rich media capabilities to the RSS information router. Whether it's Dave Winer's imagined "big red record button" on a future iPod, a suite of audio mixing tools that mix iChatAV, VoIP, microphone, MP3, and GarageBand feeds, or an intelligent router that uses attention data from aggregator services to predict where BitTorrent caches need to be positioned to allow reliable and scalable feed distribution, the resulting blend of these tools will certainly be a killer app for information professionals and consumers alike.

Steve Gillmor is a Contributing Editor at ZDNet and host of IT Conversations' The Gillmor Gang Web radio program.