The consequences of information overload

We've got the makings of a classic salon discussion right out of the playbook of Gertrude Stein emerging here at ZDNet. Dion Hinchcliffe, responding to a thought-provoking post about the nature of change by Mitch Ratcliffe, presents his perspective on what the increasing availability of information means as a change vector for societal evolution. Are we experiencing a profound, technology-driven change (as Dion advocates) or are things essentially the same except for the rhetoric (as Mitch argues)? It's a fascinating topic and one that has been argued in one form or another at every stage of societal and technological evolution dating back, probably, to the first hominids arguing about whether the newfound ability to make fire was essentially good or bad, democratizing or power consolidating, in nature.

Mitch argues that things really haven't been changed all that much by the latest wave of social software, increasingly ubiquitous access to vast stores of information (Google, Wikipedia), and reduced economic barriers to creation and consumption of all of this data. Dion counters that there is an essential difference in the (r)evolution we're currently experiencing for what has come before. There's little point in trying to do justice to either of these thought-provoking and rather lengthy posts by attempting to summarize them here. I highly recommend you read both and see which resonates more with your own thoughts on the subject. It's time well invested as it's quite obvious both of my fellow ZDNet bloggers have given the topic a lot of thought and have arrived at conclusions they consider both defensible and well worth discussing.

A couple of choice quotes may give you the flavor of what each is saying.

Mitch writes:

Unfortunately, some among the newly enlightened think something quasi-religious has happened to society. It hasn't. If we were pragmatic about it, we'd admit the truth of this. Because it feels so good to think that your generation invented everything we're going to go on claiming it is the era of S.H.I.T. (Simple Hacks of Intellectual Transactions), when no wrong will happen and those few that do will be swiftly punished by the wisdom of the crowd or the blink of personal judgment.

A collective unconsciousness hasn't been summoned out of silicon and Ethernet, but the people seeking to gain the most from the rise of networked society want you to believe it. And the fact that you get so much information thrown at you any time you make a query is proof that, so far, there's very little in the way of added value. We've got a lot of added volume, enough to make anyone think they're getting more information.

I can't argue with this and the idea that and increase in quantity somehow confers  a commensurate improvement in quality has been a frequent point of discussion in the engagements I've had with the folks working on MSN Search and the Search Champs they've invited on a semi-regular basis to spend a few days at a time trying to envision how all of this raw data we're collectively producing can be used effectively. The simple fact of the matter is that more isn't better, at least not in any Q.E.D. automatic fashion. Whether a search engine returns 10,000 results or 100 on a given query is less meaningful than how well-matched the first page or two of results are to what I was looking for. The rest is irrelevant to my direct search experience and only relevant at all if the higher data quantity drives increased relevance in the results. From a user perspective all I care about is getting the best answer, not how it's accomplished.

More to the point of the discussion, Mitch argues that the surface-level democratization so many people seem to be celebrating is a mirage -- one perpetuated by the relative few who are profiting from the illusion. The more things change, the more they stay the same.

Dion has a decidedly different and equally well-presented perspective:

Mitch however argues that the power (to change things) was always there and disruptive ideas and technologies can come from anywhere, but are more likely to come from outside the walled gardens where folks have built institutions to lock in the status quo. The fallacy in his argument is that it could come from anywhere.

Before now, the barriers to new and disruptive ideas and technologies have always been quite high. You had to be very lucky, very resourceful, or had access to real wealth in order to overcome the natural barriers in place. Influential and respected people like media baron Ruport Murdoch seem to fully realize this as he witnesses the seeming fundamental reordering of media with things like widespread blogging, peer production news, and with things like YouTube, Skype, and a horde of others fundamentally democratizing communications and media. So as I stated above, this means that the means by which things change is itself changing, in that the ability to influence, interact, and communicate with people anywhere is now essentially free of cost.

It's hard to argue with this notion - that the most recent waves of technological innovation have, in fact, democratized the process of being able to precipitate change on an extraordinary scale. The long-term impact of the increasingly social web remains to be seen -- 20-20 hindsight is always easier than future projecting as an intellectual exercise because the risk is removed by being able to look back and say, "Well, you see, it's quite obvious that this was the early sign that things had irrevocably changed."

And all rhetoric and emotionalism aside, it's hard for me to think that the advent of massive social spaces of both the semi-walled (MySpace, Digg, Flickr, LinkedIN) and free-for-all (blogs, "instant" web sites via Google Page Creator or Apple's iWeb) variety aren't a significant harbinger of permanent change. Newspapers are shrinking, both in physical page count and in authority, before our eyes. Commercial radio and television are being assaulted by podcasting and its video counterpart, whatever it will end up being labeled. Even books, while they remain an important form of intellectual exchange, are moving from a purely paper-based construct to an acceptable digital form as most recently evidenced in a by the successful publication of Getting Real by the folks at 37Signals.

Increasing accessibility to an immense and exponentially growing corpus of knowledge has to produce an effect. Again, the jury (my jury in any event) is still out as to whether that effect will be profound (as Dion argues) or will merely create a new set of information moguls raking in the profits (as Mitch seems to believe).

What do you think? Are we on the cusp in a technology-driven reweaving of societal fabric? Or are we seeing a rerun of a cycle that differs only in its trappings from what every wave technology that has preceded this one has generated?