Information delivery in a Web 2.0 world

Ever had that needle in a haystack feeling when looking for your own personal information or scouring for corporate data or files?

The perceived value of information technology is far too often equated with its ability to collect and consolidate vital business data. But as corporations become increasingly sophisticated in data use and analysis, they've also morphed their view of data itself. We've evolved to the point where having access to raw data simply doesn't cut it.

Data is not information unless you can find it. But information can't be applied to knowledge in the absence of the means by which to use it. Access must yield meaningful information in order to turn bits of code into valuable and actionable business information.

I like to think about it this way: libraries house rich caches of data in the form of books, but finding the exact piece of information you seek requires some level of research and library science expertise. Wouldn't it be ideal if the information you seek could somehow be collected and organised for you, in the format and context in which you want it? And wouldn't it be even more helpful if it was assembled for you, not just from the local collection, but from libraries in Singapore, Milan, Minneapolis and Copenhagen?

Some early examples of this new approach to information management illuminate its enormous possibilities.

How cumbersome was your digital musical collection before you installed iTunes and it swept your PC, consolidating all of your files in one beautifully organised folder? And tagging (as we call it in our personal lives) and classification (as we refer to it when we're talking about corporate content) is what makes a meaningless photo meaningful and provides clues to the importance of an archaic customer record or data file. Tools such as these are the early leaders and signal a massive shift in the way we organise, access and use information.

Being buried in data isn't a new problem, but the issue has grown exponentially in recent years, as more and more data pours through corporate networks and the internet. IDG recently dubbed this phenomenon the "digital big bang", and a quick look at data growth shows why. In digital terms, 161 exabytes (or 161 billion gigabytes) of information was created, captured and replicated in 2006. But by 2010, this number will explode to an estimated 988 exabytes.

Much of this data will be created by you and me, individuals. IDC found that 70 percent of the data is created by end users and over the web. In one day, YouTube streams more than 100 million videos, while one billion MP3 files are shared over the internet daily. The increasing convenience and ubiquity of digital devices also add to the explosion. We used our mobile phone cameras to snap about 100 billion digital photographs last year, for example.

As more visual and aural data pours online, the need to organise and access it becomes more pressing. After all, vacation photographs become rather pointless if you do the digital equivalent of throwing them in a shoebox and hoisting them into the attic.

With such drivers, it's not surprising that access to information has been one of the driving forces of new platforms and services. The arrival of Web 2.0, open APIs and open-source development have made it all possible. And the success of services such as YouTube further underscores the need for easy access to information. Videos have been online as long as we can remember, but it took YouTube to organise and turn them into meaningful content that could be accessed from anywhere, without manually trolling individual speciality sites.

Fighting an uphill battle
The exploding need to access and transform data into information has sent repercussions throughout the computer industry. Enterprises around the world are fighting an uphill battle against the sheer volume of data and its management. As a result, corporate needs have moved beyond just "How do I store all this data" to "How do I manage it all?".

In response, we've now embarked upon a more interesting phase where all of these technologies are being woven together into an information infrastructure. Now we can be thinking more progressively about information value delivery in its entirety. This is where Web 2.0 meets information infrastructure.

Web 2.0 flips the information delivery model upside down — it's now about global access, and information at your fingertips, aggregated from sources that you don't even necessarily know about, or care where they exist. Based on a set of search criteria, information in all its rich forms — media, video, audio, images, documents, text — all will be assembled together in context and delivered to users and applications for real-time experience.

This is the promise. The opportunity lies before us to finally unlock data and turn it into valuable knowledge, much as Watson and Crick transformed a string of meaningless genetic instructions into a double helix with limitless information when they cracked the DNA code.

As information is effectively harnessed, hidden insights will appear that were previously buried in mountains of unorganised data, and more and smarter discoveries will result.

It's not too much to say that a whole new world lies before us — finding effective ways to organise and access information regardless of its origin could spark a renaissance in the way knowledge workers think and perform their jobs.

Mark Lewis is chief development officer of EMC.