Throw something up on the Web and you never know what someone might do with it. That proverb of unintended positive consequences was proven yet again this week by the targeting of the power of the Splunk computer log indexing-and-search capability onto the trove of the recently released, mostly (but not entirely) anonymized AOL search data.
You probably heard all about the AOL thing, and the unfortunate result (for AOL's image). But then there's this mine of readily available data. But how to best access the intrinsic trends, quickly, easily, and for pretty much free?
I want my meta data!
Someone employed creative thinking and -- acquainted with Splunk, which is designed to index and make searchable huge amounts of time-stamped and arcane log data from network and datcenter systems -- put the Splunk server technology to work on the reams of AOL data. And, of course, they made it all open and freely available on the Web.
It became so popular, and so many people were searching on the Spunk-generated meta data results that they had to add several mirror servers to handle the demand.
Among the positive results -- in addition to the search user analysis, err ... benefits -- is a ramped-up, fully visible test drive of what Splunk can do. The Splunk performance characteristics managed to far exceed even what its designers ever intended -- thinking that the engine was to be used primarily for internal systems administrators to use to troubleshoot and fix networks and datacenters.
I learned a lot about Splunk and Splunk Base when I did some sponsored podcasts with them earlier this year. Incidentally, expect a new version of the server, Splunk 2.1, at Interop in New York in September.
I do wonder what new needles in what new haystacks the Splunk technology can unearth next?