I enjoyed a wonderful conversation with LogLogic's CEO and President, Guy Churchward, and its CMO, Mandeep Khera. The conversation started with Guy and I wandering down memory lane and chatting about places where our paths crossed over the years. Poor Mandeep wasn't able to get a word in during that conversation. Then we went on to discuss a recent survey LogLogic had conducted and its ramifications.
LogLogic's summary of the survey
Although big data continues to garner attention, it’s still largely misunderstood and underutilized for business intelligence purposes.
In a survey of over 200 security officers (titles ranging from directors to CISOs), LogLogic found that only 27% of respondents clearly understand what big data means.
Still, respondents at least know they need to worry about it—half responded that they are concerned about managing big data. The problem is that many still don’t have a centralized viewpoint to do so—59% are either not managing log data, using disparate systems, or using antiquated spreadsheets. At the same time, the amount of IT data organizations are producing is growing exponentially—with 62% producing more than 1 TB and up to 1 petabyte of data.
The conclusion is that enterprises are collecting more and more data, they know they need to analyze it, but they don’t have an easy way to do so. If the pattern continues, enterprises won’t be able to derive actionable data for both security and operational intelligence.
Snapshot analysisI've commented on surveys sponsored or executed by single vendors in the past (see Surveys and their problems for one example.) This survey appears to be a bit different. It was an attempt to discover what IT decision-makers were thinking rather than to simply put forth LogLogic's views. The results are interesting and useful. Although "Big Data" has gotten quite a bit of attention, quite a number of of these decision-makers still don't have a clear idea of what it is.
I've written on this topic in the past as well (see What is "Big Data?" for a quick review of the topic) and agree with the premise that organizations have been using some form of what is now being called "Big Data" for years. One of the most common uses of Big Data inside of organizations is the analysis of application, application framework, database, system, network and storage log files. Unfortunately, few organizations have the tools or understanding to make the best use of what is lurking in those log files.
I've spoken with a number of companies in the recent past that believe that their product or cloud service offering is the best for uncovering the hidden secrets. LogLogic is one of the few that has focused on the security and operational intelligence implications buried in that huge amount of rapidly changing data.
IT administrators might find that certain operational anomalies always occur several hours before a failure. An alert sent out when those anomalies are observed might prevent data loss or loss of a needed service. It might also be possible to catch mischief-makers attempts to penetrate security before they succeed. I agree with Guy Churchward and Mandeep Khera that it would be very wise for organizations to pause for a moment and take the time to learn what they can from this data. Tools, such as those offered by LogLogic and others, could be of great assistance.