Is 'Big Data' good or evil? New survey exposes growing debate

The potential of 'Big Data' -- those petabytes' and exabytes' worth of structured and unstructured information now being generated and gathered by users and systems across the globe -- is bright, presenting new opportunities for growth and innovation. However, the world may still not be ready to effectively handle and understand all this data.

That's the conclusion drawn from a recent survey of 1,021 Internet experts and other Internet users, published by the Pew Research Center and the Imagining the Internet Center at Elon University.

Tech experts believe the vast quantities of data -- so-called "digital exhaust" -- that humans and machines will be creating by the year 2020 could enhance productivity, improve organizational transparency, and expand the frontier of the “knowable future.” But they worry about “humanity’s dashboard” being in government and corporate hands and they are anxious about people’s ability to analyze it wisely.

A majority, 53%, felt positive about the enhanced capabilities Big Data would deliver, agreeing with the statement that "human and machine analysis of large data sets will improve social, political, and economic intelligence by 2020. The rise of what is known as Big Data will facilitate things like real-time forecasting of events); the development of "inferential software" that assesses data patterns to project outcomes; and the creation of algorithms for advanced correlations that enable new understanding of the world."

However, that's a slim majority. There's another 39% of internet experts who express concern. They agreed with the counter-argument to Big Data's benefits, which posited that "Human and machine analysis of Big Data will cause more problems than it solves by 2020. The existence of huge data sets for analysis will engender false confidence in our predictive powers and will lead many to make significant and hurtful mistakes. Moreover, analysis of Big Data will be misused by powerful people and institutions with selfish agendas who manipulate findings to make the case for what they want."

As one of the study's participants, entrepreneur Bryan Trogdon put it: “Big Data is the new oil,” observing that “the companies, governments, and organizations that are able to mine this resource will have an enormous advantage over those that don't. With speed, agility, and innovation determining the winners and losers, Big Data allows us to move from a mindset of 'measure twice, cut once' to one of 'place small bets fast.'”

The Pew/Elon report notes that user-generated information -- such as that coming from social networks or transactions -- "is only part of the story, a relatively shrinking part. Machines and implanted sensors in oceans, in the soil, in pallets of products, in gambling casino chips, in pet collars, and countless other places are generating data and sharing it directly with data 'readers' and other machines that do not involve human intervention."

Government leaders, scientists, corporate leaders, health officials, and education specialists "are anxious to see if new kinds of analysis of large data sets can yield insights into how people behave, what they might buy, and how they might respond to new products, services, and public policy programs," the report states.

Concerns voiced include privacy, as more and more data is collected about people—both as they knowingly disclose things in such things as their postings through social media and as they unknowingly share digital details about themselves as they march through life. "Not only do the advocates worry about profiling, they also worry that those who crunch Big Data with algorithms might draw the wrong conclusions about who someone is, how she might behave in the future, and how to apply the correlations that will emerge in the data analysis," the report cautions. "There are also plenty of technical problems. Much of the data being generated now is unstructured and sloppily organized. Getting it into shape for analysis is no tiny task."

Some opinions voiced:

Jeff Jarvis, professor, pundit and blogger: “Media and regulators are demonizing Big Data and its supposed threat to privacy/ Such moral panics have occurred often thanks to changes in technology...But the moral of the story remains: there is value to be found in this data, value in our newfound publicness. Google's founders have urged government regulators not to require them to quickly delete searches because, in their patterns and anomalies, they have found the ability to track the outbreak of the flu before health officials could and they believe that by similarly tracking a pandemic, millions of lives could be saved. Demonizing data, big or small, is demonizing knowledge, and that is never wise.”

Sean Mead, director of analytics at Mead, Mead & Clark, Interbrand: “Large, publicly available data sets, easier tools, wider distribution of analytics skills, and early stage artificial intelligence software will lead to a burst of economic activity and increased productivity comparable to that of the Internet and PC revolutions of the mid to late 1990s. Social movements will arise to free up access to large data repositories, to restrict the development and use of AIs, and to 'liberate' AIs.”

Heywood Sloane, principal at CogniPower: “This isn't really a question about the Internet or Big Data—it's a question about who and how much people might abuse it (or anything else), intentionally or otherwise. That is a question that is always there—thus there is a need for a countervailing forces, competition, transparency, scrutiny, and/or other ways to guard against abuse. And then be prepared to misjudge sometimes.”

Perhaps David Weinberger of Harvard University’s Berkman Center put it the best perspective: “We are just beginning to understand the range of problems Big Data can solve, even though it means acknowledging that we're less unpredictable, free, madcap creatures than we'd like to think."
(Photo: Joe McKendrick)

This post was originally published on Smartplanet.com