In just the next two years, enterprise data collection is expected to increase data is projected to increase at a 42.2 percent annual growth rate, according to a new report from Seagate and IDC. However, a substantial chunk of data remains unleveraged, due to challenges in data management and security.
If enterprises don't overcome those challenges and implement solid data management techniques, they run the risk of hoarding data without extracting real value from it, the report says.
"One thing is certain: Companies want to have vibrant data lakes where fresh data is taken in and old, stale data is moved to low-cost storage domains," the report says. "No company wants their data lake to turn into a data swamp where unleveraged yet potentially useful data sits dormant on storage media."
The new report draws on a global survey sponsored by Seagate and conducted by IDC in December 2019 and January 2020. The research firm surveyed 1,500 respondents globally (375 in North America, 475 in Europe, 500 in APJ, and 150 in China) from mid-sized businesses to larger enterprises.
The growth in data collection, the survey found, can be attributed to the increasing use of analytics, the proliferation of IoT devices and cloud migration initiatives.
Survey respondents estimated their organizations collect only 56 percent of the data potentially available through their operations. Out of that 56 percent, only 57 percent of data was used by the organization -- 43 percent went largely unleveraged.
The survey also shed light on the dispersed nature of enterprise data. About 30 percent of stored data is found in internal data centers, 20 percent in third-party data centers, 19 percent in edge data center or remote locations, 22 percent in cloud repositories, and 9 percent in other locations. This distribution isn't expected to change significantly over the next two years.
Meanwhile, according to the report, managing data in multi-cloud environments is a top data management challenge expected over the next two years. A multi-cloud ecosystem can make data management difficult, given it can result in disparate management tools, separate workflows, a lack of unified security and challenges related to moving data.
"A key solution to these data storage management challenges has to do with how business owners see the stored data," the report says. "The idea is to see it—all of it—as if through a single pane of glass. It goes beyond data democratization and into storage unification. CIOs should be able to look across the multiple cloud ecosystems in a seamless manner."
Another key factor to better data management is security -- two thirds of survey respondents reported insufficient data security.
The report noted different data management solutions enterprises could employ to better leverage their data, like analytics-enabled data orchestration and well-functioning data architecture, with an emphasis on "dataOps" -- connecting data creators with data consumers.
"DataOps is the practice of bringing disparate data systems into an understandable entity," the report says. "The core functionality needed for DataOps is metadata management, data classification and policy management."