Data explosion will cause compliance headache
Researchers are predicting a sixfold increase in the amount of digital information created over the next four years, which could have serious implications for IT departments.
According to a report released this week, the amount of data created by emails, digital cameras, keystrokes and so on is set to increase from 161 exabytes in 2006 to 988 exabytes in 2010. (An exabyte is a billion gigabytes.)
Although much of this digital information will be generated by individuals armed with consumer electronics, organisations and businesses will be responsible for the security, privacy, reliability and compliance of around 85 percent of the data, according to the report's author, analyst group IDC.
"This rapidly expanding responsibility will put pressure on existing computing operations," the report, titled The Expanding Digital Universe, claims. "IT managers will see the span of their domain considerably enlarged."
The report cites the example of Wal-Mart, which supposedly has the largest database of customer transactions in the world. In 2000 the database was reported to be 110 terabytes, but this had grown to 500 terabytes by 2004.
Specifically the report claims that the growth in the so-called "digital universe" will have implications in terms of compliance with legislation for most businesses.
Around 20 percent of the 161 exabytes of data created last year is subject to compliance rules such as Sarbanes-Oxley, Basel II and other government legislation, according to IDC. The researchers claim that companies will have to improve their IT infrastructure to make sure that their compliance strategies can cope with the rise in data over the next four years.
Developments that could affect compliance include the rise of VoIP, web conferencing being embedded into business applications, and surveillance and security images, which are expected to grow more than tenfold over the next four years.
IDC estimates that organisations employing around 1,000 staff will lose $5.7m (£3m) annually in terms of time spent re-formatting information between applications. Not being able to find information would cost a typical company of the same size another $5.3m (£2.7m) a year.
"Adopting a comprehensive and disciplined approach to managing information and understanding its value is key to reducing the hidden — and not so hidden — costs associated with the information explosion," the report said.