Hi Joe,
Great post.
Simple, traditional data federation is not data virtualization.
Data virtualization is about hiding and handling complexity. Enterprise data is complex. It requires rich transformations not limited to SQL or XQuery. Data federation assumes data is ready for consumption ? and of good quality, which is not reality. You can?t propagate bad data in real-time and waste time post-processing. Data transformations must happen on the fly, on federated data.
http://www.informatica.com/Pages/data_virtualization_index.aspx
http://www.informatica.com/products_services/data_services/Pages/index.aspx
Ash Parikh
Enterprise architects are getting busier by the month. Not only do they need plan out service-oriented architecture and cloud services at the application level, but need to start considering how data can fit into the agile, flexible organization — especially since many are starting to become overwhelmed by multi-terabytes and even petabytes’ worth of data.
Data virtualization, now enabled by today’s generation of solutions, is seen by many as the latest great enabler for getting information out across the enterprise. Once data virtualization takes hold, ‘Information as a Service‘ — in which data and analysis is available, on demand, to anyone who needs it — becomes more of a reality.
But does anyone really “get” data virtualization and IaaS? Yes, says Forrester Research’s Brian Hopkins, who recently published a report with a set of case studies on everything you need to know about data virtualization. (Downloadable here from the Composite Software site.) (Thanks to Loraine Lawson for the pointer to this report, and great analyses on what’s been happening in this space.)
Data virtualization offers a means to get around the relatively clunky ETL (extract, transform, load) paradigm that has dominated integration projects for the past decade or so, Hopkins says.
However, it’s going to take time to get there. Currently, Hopkins reports, fewer than 20% of IT organizations are looking at data virtualization technology, due to the perceived costs of such solutions, or confusion in the market. However, things are about to change, Hopkins writes:
“Over the next 18 to 36 months, we expect this market attitude to change as technology advancement, more third-party integration, and new usage patterns lead to increasing awareness of data virtualization’s potential. Already, many early adopters are having signifcant success with recent releases of the market-leading products. For example, one interviewee stated: ‘When we realized that we didn’t have to physically move data around for integration, the technology started to really make sense. Now we have gone from point solutions to an enterprise deployment [of data virtualization].’”
What’s happening in the technology space that is making data virtualization and IaaS more of a reality? Hopkins cited six trends:
- Improved query performance.
- Distributed caching enabling enterprise-scale operations.
- Improved discovery tools that make virtual data stores easier to create.
- Data masking that adds element-level protection to virtual data sources.
- More out-of-the-box third-party integrations that create a true enterprise platform.
- Integration of big data that is expanding the potential for business insight.
