Managing all the data in the world
If you think you have a lot of data flooding into your organization, imagine how much you would have to deal with if you were tracking every weather pattern and climatological event happening on the planet. Would an SOA approach be of help here?
But of course it would. eWeek's Peter Coffee published this interesting account of how an SOA-focused deployment is helping the National Aeronautics and Space Administration to streamline access to its mounds of data, as well as link users of the data to the services they need to process the data.
As is always the case with SOA, the architecture did not appear overnight at the space agency's facilities. Rather, SOA work has been underway since 2000 at NASA's Earth Science Data and Information System project (ESDIS), which deploys and manages orbiting satellites for a variety of purposes, from weather and climate monitoring to environment to homeland security.
The output from a large array of spacecraft is petabytes of data, along with the range of demands, is too much for conventional systems and databases. Robin Pfister, lead information management system engineer for the ESDIS effort at NASA's Goddard Space Flight Center, notes that "in 2000, we had one human-machine interface for data search and access. Due to different standards and search approaches used in distributed archives, result sets were difficult for an end user to evaluate." Results were not formatted consistently, and as a result, 25 percent of data was lost.
By 2002, ESDIS had launched a metadata clearinghouse and order broker called ECHO (Earth Observing System Clearinghouse). The system supports a common data model of 60 million items, and provides access to specific data sets, with a 100 percent rate of return on all relevant results to user queries.
Data was first; the application development side came next. This year, a UDDI registry was added to better enable users to identify resources for building or enhancing decision-support applications and services to help them sift through the data.
As Coffee notes in the article, ECHO allows a user to discover an interesting piece of data and ask: 'What can I do with this data?' The system then can offer pointers to appropriate services.