Native XML databases boost e-business transaction speeds

Implementing a native XML database on the middle tier can accelerate transaction processing today and leave you well prepared for Web services tomorrow.
Written by Maggie Biggs, Contributor
Extensible Markup Language (XML) is quickly becoming the de facto standard for exchanging corporate data via structured documents--whether internally, with business partners, or via public applications across the Internet. However, the data you need to exchange will likely be stored today in a relational database or some other data format rather than in an XML document. Therefore, you need to implement a process to translate data from its current format into XML and from XML back to the formats you use internally to process data.

You could use some of the tools that are available with relational databases such as Oracle's Oracle9i and IBM's DB2. These tools can translate structured and even unstructured data to and from XML. However, if you use these tools to translate data on-the-fly when an XML exchange is requested, you may actually increase the processing time of XML-based transactions as well as potentially slow other applications that might be accessing the relational database.

Alternatively, you might consider another approach--native XML databases. These databases--relatively new to the marketplace--do not replace your relational databases. Rather, they act as an intermediate cache between your Web applications and your back-end data sources (e.g., relational databases, data stored in file systems) to speed up application performance.

Beyond speed, native XML databases are useful because they provide a standard way to let you translate data from multiple back-end data sources into XML and vice versa. This is an improvement over implementing XML translation techniques for each data source you might have. Thus, all of your applications that need to exchange XML data can use one native XML database to cache XML documents. This will reduce the overall effort needed to manage XML translations.To determine if your company should implement a native XML database, consider which data sources you have that might require XML translation. If you have more than one data source that you will use for XML-based applications, a native XML database might be in order. Moreover, if you have large groups of data that require XML translation, a native XML database is also an attractive option to minimize the impact of translation processing. XML databases include application programming interfaces (APIs) that you can use to integrate XML data with application.

For example, you might need to exchange data with other companies using XML to support your billing and payment processes. This data might be stored in one database. At the same time, you might need to exchange XML data with other companies to support online ordering and product shipments. These applications might use an entirely different database or even data stored in your file system.

A native XML database--implemented on the middle tier, usually along side an application server--would allow you to process XML translations for billing, payment, ordering, and product shipment on-the-fly or in batch mode without impacting either of your back-end databases, your file system performance, or the speed of your Web applications.Native XML databases provide access to back-end data sources (e.g., relational data, file system data) via standard database access methods, such as Java Database Connectivity (JDBC) and Open Database Connectivity (ODBC). This means that almost any database--from FileMaker, MySQL, and PostgreSQL to Oracle, DB2, and Sybase--is accessible as a source or target for XML translations

Native XML databases available today are in their first few release cycles. Most of the early adopters of the technology are in the financial or manufacturing sectors and are mainly using native XML databases to speed up transaction processing. Two of the most mature native XML databases are Software AG's Tamino XML database and Ipedo's Ipedo XML Database.

The Tamino XML database offers easy accessibility to data in relational databases through its support for ODBC. It has a knowledge base where XML document type definitions--called DTDs--can be stored. Also included is an administrative interface that helps you manage collections of data. Tamino is available for the Solaris and Windows operating systems and it can work with such leading Web servers as Apache and iPlanet and with middle-tier application servers.

The Ipedo XML database is available on Solaris, Windows, and Linux. Like the Tamino XML database, Ipedo's native XML database supports the storage of XML documents and DTDs. Ipedo includes a set of Java-based APIs that you can use to link the database with Web applications.

Ipedo is also unique among native XML databases in that it uses main-memory database technology to speed processing even further. Main-memory database technology uses memory to process and store data rather than using disk-based methods that a typical relational database might. Disk-based database-processing methods increase CPU use and disk I/O operations, whereas main-memory databases perform their work in memory--a process that uses much less CPU and only occasional disk I/O cycles to speed up performance.As with any technology, native XML databases do have a few drawbacks. For starters, the technology is rather new and not yet widely proven via successful implementations. Moreover, commercial native XML databases can be costly to implement--some are priced at $40,000 and higher per CPU. However, some less-costly native XML databases are arriving. Many of the more budget-minded native XML databases were developed in the open-source community and are now available as commercial products. These lower-priced products are worth examining, as their functionality is nearly the same as their more expensive rivals (see chart).

Native XML databases are fairly easy to install. However, you'll need some technical resources on hand to implement the integration among the native XML database, your back-end data sources, and your applications. An experienced software developer can complete the integration work in short order, but he or she will need to be experienced with APIs and database connectivity. Moreover, you might need an additional technical expert who understands database administration and tuning to get the most out of your native XML database.

However, if your company plans to use XML to any great extent over the longer term, these drawbacks are not that great when compared with the benefits you'll gain by increasing processing speeds, standardizing on XML, and automating business processes in a standard manner. Native XML databases can give your company an edge right now when they're used as an intermediate cache in transaction-processing environments. This is particularly true if you exchange dissimilar data with a large number of other companies. By shifting to data exchange that uses a single format--XML--you'll reduce the amount of work needed to process data while also gaining the performance boost that comes with a native XML database.

However, native XML databases will also play a key role as you implement Web services. As you begin to consider your Web services strategy and the use of technologies such as SOAP, UDDI, and ebXML, you'll want to consider using native XML databases. XML is a key part of Web services technology and those using native XML databases will find it easier to implement Web services in the future.

Taking action
Before investing in native XML database technologies, test the XML capabilities of your existing databases. It's likely that your database will be able to translate from its relational form into XML document structures. You'll need to measure how quickly that translation is performed.

Perhaps more importantly, if you use multiple back-end data sources, such as multiple relational databases from different vendors, object-oriented databases, or data stored in file systems, you'll need to see if one of your existing databases can accept and translate data from other data sources into XML. You want to avoid having multiple databases doing XML translations, as this will only add to processing time.

Most native XML databases can be downloaded (or ordered on CD) in a trial form. You'll definitely want to evaluate several native XML databases before deciding which approach is best for your company.

Maggie Biggs has more than 15 years of business and IT experience in the financial sector. She has implemented multiple integration projects that leverage Java and XML.

Product Description Platforms

db/XML Core Native XML database; XPath support; Java and CORBA access for applications Any Java-capable platform

DataMirror's DB/XML Vision Creates XML documents containing hierarchical data from any database Any Java-capable platform

SourceForge's eXist Native XML database; data stored in XML-DB or external RDBMS; XPath support; HTTP and XML-RPC accessible Any Java-capable platform, Linux, Solaris, Windows 98/2000

GMD-IPSI XQL Java-based storage and query application for large XML documents Any Java-capable platform

XML Global Technologies' GoXML DB Native XML database; XPath support; Java accessible Linux 2.2 or later, Solaris 8, Windows 2000

Ipedo's Ipedo XML Database Native XML database with XSLT processing; XPath support; application support via SOAP, DOM; integrates with databases and file systems Linux, Solaris, Windows 2000

Data Ex Machina's Natix XML database Linux 2.2 or later

Software AG's Tamino XML Database Native XML database; integrates with relational databases; application support via C++ and Java Solaris, Windows 2000, Windows NT

X-Hive/DB Native XML data; integrates with relational and flat-file data; accessible via Java; integrates with SoftQuad's XMetaL AIX, HP-UX, Linux, Solaris, Windows 2000, Windows NT

IBM's XML Lightweight Extractor Reads relational data and assembles the retrieved data items into XML documents OS/390, OS/400, Unix, Windows

Editorial standards