Are open source databases for real?

It isn't often that technology and television share a story. But in the case of the open source movement and reality television, the parallels are eerie: both came to public attention in the late 1990s and subverted the usual order by putting ordinary people centre stage. Reality television has grown beyond well beyond its roots to infest almost every aspect of modern life, while open source has gone from a geek's plaything to underpinning modern computing tasks like web serving, application serving and embedded computing.

These applications are the open source movement's biggest stars and hog the limelight from open source databases, which seem to be voted off the island before the computing public can appreciate the subtleties of their personality.

"We have not had a single byte of data lost through any glitches or problems with the database in seven years, across perhaps 50 major projects."

This lack of interest is about to change. Vendors are changing channels and spending more time watching this end of the open source market. And in 2004, the itchy thumbs of the technology buying public will decide whether databases will continue the open source success story, or cross the boredom threshold like the surfeit of nubile renovators, chefs and husband-hunters currently plaguing our weekday evenings.

To understand the prospects of open source databases three factors need to be assessed: the quality of open source databases themselves; the quality of support from Linux vendors for all databases; and the extent to which commercial database vendors are willing to commit to their offerings on open source.

On this last score there seems little to inhibit open source databases, as commercial database vendors lavish Linux with care and attention to ensure that their products can run on the open source operating system and deliver the kind of experience enterprise-class users demand.

Sybase, for example, has developed versions of its database and analytics products for Linux, citing the low total cost of ownership of Linux on Intel as enabling ROI it cannot ignore if it is to serve its customers. IBM does likewise, offering DB2 and accompanying development tools for Linux, while also offering Linux on mainframes to provide high scalability.

Perhaps the best example of how commercial database vendors are making Linux a viable environment for the enterprise database is Oracle, which advocates Linux as the optimal environment for its recently-released 10g database. "You want a lean, efficient operating system [for grid computing] ," says the company's Director of Business and Technology Solutions Roland Slee, as Oracle believes that both operating systems and servers are perennially fallible and that it is the database itself which must deliver the high levels of reliability, availability, security and manageability (RASM) aspired to by all enterprise vendors. Oracle therefore takes responsibility for delivering RASM of its database on Linux, even providing its own clusterware – and ignoring these features n Linux – to create its vaunted grids.

The company has also worked with Red Hat to create new features for the Linux kernel so it is better suited to grid computing and even goes so far as to provide level 3 support for Linux, offering to re-write the operating system if required to sustain customers' systems.

Oracle's efforts send mixed messages about the maturity of Linux to support large and complex enterprise databases, as by taking on many of the trickiest and most important technical and support roles for itself it raises questions about Linux's suitability as an environment for databases which do not follow a similar path and take it upon themselves to deliver RASM.

"It's likely to take two or three years before organisations are comfortable with the technology of these products."

Linux vendors, however, are working hard to imbue their distributions with the features database vendors prize. Red Hat, for example, operates a certification program to ensure that open source databases – and any other application – conform to best practices that ensure optimal performance.

The company also adds features database vendors indicate will benefit users, says the company's Australian Director of Engineering Paul Gampe. "We have configured the kernel to provide up to 64 gigabytes of main memory, which Oracle cited as important," he says. "We also have a native POSIX threading library," plus other features commercial database vendors appreciate as they allow their wares to hum along at the speeds enterprises demand.

The efforts of Red Hat and other distributions mean that Linux seems well-suited to hosting enterprise databases in less exotic configurations than Oracle's grids.

Open source databases also seem up to the task, with PostgresSQL and MySQL leading the way with impressive scalability and reliability, plus standard implementations of SQL which make skills transfer to the open source products simple, at least in theory.

Such is their performance that several Australian businesses now specialise in open source database solutions. Melbourne-based Cybersource, for example, has used open source databases since 1997. According to founder and Director Con Zymaris: "We have not had a single byte of data lost through any glitches or problems with the database in seven years, across perhaps 50 major projects."

Stephen Barker, Director of Sydney-based SquizPty Ltd, a company specialising in solutions based on open source databases which offers its own open source content management system, is similarly enthusiastic about the quality of open source databases. "We believe Postgres can compete in any reasonable sort of environment," he says, and cites a long list of happy but publicity-shy corporate and government customers to prove his assertion.

But despite the high quality of open source databases, doubts remain about their suitability in the enterprise. "There are many situations where we don't get the job because we can't provide warranty or uptime guarantees," Barker says. "Users sometimes wonder who they can sue if something goes wrong. It's easy to find and sue Oracle, which is a commercial entity. But who do they sue if MYSQL hurts their business?" Lack of comprehensive benchmarks is another oft-cited concern, while the quality of documentation for some open source databases.

Gartner Analyst Phil Sargeant hears similar concerns. "A lot of organisations are still concerned about the technical maturity of these products compared with the mainstream database products," he says. "It's likely to take 2 or 3 years before organisations are comfortable with the technology of these products."

And even when the technology of open source databases matches that of commercial databases, Sergeant sees obstacles. "We must appreciate that many organisations have already deployed proprietary database products and have invested a lot of money in products and skills," he says. "Many of these organisations will be reluctant to invest in yet another product and a new learning curve with respect to support and operation."

"Users sometimes wonder who they can sue if something goes wrong. It's easy to find and sue Oracle, which is a commercial entity. But who do they sue if MySQL hurts their business."

One thing in open source database's favour, though, is Sergeant's assertion that " … traditional database vendors have failed to lower prices materially. This has the potential to send open source database products and traditional databases on a collision course. This is likely to place buying organisations in a strong bargaining position."

Database vendors, however, seem unworried. "We believe the database will be the last piece of software left standing because people won't risk the integrity of their data," says Oracle's Slee.

Nonetheless, Gartner expects strong adoption of open source databases from around 2008, by when it is likely businesses will enjoy the choice of technologically comparable databases strongly differentiated by price, a factor which has of course fuelled open source software's spectacular rise. And with the seemingly endless success of open source software – and the never-ending stream of reality television – it seems foolish to suggest that open source databases will not become part of the computing furniture before too much more time has passed.

The open source database contenders PostgresSQL Postgres traces its roots back to 1986 and the University of California at Berkeley. The University released its code to the open source community under the BSD license in 1994, and the community responded by adding SQL support and continuing to develop the software to this day. Some of the original code and design survives, both in Postgres and in Informix's database, a product which started by using Postgres' code and is now owned by IBM. Postgres is considered the most advanced open source database but the quality of its documentation is contentious. MySQL Founded in 1995, Sweden-based MySQL AB offers its eponymous database as both an open source and a commercially-licensed product, a "dual licence" strategy it says allows it to build a firm financial footing for continued improvements to its database. With over 100 employees MySQL lays claim to being the world's largest open source database organisation, boasts over 4 million installs and is praised as the most usable of the open source databases. The company also offers MaxDB, an open source database once owned by ERP giant SAP and certified to run its R/3 suite. Firebird When Borland released the beta source code for version 6.0 of its Interbase relational database product in 2000, it created the world's newest open source database. Now in version 1.5, the database has become prized for its small footprint and reliable SQL engine. BerkeleyDB An embedded database, BerkeleyDB is used by Apache, Sendmail, the Mozilla browser and even by Google. EMC uses it in some storage devices, while Sun's LDAP server rests on its code. Cisco and Sony are also users. Claims over 200 million deployments and, like MySQL, runs a dual license strategy. Other open source databases include "hsqldb" and "Mckoi", two all-Java SQL engines. eXist is a native XML database, while Backplane offers an all open-source alternative to Oracle's grid computing solution.