Conversation with VoltDB
A while ago, I spoke with VoltDB's Co-Founder & Chief Strategy Officer, Scott Jarr, about the role in-memory databases should play in an enterprise's IT infrastructure. In his view, these organizations are increasingly seeing the need to capture and analyze large streams of rapidly changing data. While he believes that the structured query language (SQL) that is used by traditional relational database engines can be up to the task, the traditional single system or clustered databases that manipulate on-disk data might not be fast enough.
VoltDB suggests that fast flows of data coming from people, social media, mobile devices and senors combined with traditional operational and application data have simply overwhelmed the capabilities of traditional databases. VoltDB's approach is to harness together the power of many industry standard, low cost, x86-based systems. The company claims that this combination can deal with real-time analytics, support real-time decision-making processes and yet be accessible using tools such as SQL and JavaScript Object Notation (JSON).
Here's how the company describes its architecture
VoltDB's in-memory architecture is designed for performance. VoltDB eliminates the significant overhead for multi-threading and locking responsible for the poor performance of traditional RDBMS's that rely on disks.
VoltDB is also designed to ensure that data is never lost. VoltDB's Snapshots and Command Logging features allow you to fully recover quickly and easily. Just bring your database back up and VoltDB will do all of the heavy lifting - restoring physical data from snapshots, rebuilding indexes, and replaying transaction logs. VoltDB will have you back to normal operations in no time.
Snapshots - VoltDB snapshots a consistent point-in-time view of the in-memory data and serializes it to local disk. Snapshots are written at each server and are consistent across servers.
Command Logs - to protect data between snapshots, VoltDB logs transaction invocations to disk. VoltDB refers to this as the command log. Command logs are also written at each server. To recover, the snapshot is restored and the command log is replayed. Together snapshots and command logs create durable, replicated copies of the database across all servers.
Transparent Scalability with Data Consistency - VoltDB's fundamental redesign of the RDBMS provides unparalleled performance and scalability on bare-metal, virtualized and cloud infrastructures.
VoltDB uses a shared-nothing architecture to achieve database parallelism. Data and the processing associated with it are distributed among all the CPU cores within the servers composing a single VoltDB cluster. By extending its shared-nothing foundation to the per-core level, VoltDB exploits and scales with the increasing core-per-CPU counts on modern commodity servers.
Scaling server capacity is easy and 100% transparent to your application. Simply add servers to scale throughput and storage capacity -- no need to build complex and costly sharding layers. You can build your applications with the confidence that they'll scale to meet increasing workloads.
And VoltDB is ACID compliant meaning you don't have to trade data consistency to achieve performance and scale. Transactions are guaranteed. Your data will be 100% accurate, 100% of the time.
Snapshot analysis
It appears that many suppliers of database software, including those that support traditional SQL and those that are using other query languages (No SQL), are turning to highly parallel, network-based, clusters or grids of systems to address the growing need for high performance, high reliability and still offer Atomicity Consistency Isolation Durability (ACID) transactions.
VoltDB is among a number of companies to offer this type of technology. Although each of the suppliers is targeting different use cases, the stories they tell sound quite a bit a like. The suppliers of traditional RDBMS systems haven't been sitting still. Most of them offer in-memory capabilities or add on products to support similar requirements for performance. Suppliers such as VoltDB, would point out that the software license costs and the costs of hardware needed for those products make them uncompetitive.
Do you need one of these databases? VoltDB believes that you do. It appears that the traditional suppliers are moving in that direction as well.