Will Microsoft tackle Oracle?

In a great paper at the latest 5th Biennial Conference on Innovative Data Systems Research (CIDR '11) Microsoft researchers present an innovative scale-out database architecture. Will this be the product that brings Oracle back to earth?

The trick It's an old strategy: bleed the profit out of your competitor's cash cow. Microsoft's been trying with SQL Server with some success, but it can't handle the big enterprise jobs that Oracle RAC and, to a lesser extent, IBM DB2, own.

Partly, that's due to SQL Server's Windows-only status. But it's also due to the limited scalability of SQL Server. Failover clusters are fine as far as they go, but that isn't very far.

With the Sun acquisition, Oracle is moving into the OS market. It's time for Microsoft to return the favor.

Stalking horse? Hyder is a transactional record management system that runs on top of an RDMS, like SQL Server. And there's no complex - and costly to manage - lock management and load balancing of today's clustered DBMSs. Hyder uses a different, simpler process that relies on high speed flash storage.

It offers good linear scaling. Need more throughput? Add another server.

Coordination is handled across the datacenter network by logging update intentions to a central, flash-based log. Each server handles its own updates and checks the log to ensure there are no conflicts.

No lock manager. Load balancing is as simple as directing the next transaction to the least-busy server.

(For a deeper intro to Hyder architecture, check out Hyder: a flash-based scale-out database).

The play Enterprises run Unix/Linux servers because they're robust and flexible - especially for big database apps. What if Microsoft could offer a highly scalable database product that ran on commodity boxes from HP and Dell?

One less reason to run Linux. One more reason to buy Windows.

But what about that log server? It could be a big server or 2 loaded with flash DIMMS. While there is room for custom performance enhancements, Hyder could run on commodity servers.

The Storage Bits take Competition is a good thing - and Oracle could use some. Microsoft has a healthy datacenter footprint and if Hyder were productized it would leverage and expand Microsoft's presence.

Dave Cutler, the brilliant DEC engineer who joined Microsoft and led the Windows NT effort, doesn't like lock managers - he'd seen the problems with DEC's VAXclusters - and prefers shared-nothing architectures.

Hyder's architecture goes a long way towards meeting those objections. And it does it on standard Windows.

There's a larger dynamic. As computer performance improvement slows - see Moore's Wall - every vendor needs new avenues for growth. Which is why Cisco is taking on HP and Dell in servers, Oracle bought Sun, and EMC snapped up Data Domain and Isilon after cornering the virtual server market with VMware.

Microsoft should pile on, giving Oracle/Sun a good whack, while strengthening its datacenter position, especially in the nascent private cloud market.

That said, Hyder is a research project, not a product. Database users are conservative, so Microsoft would need to make a long-term commitment.

But this fundamental rethinking of scale-out database architecture is the sort of innovation that Microsoft should embrace - and does, if you believe their PR. I hope they do, but I'm not holding my breath.

Comments welcome, of course. Here's a link to the Best Paper award winner paper: Hyder – A Transactional Record Manager for Shared Flash by Philip A. Bernstein & Colin W. Reid of Microsoft and Sudipto Das of UC Santa Barbara.