Must Read: Sorry, but bringing back the Start menu won't help Windows 8

Topic: Big Data

Microsoft to fold in-memory database technology into SQL Server Next

Summary: Hekaton, Microsoft's coming in-memory database engine, will be built into the next version of SQL Server. Meanwhile, SQL Server 2012 SP1 is now generally available.

Mary Jo Foley

By for All About Microsoft |

Just a few months after launching SQL Server 2012, Microsoft is starting to peel back the covers on what's coming in the next version of its database. And while the Softies aren't sharing a due date, they are talking about new in-memory database technology that will be built into the product.

mspasssummit

At the opening day of the SQL PASS Summit, Microsoft announced that it will be adding in-memory technology, codenamed Hekaton, into SQL Server Next. Hekaton is currently in private technology preview with a small set of customers, which company officials are planning to expand to 100 before the end of this calendar year.

In-memory databases, originally the domain of telephony, financial services, industrial control and other real-time-sensitive customers, rely on main memory, rather than disk storage, for achieving faster query results even when processing increasingly larger volumes of data. Most of the big commercial database players, including IBM, Oracle and SAP, have been talking publicly about their in-memory database products and strategies for a number of years. Microsoft, too, has been talking up its in-memory capabilities, on both the analytics-tool and column-store technology fronts.

Going forward, Microsoft is attempting to distinguish itself from other in-memory database players like SAP and Oracle by emphasizing the cost-savings and reduced management complexity it will achieve by folding in-memory technology into its core database, rather than making it a separate add-on.

"You will be able to take advantage of the hardware you already have and acclerate it by adding more memory," said Doug Leland, General Manager of SQL Server Marketing.

Leland said Microsoft is seeing up to 50 times (not 50 percent -- 50 times) performance improvements among the customers already involved in the Hekaton technical preview.

(Hekaton is a Greek term for "factor of 100." The aspirational goal of the team was to see 100 times performance acceleration levels. Hekaton also is a giant mythical creature, as well as a Dominican thrash-metal band, for what it's worth.)

Microsoft is making a couple of other announcements at SQL Pass, which is being attended by 4,000 developers and IT pros.

* Microsoft is readying the next version of its enterprise data-warehousing appliance, known as SQL Server 2012 Parallel Data Warehouse (PDW). This product, due in the first half of calendar 2013, will include a new data-processing engine called PolyBase, which is designed to enable queries across relational data and non-relational Hadoop data. Microsoft Technical Fellow David Dewitt is one of the principals behind PolyBase.

* Microsoft is releasing the final version of SQL Server 2012 Service Pack 1. The main feature in this service pack is support for the PowerPivot and Power View analytics capabilities in Office 2013. There are other new features in SQL Server 2012 SP1, as well, detailed in this MSDN post.

Microsoft is not sharing the codename for its next version of SQL Server, nor a tentative arrival date for it. Anyone got any information on either?

 

 

Topics: Big Data, Data Management, Microsoft

About

Mary Jo has covered the tech industry for more than 25 years for a variety of publications and Web sites, and is a frequent guest on radio, TV and podcasts, speaking about all things Microsoft-related. She is the author of Microsoft 2.0: How Microsoft plans to stay relevant in the post-Gates era (John Wiley & Sons, 2008).

Contact Disclosure

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

12 comments
Log in or register to join the discussion

  • Copying Memcached, And Doing It Poorly

    Yet another attempt to mimic an innovative open-source product. Want to see where the proprietary products are going? Just see where the open-source ones already are.
    ldo17
    Reply Vote

    • I'm not completely...

      I'm not completely familiar with memcached... But it sounds like pretty much every other cache which is just a key-value pair.

      "Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering." (straight from memcached.org)

      How is Microsoft copying this, exactly? Can you query memcached like a database? Because it certainly doesn't sound like it, so I don't see how they're copying it.

      Also, don't know how this is innovative either. Something doesn't always need to be innovative, and just because something is new doesn't MAKE it innovative, even if it's never been done before. Just sounds like an expected evolution.
      ian.aldrighetti
      Reply 1 Vote

    • I was doing "in memory database" some 30 years ago, before it got a catchy

      name, like it has today.

      My "in memory" database, was a separate program, which resided in the computer's ram all the time, and was invoked whenever another process/program needed to issue a query to get at a specific value or values, for a specified key. That "memory resident" database contained some 10,000 records, which sped up processing tremendously.

      So, the technology or idea, is as old as programming and database technology.
      adornoe
      Reply 2 Votes

      • How do you do join?

        These "poor man" hand made in memory arrays are NOT database. They are just good for simple lookup. When the complexity of the logic increases, let's say you need to search something that need to join 3 tables each having 10K records. Then you will write something that need to scan and compare 10^12 loops. Your computer will burn before you can find anything. Not saying when you the search is of type Outer Join like find customers who had ordered for more than $1000 last month who didn't buy Product A, B, C. Good lucks with your "hand made" memory DB. Just to prove the point, I am ready to take on any challenge against anyone doing "programmatic hand made" in memory "database" to solve a decent relational DB scenario involving xxx MB tables, join, aggregate.

        Here I believe we are talking about combining the relational engine and in-memory techniques. Not sure what is the complexity involved but I assume there are serious maths and optimization behind.
        RelaxWalk
        Reply 1 Vote

        • I never said it was relational DB lookup...

          It was a simple table, with a look-up via a key, and returning of a value or values. It was basically, the same as holding a file in memory, with the capability of accessing via key values.

          In your example of in memory look-up of 3 tables, I didn't have that, but, I had it for 2 tables, where, once a "record" was found via a key, the returned value could be used to access/call another "in memory" program that contained the other file with key/value access capabilities. Granted that, both tables/files could not be that big, otherwise, implementation of the db on hard disks would be preferable.
          adornoe
          Reply Vote

    • I use Memcached like most open source it is half baked

      It has terrible tooling, terrible management, terrible diagnostics... like most open source code it seems like someones toy they made to prove they could do it and then abandoned it onto a website for free for anyone who wants to use it instead of buying something of quality or building one themselves.
      ggibson1
      Reply 2 Votes

      • Memcached is not an in-memory DB

        Folks Memcached is not an in-memory DB, its a caching layer that can sit in-front on your DB to allow for lower latency and high scaling performance. You can basically store reference data, or very hot data that will not change very much (cash eviction) in the cache and then read from it via your Web/Services Layer to avoid round trip DB calls. Microsoft already has a similar technology to Memcached with AppFabric Cache (on-premises) and also Windows Azure Caching (cloud) (*note which also supports the Memcached Wire protocol)

        An in-memory DB in this context would be something similar to SAP Hana etc...
        conorjb
        Reply Vote

    • How come Memecached has a DB engine?

      I suppose if the authors of Memcache project knew that it could replace a DB engine, they would do it already. Here is a very simple issue, can you please show me how Memcache can solve this?

      A table has 1 billion rows. There are an unpredictable numbers of rows which have duplicated values on ColumnA. List all the offending records, sorted by the count of duplicates in descending order. Then delete the duplicates, which means in each group of duplicates, only the latest record is kept.

      This is kids play in a relational DB, especially with the T-SQL language. Even if you spend the time to write a specific program to solve this issue using Memcache. I am pretty sure it cannot do faster (if at all?) than the "classic" SQL Server, which doesn't use in-memory techniques.
      RelaxWalk
      Reply 1 Vote

    • So you have no idea what we're even talking about - got it!

      So, let me get this straight, @Ido17:

      1) Microsoft is "Doint it Poorly" in your opinion. How did you come to this conclusion, exactly? You're one of the customers currently testing out the technology? If that's the case, I hope you have your resume prepared, because you're just about to get fired for violating an NDA that you would have had to sign on behalf of your company.

      Oh.. you mean you're just guessing that Microsoft is doing it poorly.. okay that makes more sense.. In other words, you're talking out your ass.

      2) You say this is mimicing an innovative open source program. Well, okay. I guess you could say that. Except the open source program you reference is not an in-memory database and does something completely different from what an in-memory database would do. So you could say that.. but you'd be wrong.

      If you knew anything about database management and software development, you'd know that most of what Memcached does is make up for software with poorly written SQL interfaces. It is only effective if you have continuous calls to the database where you want to return the same information often. An efficiently authored SQL application will not do this.

      3) You might want to look a little deeper and maybe actually do some database development or administration on MySQL, Postgre, MS SQL, and/or Oracle before making a judgement on which technologies are leading whom. Your last sentence is pure hillarity in that regard.
      daftkey
      Reply Vote

  • Was going to build same technology

    As @adornoe mentioned In-Memory DB is old technology, as old as DB. But it is getting hot in BI world as it does aggregation 20-50 times faster than relational DB as it is column based. By the way Excel is the best example of In-memory DB. It is being built as direct competition to SAP Hana and worlds dominant BI technology Qlickvew.
    Of course good to have within SQL Server but it is bad news for those who were going to this market
    Lumian
    Reply Vote

  • Locking

    MS-SQL v Oracle and everyone else. If it wasn't for the locking model I'd be happy with MS-SQL. As for in memory, I'd have to performance test it, they've either got it right or not but impossible to guess until then
    shard_
    Reply Vote

  • code-name for "SQL Server Next"

    MJ. Any follow-up on the code-name for "SQL Server Next" (v 11.5 or v12.0)?
    mdgraves
    Reply Vote
CNET Join | Log In | Privacy | Cookies Close