Baidu swaps DRAM for Optane to power in-memory database

Off the back of a three-year collaboration agreement signed earlier this month between Intel and Baidu, the pair have announced that Baidu is adding Optane DC persistent memory to its Feed Stream service.

Baidu's aggregation and recommendation engine, Feed Stream, relies on an in-memory database called Feed-Cube, which will run on systems using both traditional DRAM and Intel's Optane memory, before being migrated to systems solely using Optane.

Driving the Chinese giant to look at DRAM replacements was overall cost, and after testing NVMe SSDs, which were unsurprisingly slower than DRAM, Baidu settled on using Optane to store Feed-Cube's data files while DRAM retained the hash tables.

In a test of 200,000 queries per second fetching 100 key-value pairs, the DRAM+ Optane system was 24% slower on access times and used 7% more CPU than the DRAM-only system.

"At the same time, single-server DRAM usage drops by more than half, which will undoubtedly reduce costs in terms of the petabyte-level storage capacity of Feed-Cube," the pair said.

Moving to a full Optane system required Baidu to create a memory allocation library for Optane, to replace the standard malloc and free calls programmers are used to.

Testing the three configurations against 500,000 queries per second found pure Optane system was just under 10% slower than pure DRAM, with the hybrid system sitting between them.

Baidu said it was looking at using Optane elsewhere in its business, such as in cases of power failure where services need to be reloaded from drives and can take up to 10 minutes to be restored.

"With the high-speed read-and-write performance and non-volatility of Intel Optane DC persistent memory, the loading time is now reduced to just seconds," it said.

The company is also looking to add Optane into its systems running Apache Spark.

In April, Intel announced a new member of its Optane series, the H10, which adds some of the company's 3D NAND storage beside its Optane 3D XPoint memory on a single M.2 drive.

Intel unveiled its Optane DC persistent memory in May 2018.

For its second quarter results posted earlier this month, Baidu recorded steady revenue of 26.3 billion yuan ($3.84 billion) for the three months to June 2019, while net income dropped 64% to 2.41 billion yuan.

Related Coverage

Intel puts Optane memory and NAND storage together on H10 SSD

Due to appear in May, the 3D XPoint and storage combo is aimed at the thin laptop market.

First Optane Performance tests show benefits and limits of Intel's NVDIMMs

Non-volatile Solid State Drives (SSDs) revolutionized data storage. Will Intel's non-volatile DIMMs do the same to servers? Researchers at UC San Diego just released their test results. Here's what you need to know.

Intel, Baidu sign 3-year deal to collaborate in cloud, AI, 5G

The agreement revolves around Intel optimizing its platforms and products for Baidu's core business areas.

How Intel Optane DC Persistent Memory could up capacity, lower cost of in-memory databases (TechRepublic)

The new memory solution leverages 3D XPoint and offers DIMMs in sizes up to 512GB.

Baidu receives Q2 revenue jolt from AI and streaming services

The company's artificial intelligence segment also experienced increased uptake, with its AI Open Platform growing 37% year on year to having 1.3 million developer accounts.