Industry challenge: Developing storage for the incoming data deluge
While we weren't looking, WD assembled a formidable array of storage technologies, beyond their leadership in disks and SSDs. That includes all-flash arrays, scale-out object storage, and industry-leading enclosures, with patented cooling and anti-vibration technology. Today's announcements take them to another level.
CPU performance hit a wall years ago. Multi-core chips have helped, but GPUs - with hundreds or thousands of execution units - have kept the industry pushing the performance envelope, especially for machine learning. So there's been a renewed focus on moving processing closer to data, as the growth of big data has made it costly to move the data to processors.
Fast SSDs, including the NVMe/PCIe ones pioneered by WD acquisition Fusion-io, are still improving in performance, capacity, and cost. But the cloud - and most enterprises - are still relying on hard drives for capacity, where WD's HGST unit has led the way with their helium filled drives.
At the same time, the growth in realtime systems, such as computer vision for self-driving cars, and streaming data analysis, have put a premium on managing data that is both fast and big. In what may be their most consequential announcement this morning, WD is now offering flexible and efficient drive enclosures with powerful built-in servers that support the Docker container framework. Moving apps to your data has never been easier or more cost-effective.
WD, founded in 1970, has a long and varied history. It began as a chipmaker - for a time it was the world's largest calculator chip company - but it found lasting success creating hard drive controllers, especially the ATA drives popularized by the IBM PC and its many clones. In the mid-80s they bought the assets of HDD vendor Tandon and began manufacturing disks.
Over the years they became vertically integrated, acquiring a read/write head vendor, and later a disk media manufacturer, making them vertically integrated. You may have first become aware of them with their 10,000 RPM Raptor and VelociRaptor drives, a popular choice for gamers and servers.
WD started spreading its wings when they acquired HGST, which itself was formed when Hitachi bought IBM's once leading disk business. HGST continue to be known for excellent reliability, and they've led the industry in the move to helium-filled server drives.
Since then they've acquired Amplidata, an early and under-marketed leader in scale-out object storage, and most recently, Tegile, an SSD-based array vendor.
WD is the only company that builds storage devices, both flash and disk, from the bit up, and integrates them into arrays with advanced software and hardware. Now they're putting all the pieces together, creating powerful systems based on optimizing each layer in the storage stack, from bits to arrays and interconnects, which they call that symbiotic design.
Another company has taken a similar, highly integrated, approach to building products. Apple's done pretty well with that, and WD is trying to do the same with storage.
There was too much in the briefing I attended to easily cover in one piece. I've asked WD for additional in-depth briefings on such things as their new file-on-top-of-object storage, microwave-assisted magnetic recording, and storage fabrics.
But as a long-time participant and observer of the storage industry, I believe we're seeing the emergence of an inventive and resourceful storage systems company that is bringing real creativity and value to our data-centric world.
Courteous comments welcome, as always. I've done work over the years for several of the companies that WD has acquired and for WD itself. They also paid for my airfare to attend yesterday's briefing in Silicon Valley.