OpenDrives: Building an 8K video editing rig
special feature
Ultra HD 4K video is bogging down systems everywhere. So, I was surprised to see native 8K video -- files are four-times larger than 4K -- being natively edited.
It is no newcomer, having been in shipping for seven years. OpenDrives was founded to provide high performance/cost effective NAS storage to bleeding edge media and entertainment (M&E) companies, which is where most of the team earned its stripes. But its performance and cost is winning it converts in other fields, such gene sequencing, where mountains of data are generated daily.
Performance
Its storage systems delivers up to 18GB/sec of bandwidth, enough to support multiple uncompressed 4K streams, or, as I saw, uncompressed 8L video. And it does this at a price that, it says, is up to one-third that of the industry heavyweights.
Hardware
As the name suggests, it uses commodity hardware as much as possible, including Intel Xeon CPUs, HGST NVMe drives, Arista and Mellanox networking, and HP servers. With considerable tuning, it is able to fully saturate its hardware internally.
In fact, it finds that optimizing the network is key to getting its performance to the workstation user, so it is working with Intel to integrate 100Gb/sec into its next-generation products. Today, it supports up to four 40Gb/sec links for its external network.
Software
Anyone can buy high-end hardware. OpenDrives' secret is in its software, starting with its use of the modern OpenZFS file system, which it has forked and tuned for performance. It also has a patented cache management system with aggressive pre-fetch to eliminate inter frame delays.
It uses standard network protocols -- SMB, NFS, NDMP, and others -- and standard file types, so there's no data locking.
In addition to high performance, it also offers snapshots, replication, deduplication, mirroring, optional data in-line data compression, even variable block sizes. All included in the product's price.
The Storage Bits take
Given the single-minded performance focus of OpenDrives, it's clear that 8K video editing will remain the domain of high-end systems for some years to come. But that's OK. Even a big budget feature like Gone Girl was shot in 6K, not 8K.
But this points up a larger sea change in storage, from the I/O-focused enterprise systems of 20 years ago, to the bandwidth and performance focused systems of today, driven by the incredible development of digital video over the last decade, and now, increasingly, by big data. Data storage is becoming ever more central to modern information infrastructures.
Courteous comments welcome, of course.
Previous and related coverage
Can tech make your videos more interesting?
Video cameras are so good and storage so cheap that we can make lengthy videos of our everyday activities that even the makers don't want to watch. Techies are hard at work to enable automated editing that emphasizes the interesting bits. Here's a report from the front lines.
Four methods hackers use to steal data from air-gapped computers
Air-gapped computers are seen as high-value targets, so considerable research has gone into taking data from them -- without a network connection. Here's what you need to know.