​Ceph open-source storage takes an organizational step forward

Under the guidance of The Linux Foundation, the Ceph open-source project will now be directed by the Ceph Foundation.
Written by Steven Vaughan-Nichols, Senior Contributing Editor

In Berlin, The Linux Foundation, announced over 30 global technology leaders have formed a new foundation to support the Ceph open-source, unified distributed storage system project community: The Ceph Foundation.

Ceph is a unified distributed storage system. It provides applications with object, block, and file system interfaces. Ceph is also a software-defined storage (SDS) program. It's designed to run on commercial off-the-shelf (COTS) hardware.

Behind the scenes, Ceph uses an Object Store to store data. Within it, each piece of data is stored in a flat, non-hierarchical namespace and identified by an arbitrary, unique identifier. Each item's metadata are stored along with the data itself.

Also: Linux adds a code of conduct for programmers

Ceph stores your data on a Ceph Block Device (CBD). This is a virtual drive that can be attached to bare-metal or virtual machine (VM) Linux-based servers. To manage its storage, Ceph uses the Ceph Reliable Autonomic Distributed Object Store (RADOS), which facilitates block storage capabilities such as snapshots and replication.

Within RADOS, an object is the unit of storage. In turn, objects are stored in object pools. Each pool has a name (e.g., "foo") and forms a distinct object namespace. Each pool also defines how the object is stored, designates a replication level (2x, 3x, etc.), and delineates a mapping rule, describing how replicas should be distributed across the storage cluster. (For instance, each replica should live in a separate rack.)

A Ceph storage cluster is comprised of object storage daemons/devices (OSDs). This cluster can store multiple pools and makes Ceph extremely scalable. You can start with little more storage than you have on your desktop and surge up to petabytes.

Also: Best Black Friday 2018 deals: Business Bargain Hunter's top picks

While these things are all neat, what makes most people -- and anyone who cares about the bottom-line -- excited about Ceph is that it makes managing storage affordable. From the user's viewpoint, you're not concerned about the hardware, whether it's old style rusted iron hard-drives or RAID SSDs. You can access the data in whichever way makes the most sense for you.

Block and file storage are critical to any company. It's an especially important components of infrastructure platforms like OpenStack and Kubernetes. According to recent user surveys, roughly two-thirds of OpenStack clouds use Ceph block storage.

Efficient, agile, and massively scalable, Ceph significantly lowers the cost of storing enterprise information in the private cloud and provides high availability to any object, file, and block data. Unstructured data makes up 80 percent and more of enterprise data. This data type is growing at the rate of 55 percent to 65 percent per year, and is common with rich-media, predictive analytics, sensors, social networks, and satellite imagery.

The growth of new cloud, container and artificial intelligence/machine learning applications are driving Ceph's growth. For example, Ceph combined with analytics and machine learning enables enterprises to comb through mass amounts of unstructured data to quickly spot patterns with customer behavior, online customer conversations, and potential noncompliance scenarios.

Thus, "Ceph has a long track record of success when it comes to helping organizations with effectively managing high growth and expanding data storage demands," said Jim Zemlin, The Linux Foundation's executive director in a statement. "Under the Linux Foundation, the Ceph Foundation will be able to harness investments from a much broader group to help support the infrastructure needed to continue the success and stability of the Ceph ecosystem."

CNET: Best Black Friday deals 2018 | Best Holiday gifts 2018 | Best TVs to give for the holidays

Ceph is already used by cloud providers and enterprises around the world, including: Financial institutions, Bloomberg and Fidelity; cloud providers, Rackspace and Linode; academic and government institutions, Massachusetts Open Cloud; telecommunications infrastructure providers, Deutsche Telekom; auto manufacturers, BMW; software-as-a-service providers, SAP and Salesforce; and many others.

Ceph is also used by Rook, a Cloud Native Computing Foundation project that brings seamless provisioning of file, block, and object storage services into the Kubernetes environment, running the Ceph storage infrastructure in containers alongside applications that are consuming that storage.

The Ceph Foundation's founding premier members of include Amihan, Canonical, China Mobile, DigitalOcean, Intel, OVH, ProphetStor Data Services, Red Hat, SoftIron, SUSE, Western Digital, XSKY Data Technology, and ZTE. The Ceph Foundation will organize and distribute financial contributions in a coordinated, vendor-neutral fashion for immediate community benefit. The goal is to galvanize Ceph adoption.

TechRepublic: A guide to tech and non-tech holiday gifts to buy online | Photos: Cool gifts for bosses to buy for employees | The do's and don'ts of giving holiday gifts to your coworkers

"A guiding vision for Ceph is to be the state of the art for reliable, scale-out storage, and to do so with 100 percent open source," said Sage Weil, Ceph co-creator and Red Hat Ceph project leader and chief architect in a statement. "While early public cloud providers popularized self-service storage infrastructure, Ceph brings the same set of capabilities to service providers, enterprises, and individuals alike, with the power of a robust development and user community to drive future innovation in the storage space. Today's launch of the Ceph Foundation is a testament to the strength of a diverse open-source community coming together to address the explosive growth in data storage and services."

The Ceph Foundation is the successor framework to the Ceph Advisory Board. Want to join up? You can get involved in the Ceph community -- or learn more about Foundation membership, and then decide if Ceph is for you.

Linux survival guide: These 21 applications let you move easily between Linux and Windows

Previous and related coverage:

Linux on Windows 10: Running Ubuntu VMs just got a lot easier, says Microsoft

Ubuntu VMs can now be launched from Hyper-V Quick Create and use RDP for enhanced session mode.

Linux distribution comparison chart

If you're new to the world of Linux and trying to figure out which distribution is right for your needs, this chart can help.

Related stories:

Editorial standards