Best practices for resource overprovisioning. Oxymoron or real thing?

Best practices for resource overprovisioning. Oxymoron or real thing?

Summary: For those of you who use VMware vSphere, you know that resource overprovisioning is an option. Some see it as a pariah, while others see it as a feature.

SHARE:

I've used VMware's software for more than ten years now and what we now call vSphere for five or more years. Personally, I've never thought overprovisioning was a great idea, although I do see some benefit from it.

From a purely intuitive standpoint, it's a really bad idea. It introduces risk by allocating resources you don't have to too many systems. However, from a more technical viewpoint, it has the advantage of fully utilizing resources that do exist.

The primary point of moving toward a virtualized infrastructure for many businesses is to alleviate the number of systems that are underutilized. Most servers run at 10 to 15-percent CPU utilization, 30-percent or less memory utilization and disk space, surprisingly, is the biggest waste of all resources. More surprising is that those same resources (CPU, Memory and Disk) are the most wasted in virtualized environments as well.

Part of the reason that we still have so much waste in our virtual environments is that we overbuild virtual machines (VMs). We allocate too many vCPUs, too much RAM and far too much disk space. We still tend to think in terms of physical systems with double-digit RAM allocations and multiple CPUs. But, VMs are different. They look like actual operating systems but what they really are to the host is applications. That's something to keep in mind when creating your own virtual infrastructure.

I take a fairly conservative approach to support and tend to err on the side of caution so overprovisioning, for me, has always been a bit of an arguable point. That said, there is a time and place for everything, even overprovisioning. Vkernel (Dell) has outlined some best practices for overprovisioning that are conservative and appropriate for most environments. In fact, I think they're spot on.

Perhaps their guidelines should be called rightprovisioning instead of overprovisioning.

One CPU per VM - This is one point that's hard to convince anyone of unless they're really 'in the know' about such things.

Add vCPUs as Required - Good advice. Too often, I see administrators and system creators request 4 vCPUs only to be disapointed by performance. vCPUs should be rationed as required by the applications running on the VMs.

CPU Recommendations - 1:1 to 3:1 vCPU to pCPU ratios. This is a very conservative number and will greatly limit the number of VMs per host, unless you understand what's being said. You have to remember that when you allocate a vCPU, you're actually allocating a Core, not a physical CPU. They do not map directly, so for a 16 core system, you can safely allocate 48 vCPUs. That falls at the recommended 3:1 ratio.

This number gives you some idea of how many single and multi-processor systems that you can deploy onto a given host. Watch your CPU Utilization numbers for your hosts and your VMs to see how things are working. Additionally, you should watch CPU Ready metrics to see how long your systems have to wait for host CPU resources.

Memory resources and their overprovisioning add another arguable point to the mix. Administrators often won't oversubscribe memory at all. If you do oversubscribe your memory resources, pay close attention to host memory utilization. If it remains in the "red" zone of 90% or higher sustained usage, you should consider adding more RAM, if possible. Of course, adding another host to a cluster also provides relief. 

Overprovisioning of storage derives from the ability to use thin provisioning for virtual disks. Thin provisioning means that when you add a virtual disk to a VM, you can designate it as "Allocate on demand." If you setup a 100 GB disk for a VM, it might only use 10GB initially for applications and other files but as data grows the disk can grow dynamically to a maximum of 100GB.

That sounds like an excellent way of saving space and it is. The downside is that if you've setup several VMs with thin provisioned disks and they all grow over time, and they will, then at some point, you're going to run out of space.

Overprovisioning saves space and therefore better leverages your disk resources. Expensive SAN is often wasted on "thick" provisioned disks. It is a tradeoff and you have to measure the risk in your environment and maintain a level of vigilance through disk space monitoring.

I think that overprovisioning of CPU and storage resources isn't such a bad thing. I believe though, that you have to keep an eye on what you're doing. You can't just starting deploying VMs in a random or haphazard fashion. You have to do some calculations and some careful planning to make it work. You also have to plan for rapid expansion and mitigation in case of a severe resource constraint. In other words, when things start to go bad, you'd better have more space available somewhere or another host standing by.

What do you think? Is overprovisioning a good idea, bad idea or a feature that should be used conservatively? Talk back and let me know.

Topics: Virtualization, Servers, VMware

About

Kenneth 'Ken' Hess is a full-time Windows and Linux system administrator with 20 years of experience with Mac, Linux, UNIX, and Windows systems in large multi-data center environments.

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

6 comments
Log in or register to join the discussion
  • BS Jargon

    I am heartily sick of the way the IT industry mangles words in what seems to be a geek version of double–speak.

    "Provisioning" means to provide something. "Over provisioning" should mean to provide more than is required. If you allocate more of a resource than you have, that's over allocating. So why not call it "over allocating"?

    Using existing words with their existing meaning makes things clear without the need for further explanation. Perhaps the intent of jargon is to obfuscate, reducing comprehension and therefore critical review. Or maybe it's just a penchant for jargon. But I doubt it.

    It's a lot like "utilizing", which can be replaced in 100% of cases with "using", which is clearer, has fewer characters and two less syllables.
    Fred Fredrickson
    • Over-Subscribing is the term you are looking for

      Over subscribing would be the right term where more resources are subscribed than actually available.
      Megabyte1024
      • You're right but

        Oversubscription is one term. Overprovisioning usually refers to storage but I didn't want to mix three different terms:

        Overcommitment for RAM, oversubscription for CPU and Overprovisioning for Storage so I stuck with just one that encompasses it all.
        khess
    • I didn't make up the term

      I just use the terms that people understand.
      khess
  • How do we know when to add resources?

    Nice article.. But since we talk about VMware we might need to be a bit more elloborate.
    How long can we continue to oversubscribe the resources? Is monitoring the host the only thing that is needed to ensure that the compute aspect doesn't break.
    What are the things that needs to be focused while considering Resource pools, shares and limits.
    It would be nice if you could elaborate on this.
    Megabyte1024
    • For Monitoring...

      I would monitor the VMs, the Cluster as a whole and the hosts for CPU, memory and disk. There are hundreds of different parameters to measure, several of which can be directly measured in vSphere itself.
      khess