Video: Intel addresses Meltdown and Spectre security flaws at CES 2018
The Meltdown and Spectre processor bugs are worrying for desktop users -- and having a computer lock-up because of a badly written Intel or AMD CPU patch is truly annoying. But the bottom line is: PCs, whether they're running Linux, macOS, or Windows, won't see much of a performance hit. The real pain from Meltdown and Spectre will be felt on the cloud with the server, not on the PC.
That's because Meltdown and Spectre can break through the memory walls between applications and your operating system's dedicated memory. On a PC, this means trolling for your passwords and the like. On a cloud, the crown-jewels of your company may be one breach away from being stolen.
SANS security expert Jake William warned, "Meltdown may target kernel addresses that are shared between the container and host kernel in many paravirtualization instances (e.g. Xen) and kernel sandboxes (e.g. Docker)."
Hyper-V, Microsoft's hypervisor, doesn't use paravirtulation, but it's still vulnerable. Terry Myserson, Microsoft's executive VP of Windows and Devices Group, explained in a blog, "In an environment where multiple servers are sharing capabilities (such as exists in some cloud services configurations), these vulnerabilities could mean it is possible for someone to access information in one virtual machine from another."
Microsoft was made aware of these problems early on, and the company has installed Azure and Hyper-V patches to block them. But, Myerson warned, that's not enough. "Windows Server customers, running either on-premises or in the cloud, also need to evaluate whether to apply additional security mitigations within each of their Windows Server VM guest or physical instances."
Why? Because, "these mitigations are needed when you are running untrusted code within your Windows Server instances (for example, you allow one of your customers to upload a binary or code snippet that you then run within your Windows Server instance) and you want to isolate the application binary or code to ensure it can't access memory within the Windows Server instance that it should not have access to. You do not need to apply these mitigations to isolate your Windows Server VMs from other VMs on a virtualized server, as they are instead only needed to isolate untrusted code running within a specific Windows Server instance," Myerson said.
To start protecting your servers -- whether they're running on bare-iron in your server closer or on a cloud -- you must patch your servers for three vulnerabilities: CVE-2017-5715 (branch target injection), CVE-2017-5753 (bounds check bypass), and CVE-2017-5754 (rogue data cache load).
These patches are not available for all Windows Server versions. All the long, out-of-date Server 2003 versions and 2008 and 2012 are open to attack. Microsoft is working on patches for 2008 and 2012. If you've been dragging your feet about updating 2003, stop. It's well past time -- not just for these security holes, but for all the others that have opened in recent years.
Patching isn't enough. You'll need to do more. Just as on desktop Windows, you must be certain to use a compatible anti-virus program for the patches to avoid BSODing your server. If you don't run anti-virus software on your server, you must use regedit to set the following registry key:
Key="HKEY_LOCAL_MACHINE" Subkey="SOFTWARE\Microsoft\Windows\CurrentVersion\QualityCompat" Value="cadca5fe-87d3-4b96-b7fb-a231484277cc" Type="REG_DWORD" Data="0x00000000"
Anti-virus or not, you must also make other registry changes. This is especially true if your server are Hyper-V hosts or Remote Desktop Services Hosts (RDSH), or your server instances are running containers or untrusted database extensions, untrusted web content, or workloads that run code from external sources. In short, many, if not most, of your servers.
These additions to the registry are:
reg add "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management" /v FeatureSettingsOverride /t REG_DWORD /d 0 /f
reg add "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management" /v FeatureSettingsOverrideMask /t REG_DWORD /d 3 /f
reg add "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Virtualization" /v MinVmVersionForCpuBasedMitigations /t REG_SZ /d "1.0" /f
You're not done yet. Now, you must apply the chip firmware to your servers' hardware. This firmware should be provided from your hardware vendor.
Once all this is done, you'll need to reboot your servers.
On Azure, Microsoft automatically reboots your servers and VMs as the patches are rolled out. You can see the status of your VMs and if the reboot completed within the Azure Service Health Planned Maintenance Section in your Azure Portal.
But while Microsoft takes care of this at the Hyper-V level -- and says you don't need to update your VM images -- it also warns you should continue to apply security best practices for your Linux and Windows VM images. Let met cut to the chase: Update your images. If these security problems can break out of VMs, all bets are off on what may be attackable and you want your server instances to be as safe as possible by patching them.
Microsoft states, "The majority of Azure customers should not see a noticeable performance impact with this update. We've worked to optimize the CPU and disk I/O path and are not seeing noticeable performance impact after the fix has been applied. A small set of customers may experience some networking performance impact. This can be addressed by turning on Azure Accelerated Networking (Windows, Linux), which is a free capability available to all Azure customers."
Accelerated Networking is a new feature that's just become generally available. It bypasses Azure's host and virtual switch to speed up VM network traffic. It works by reducing the load on the VMs and moving it to Azure's in-house programmable SmartNICs. To use it, you must start a new VM and attach a new network interface card to it when it's created. To manage it, you must also use the newer Azure Resource Manager management portal.
Even with Accelerated Networking, I think that's optimistic of them. We know for a fact patched Linux systems will see slowdowns with some workloads regardless of what cloud they're running on. There's no reason to think Windows Server won't face similar performance issues.
In addition, there have been some reports of Azure VMs failing after the patches.
Therefore, after patching, start testing your servers to make sure they work the way you expect them to, and then start performance testing. The sooner you know what you're dealing with, the sooner you can fix problems and start tuning your cloud and server resources to deal with under-performing services.
Brace yourself sysadmins, you're going to have a lot of work on your hands.