Security Advisory about Processor Speculation Leaks (Meltdown & Spectre)
Reference: i0.kym-cdn.com
Info: Open Telekom Cloud Security Advisory about Processor Speculation Leaks (Meltdown/Spectre)
Version: 2.1
Date: 2018-08-17
About This Article
On 2018-01-04, security researchers published findings that uncovered how contemporary Processors (CPUs) leak protected data by a combination of their caches and speculative execution. [Google Project Zero]
These findings have profound consequences for security considerations in IT and unlike simple software bugs, dealing with them will keep researchers, the IT industry and users busy for a while.
In this document we attempt to quickly outline the vulnerabilities that have been discovered and share our understanding and assessment of the situation.
We will go into some detail how the consequences affect our customers on the Open Telekom Cloud (OTC) and what mitigation we plan to apply and what advice we can give to our customers.
This document will get updated as new information emerges and will also reflect the updated status as we roll out workarounds for the CPU bugs.
Executive Summary
Bugs in the implementation of many modern out-of-order CPU implementations lead to a side-channel that allows attackers to get read access to normally protected memory regions on a computer. This allows to bypass system security policies and to overcome security checks, process, container, and even virtualization boundaries. Passwords, cryptographic keys and other confidential information are possibly at risk.
The intel CPUs used in Open Telekom Cloud are affected.
We have started deploying microcode fixes, OS kernel and hypervisor workarounds in OTC to protect the security of our customers and our infrastructure. This process will contniue over a few days. Deployment does result in reboots of our infrastructure which will in turn result in customer VMs to be rebooted. We try to warn customers before the reboot of the hosts hits their individual VMs.
Beyond adding the workarounds to our infrastructure, we also need customers to deploy updates to their operating system kernels.
We have started publishing new public images that contain the microcode patches from intel and the kernel workarounds and ask our customers to start using the updated images (or install the online updates for kernel and microcode and reboot). More image updates will happen as workarounds are published by the vendors.
With the deployed workarounds in our infrastructure, we will have reestablished the safe isolation of containers and virtual machines in OTC again (at least for all known attack vectors) and thus resolved the worst problems.
The (somewhat less severe) userspace problem (Spectre-1) remains though and will need workarounds in all affected applications, most urgently interpreter/JIT runtimes. These will come over time (we expect months!) from the application and operating system developers and we will install the updates where we make use of affected software and otherwise provide the updates to our customers via online updates and refreshed images.
(Please use the links in the following sections to read about details.)
The Bug in Detail
In this section we explain processor design flaws in detail.
The Three Attack Scenarios
Researchers have given the names Spectre (two attack scenarios, GPZ variants 1 and 2) and Meltdown (GPZ variant 3) to the three attack scenarios that they explored and described.
OTC Exposure
The processors in OTC are exposed to the flaw. Read our assessment ...
OTC Patching
The Open Telekom Cloud team is deploying a number of improvements into the infrastructure to work around the processor issues and to ensure that no unauthorized reads can happen any more. This includes patches to our infrastructure as well as recommendations for updates to our customer. Read how we patch and on the schedule of it in this section.
OTC Patch Status February
After the January patching was done quickly and successfully, it was incomplete, mostly due to the microcode updates that were not and still are not availble in stable form on all used CPUs. See our February status page.
OTC Patch Status March
There are some news in March, most notably the release of stable microcode by intel for most of their CPUs from the last few years. See our March status page.
OTC Patch Status April and May
We continue securing our platform with further hotpatches for Xen hypervisors. Retpoline updates for KVM hosts are being prepared. See our April and May status page.
Benchmarks
We have done some microbenchmarking to understand the worst case performance impact of the KPTI, IBRS and retpoline mitigations. In particular, we compare IBRS vs. Retpolines mtitigation against Spectre-2 (BTI).
Outlook
With updated images, intel microcode updates, the KPTI kernel workarounds in our infrastructure, and workarounds in our KVM and XEN hypervisors, we will have addressed Spectre-2 and Meltdown-3, thus resolving all known scenarios that threaten process, container and virtualization isolation. But more remains to be done as described in the Outlook section.
Spectre-NG
The german computer magazine publisher heise has published an article about 8 new attack vectors against CPUs with out-of-order execution that were named Spectre-NG (next generation). As of now (May 4), we have reason to believe that these issues are real and severe, though not a lot of details are known. Read our initial thoughts here.
Variants 3a and 4
Meanwhile, variants 3a and 4 have become public and we have a page on them.
L1 Terminal Fault/Foreshadow
On Aug 15, a Spectre-NG class vulnerability called Foreshadow/Foreshadow-NG by the researchers and L1 Terminal Fault by intel has been published. Read our L1TF analysis here.
References and Credits
The issues have been disclosed and researched by Google Project Zero with some overlapping research from Graz University of Technology with two research articles on Meltdown and Spectre.
There is good coverage (as usual) on the issue and the workarounds in the Linux kernel by LWN (subscription necessary) and some collection of LWN pointers.
Vendor statements are available from AMD and ARM at this point. Whhile the reaction from intel was vague intially, they have now published an intel whitepaper with a lot more useful information and finally (as of Jan 10), also some intel performance results. A security advisory from Huawei for their Hardware and for EulerOS is also available and there's also a statement from the Xen project.
Advisories are available from Operating system vendors RedHat, SUSE, Ubuntu, Microsoft and others.
There is a check script that can be used to check whether fixes are in the used kernel. (It currently does not detect the mitigation that SUSE has done.)
Finally, to not lose your good humor: There is also xkcd coverage.
Press coverage on this topic is huge meanwhile, of course. Just one pointer to the ix article (in german) that I authored.
Disclaimer: The author has been careful to avoid providing wrong information and has used his knowledge on processors and operating systems and trustable sources to make judgements. However, there is always a risk that information is wrong and advice is misguided. Neither the author nor his employer can take any responsibility for bad advice or any damage that might occur as conseqence.
Trademarks: Many trademarks were mentioned in here and they are and remain of course properties of their respective owners.