Open Telekom Cloud Exposure
The CPUs used in OTC are all vulnerable to all three attacks.
This is bad news for us and for our customers.
The vulnerabilities mean that:
- Runtime environments (such as browser's JavaScript interpreters or the kernel's eBPF machine) and other userspace applications with problematic code patterns are allowing unauthorized memory reads (Spectre-1). The mitigation is a one-by-one effort, where additional safeguards need to be implemented in each affected environment, possibly with some help from the system compilers.
- The hypervisor separation can be worked around by malicious VMs which can trick the CPU to slowly leak data from the host memory and thus also from other VMs. (Spectre-2)
- Userspace processes can read kernel memory, allowing to read protected contents from other processes or other containers. (Meltdown-3)
The three attack scenarios are depicted on the right side diagram.. Fortunately, not all attacks are possible in OTC.
Hypervisor
With our partner Huawei, comprehensive reviews have been conducted and we found that there are no code patterns in the KVM nor XEN hypervisors that we use that would allow attackers to do out-of-bound reads as a consequence of Spectre-1. Code scanning tools (coverity with intel configs) have been used in addition to manual code reviews to confirm this.
Both hypervisors are vulnerable against the Spectre-2 branch target injection attacks, allowing malicious guests to read host memory. Mitigating this attack surface has first priority for the Open Telekom Cloud operations team.
Neither hypervisor is vulnerable against the Meltdown-3 attack scenario, as both hypervisors use fully separated address spaces (along with EPT for performance) which prevents also speculative access of the CPUs.
Host Kernels
The host kernels suffer under all three vulnerabilities; out-of-bound reads (Spectre-1) can e.g. be triggered via eBPF programs; the kernels on our hosts though do not allow unprivileged users to submit BPF programs to the kernels.
Both Spectre-2 and Meltdown-3 attacks from userspace against the host kernel are possible until the mitigations are deployed.
All code running on host systems is fully controlled by the T-Systems Operations team; no user-controlled code ever gets to be executed there. The attack risk is thus considered very low here.
Guest Kernels
The guest kernels are likewise affected by all three vulnerabilities.
The vulnerabilities mean that process and container separation within the guest is not fully effective. This may break security assumptions from application developers and thus result in one piece of an application being able to attack other pieces or other application running in the same virtual machine.
For the ECS (Virtual Machines, Bare Metal) the kernels are under control of our customers. We strongly recommend them to update and reboot their VMs with kernels that deploy workarounds. (We have republished most images, see below.)
There are also SaaS and PaaS services such as RDS, CCE, MRS, DMS, ELB, ... where guest kernels are under the control of the T-Systems team. These will be patched by T-Systems.
Guest Userspace
The out-of-bound read (Spectre-1) is a danger to userspace applications. Typically, a potential attacker needs to have fairly good control of an application to run code to do timing analysis -- this is why interpreters, JITs and similar runtime environments are most exposed to this.
JavaScript exploits have been demonstrated by the researchers.
Risk Assessment
| Component | Spectre-1 | Spectre-2 | Meltdown-3 |
|---|---|---|---|
| OTC XEN | not affected | critical | not affected |
| OTC KVM | not affected | critical | not affected |
| OTC hosts | low | low | low |
| Guest | medium | high | high |
General recommendations
Until the patches can be deployed, there are a few things customers can do to limit the risks:
- By using Dedicated Hosts, no other OTC customer will have access to the same machine. Potential exploits against the hypervisor will thus only cause data leakage between VMs belonging to the same customer.
- Bare Metal hosts are also not shared between customers thus limiting information disclosure to processes or containers belonging to the same customer.
- Being careful running untrusted code (such as JavaScript from the web) in seemingly secure sandbox envirronment is always a good idea. Those sandboxes are less secure due to Spectre-1.
- Disabling the use of BPF for unprivileged users on Linux as described above.
- Unrelated to OTC, we also recommend users to enable the firstpary/site isolation settings in Firefox and Chrome respectively and install Operating and Application level fixes in a very timely manner. Antivirus products are not a very effective protection against these attacks -- if AntiVirus products fail to be compatible with the fixes, we would clearly prioritize the fixes and chose an AntiVirus vendor that has updated his engine or does not break or (at least temporarily) revert to MSFT Defender.