RIP Hyper-Threading? ChromeOS axes key Intel CPU feature over data-leak flaws – Microsoft, Apple suggest snub In conjunction with Intel's coordinated disclosure today about a family of security vulnerabilities discovered in millions of its processors, Google has turned off Hyper-Threading in Chrome OS to fully protect its users. Meanwhile, Apple, Microsoft, IBM's Red Hat, QubesOS, and Xen advised customers that they may wish to take …
completely unrelatedly, more 'classic media'
https://www.youtube.com/watch?v=yhuMLpdnOjY (Tom Lehrer "poisoning processors in the park" )
written circa early 1950's when he was working as a mathematician with all the other mathematicians at the world's largest employer of mathematicians!
To be clear, they're not affected because the attack is architecture-specific and the researchers didn't test with AMD or ARM architecture in mind. AMD has only had full SMT with Zen, so there likely are vulnerabilities but there are only two generations (one and a half really) to draw from.
CVE-2018-12130 covers this best:
"A flaw was found in the implementation of the "fill buffer", a mechanism used by modern CPUs when a cache-miss is made on L1 CPU cache. If an attacker can generate a load operation that would create a page fault, the execution will continue speculatively with incorrect data from the fill buffer while the data is fetched from higher level caches. This response time can be measured to infer data in the fill buffer."
Do ARM and AMD CPU's continue to speculatively execute instructions following an exception generating a page fault?
As far as I am aware, the answer is no. Intel x86 is the only CPU architecture that does not halt speculative execution in the event of an page fault occurring.
AMD have a formal statement regarding this (https://www.amd.com/system/files/documents/security-whitepaper.pdf) - I'm unsure if ARM has anything as detailed as AMD's response but similar concepts apply.
The researchers did test with AMD and ARM: "they were unable to replicate any of their attack primitives"
Yet. They used tricks that worked on Intel chips, and those didn't work. This says exactly nothing about whether or not there are tricks which do work on AMD chips. Absence of evidence is not evidence of absence. It may be that they're just not looking in the right spot. It took years to spot Spectre and Meltdown with Intel chips, and more years after that to spot the current problems. Give 'em time.
Hmmm
"so there likely are vulnerabilities" says who? based on what evidence? The fact that Intel processors have been back doored 10 ways from Sunday by Israeli engineers has no bearing what so ever on AMD's Zen architecture.
At this point is has become clear that many of these so called "bugs" and "flaws" are in fact back doors put into the original design specs by Intel's Israeli engineers (aka Mossad agents)......AMD does not have any design facilities in Israel and therefore there has been far less chance that the Mossad could gain the necessary access to compromise Zen's architecture.
may reduce performance by up to 40 per cent, with the most impact on intensive computing tasks that are highly multithreaded."
Hyperthreading aids work loads with, usually, L3 cache misses, computationally intense apps are typically designed to minimise cache misses. All the tech does is to allow a hardware based context switch when execution stalls due to memory accesses.The highly multithreaded bit I get
What about using affinity masks (or similar) to allow paired hyper-threads (those on the same physical core) to be used by a process, but not allow the other half to be used by a different process?
Wouldn't that allow highly threaded applications to continue using all available threads? Other applications would still see some performance reduction.
Could be tricky to manage within the scheduler.
I was wondering about that too. Given that threads already have access to all the data in a process, restricting SMT to threads with the same process ID presumably wouldn't allow any new channels for information leakage.
I don't know exactly how Intel's hyperthreads are managed, but I assume the thread selection is handled by the OS like in other CPUs. While that would be a big performance penalty for virtualized workloads and things like web browsers, it probably would be a cheap/free mitigation for compute intensive single process workloads like rendering, simulation, etc.
We might once again live in a world where people stop bolting on multi-process to their programs when multi-thread would do.
"I was wondering about that too. Given that threads already have access to all the data in a process, restricting SMT to threads with the same process ID presumably wouldn't allow any new channels for information leakage."
The threads only have access to their own data when everything is working correctly.
The flaw with Intel's hyper-threading is that instructions continue to be processed with no further checks in the event of speculative execution causing an exception (including a memory access fault) for a number of processor cycles. This window allows access to any data cached by the CPU as long as instructions are sequenced correctly.
> The threads only have access to their own data when everything is working correctly.
Not so. The process owns the overall address space, and all threads in that process have access to all the user data within the process. This is why multi-threaded programming is hard; it's remarkably easy to accidentally stomp on another thread's data if you Do It Wrong.
Whilst there may be thread-local storage in play, it's usually very small in comparison to the memory owned by the process.
"Not so."
I accept I could have made this clearer.
The attack works by bypassing hardware/OS/process memory protections to allow access to other process memory spaces.
When a thread is being speculatively executed, introducing an instruction that causes a page fault will result in the following instructions being executed without further exception checks rather than thread execution being halted/paused pending the outcome of the exception.
This isn't a case of multi-threaded programming being hard - it's a case of hardware not enforcing protections as expected.
In terms of storage, I believe you potentially have access to other entries in the CPU caches as entries outside this will not be retrieved in the window between the page fault being raised and the exception causing speculative execution to end.
Intel is fine with its technology, and leaves the decision to disable Hyper-Threading to its industry partners.
"Intel is not recommending disabling HT," a company spokesperson told The Register in an email.
Well what a surprise that Intel have not recommended disabling HT, doing so would be commercial suicide as they have been pushing how wonderful it is since the days of the Pentium 4. Those lovely Xeon CPUs used in data centres with loads of logical processors because of hyper threading suddenly loose half of the cores available to the OS and then don't look as attractive to buyers. And selling to server makers is where Intel makes most of its money these days with PC sales being down.
I love how all the verbiage assumes that hyperthreading is just this thing that some desktop users might have enabled, so you probably want to turn it off. Not much mention of the server market, which relies on multi-core SMT processors do heavy lifting.
Disabling hyper threading in a data center environment would be catastrophic. In our environment, as an example, we have just a few hosts that aren't utilizing virtualization. Most of our workload and infra are virtualized.
I agree with this. Tell me how many "in the wild" exploits there are - particularly from the browser and I'll make the decision to run with or without HT on my desktop thank you very much.
The decision is functionally no different as to whether I choose to install a virus scanner on my Mac or not.
Servers - particularly highly virtualised ones are a different matter entirely.
You can do exactly that. Not many people have decided to prevent you from using hyperthreading. Google has, but their OS is so lightweight (just a big browser) that it is unlikely to have a need for it and BSD has always been a configure all you like OS. Other things leave it on. However, we can't give you an accurate number of ongoing exploits because first, this is new and people haven't really had a chance to try to use it yet and second, nobody tells us when they're starting to exploit this and it doesn't have a simple definable signature we can search for. You'll have to make your decision on whether to hyperthread or not to hyperthread based on the technical descriptions alone.
So you have spent years tuning your environment for hyper threading, either on Pre-Prem or in the cloud with 100s of different applications all with different performance characteristics.
All of a sudden you/cloud provider may have to disable hyper-threading. Potentially completely screwing your performance or massively increasing cost.
"So you have spent years tuning your environment for hyper threading"
I wonder how many environments "tune" for hyper threading? Aside from deciding to disable it (typically for licensing reasons), finding definitive guidelines for and against hyper-threading is usually left as an exercise for the customer and likely affects low core count CPU's more than "typical" 4+ core CPU's in a 2P server.
On the other hand, virtualization has increased average CPU utilization and has benefited from SMT. I wouldn't like to speculate what effect it has on companies that benefit from the additional processing power for "cloud"-type services. Particularly if you need to guarantee security for multi-tenant environments.
> ... disabling Intel's Hyper-Threading feature is a sure fire way to kill off the bugs.
Except it's not. It's just the best mitigation available for the presently deployed Intel hardware. Some of the problems aren't fixed by disabling Hyper-Threading, so for important application it's hardware upgrade time.
>so for important application it's hardware upgrade time.
The question is whether Intel processors without hyper-threading in their feature list are immune to the MDS exploits. Ie. do the chips listed as not supporting hyper-threading not have the relevant silicon etc. or are they simply full feature chips that have had sections of silicon disabled through firmware configuration.
Suspect best not to rush into that upgrade...
This post has been deleted by its author
The SMT silicon is still there, but a couple of fuses are blown so that the scheduler ignores them. For the most part, Intel only makes a handful of silicon designs, but will disable chunks of silicon depending on what failed in testing. The ones without SMT are just SMT chips where some of the SMT hardware didn't test clean so it gets disabled and a different model number applied. Its not an uncommon practice, silicon slingers have been doing it since integrated circuits have been a thing. Sometimes Intel will disable features on otherwise good chips if there is demand for the lower bin chips.
Really, it is down to whether the vulnerable bit of silicon is in the disabled chunk or in the still-enabled chunks (EG, in the thread's execution silicon itself or in the core's scheduler)
"it's hardware upgrade time"
It doesn't have to be, all you need to do is to make sure that your CPU is only running one thread at a time. Then, after each thread has run, remove power from the computer for five minutes to make sure all the various caches and buffers have time to reset, then you're completely safe to run an new thread, with no danger of information being intercepted. Simples!
In high security environments you may wish to physically destroy the CPU, and replace it with a new one in between each thread, but that's getting a little paranoid.
Quite. In particular, the RIDL paper can exfiltrate data you shouldn't have access to even in the absence of hyperthreading. There's a little more noise, but if you have TSX even that is driven down to irrelevantly low levels. (TSX: not so useful for real work, but oh so useful for attackers!)
HT has never performed well though somehow defended by so many people when benchmarks come out.
At absolute best a hyper core is worth 15% of a proper one before security mitigations came in. I remember Intel themselves saying HT used a few% more silicon such that it didn't sound worth it then.
Wasn't Intel HT permanently disabled on some devices anyway because of a nasty lockup bug ?
If you compare otherwise similar i5 and i7 with HT you'l find the latter typically has a larger cache and is clocked faster to make HT look good. So remove the HT crap and let's have the larger cache.
"If you compare otherwise similar i5 and i7 with HT you'l find the latter typically has a larger cache and is clocked faster to make HT look good."
Larger cache, yes; clocked noticably faster, generally no (e.g. i5-3570K vs i7-3770K).
"So remove the HT crap and let's have the larger cache."
Intel's doing that in the current generation of CPUs - i5's are 6 core, i7's are 8 core, neither have HyperThreading.
Not true, HT can achieve up to 30% performance increase over non HT, and the following article shows >20% improvement in non synthetic benchmarks (such as kernel compilation)
https://www.phoronix.com/scan.php?page=article&item=intel-ht-2018
Early HT wasn't worth much, modern HT has evolved.I will be turning it off on my system though.
It depends very heavily on what the chips are doing. For operating systems under heavy use, the extra threads can produce real benefits because a lot of threads are cycling through those times they need to compute and those times where they don't. Whenever you really need performance on a multithreaded process that doesn't use much disk, network, etc., you would be best advised not to use it because four threads running under max capacity are usually better (exceptions apply) than eight threads that keep handing control over to others. This again varies, including how much memory each thread uses and how it makes use of a cache.
"Unfortunately for Apple customers with older Macs, Intel has not made microcode fixes available for Mac models from 2010 or earlier."
Given that these problems only affect Intel processors produced from 2011 onwards, it doesn't seem particularly unfortunate that older PCs won't be getting a fix.
You're confused. All Intel architectures from Nehalem (2008) onwards are affected. Intel just aren't providing any mitigations for them. They did the same thing with Spectre and Meltdown, though a handful of Nehalem and Westmere chips got patches in that case, with the rest and anything prior being later abandoned as Intel decided it wasn't worth the trouble (and publically stated as much).
https://www.intel.com/content/dam/www/public/us/en/documents/corporate-information/SA00233-microcode-update-guidance_05132019.pdf
You can find all the affected architectures that won't receive updates there. It's a long list.
Xen, which makes a hypervisor used by AWS (advisory) and other cloud providers others, has issued an advisory that details the risks of Hyper-Threading while refusing to disable the technology by default because doing so would be too disruptive. Mitigations and fixes are available from the aforementioned link.
Or of course you could try and RTFA!
"Do AWS/Azure use HT on their servers? I'm guessing they don't."
AWS enable hyperthreading by default, but it can be disabled on everything except T2 instances.
Azure has hyperthreading enabled by default on MOST VM options and again it can be disabled. Their "protect" instances have hyperthreading disabled by default.
"There's no mention of Vmware on this"
Virtualized environments are the most vulnerable as potentially an untrusted VM already has access to run code on your CPU that can trigger these vulnerabilities to allow access to other VM's. That unpatched Win7 VM that you gave to the developers to quickly test X when their application wasn't working as expected could now be the downfall of everything else on the hypervisor...
Some CPUs can have 4 or more threads per CPU. So yeah, this can be a massive impact depending on who/what uses what. I don't know about AWS/Azure, but some of the smaller players were really concerned when the first few Spectre/Meltdown exploits came out.
Imagine your serve with 12 core/48 threads getting reduced to 12 permanently due to security requirements. 4X cost increase!
"Imagine your serve with 12 core/48 threads getting reduced to 12 permanently due to security requirements. 4X cost increase!"
I'm not aware of any Intel x86 CPU's providing more than 2 threads per core and other architectures are not affected.
HT never gave a 100% equivalent performance, typically 10% would be considered above average for most workloads and a ideal workload gets to around 30% before I/O and memory bandwidth prove to be the bottle neck. Plus you're paying for CPU time rather than just straight 100% CPU, assuming you aren't running at 100% CPU already, in which case upgrading is likely to be a no brainer.
At worst for a 48 vCPU instance, you are likely to increase costs by around 50% and get significantly more performance as a result (64 vCPU's), assuming you can use it. Depending on how many vCPU's you currently use vs the available options, your cast difference may vary.
> I'm not aware of any Intel x86 CPU's providing more than 2 threads per core and other architectures are not affected.
POWER systems have 4 threads per core, and many users of such were/are very concerned re Spectre/Meltdown.
This latest stuff doesn't seem to affect them at this stage, but who knows what the future will hold.
I will not be turning off HT in ChromeOS because mine runs on ARM. For me, that's one of its benefits right now.
I know the real world risk is minimal if you are not running a server and working in an environment where someone might want to spy on you, but at this point Intel's superior performance seems to have been dearly bought.
Hyperthreading was created after OS/2 died, and is entirely different from process threads used by all other modern operating systems. Hyperthreading is a method of creating virtual CPUs from unused resource in the scheduling queue of real CPU cores. Process threads share the same memory space, and are faster to switch between than different processes.
You can try a modern bare-metal OS/2 for yourself by buying Arca OS, and Warpstock EU is on in Amsterdam this weekend. Despite being an historic OS/2 fan, it doesn't compare to any modern OS. Arca Noae have done a decent job of making OS/2 easier to install and use on modern hardware, but it doesn't compare with a modern OS, plus the community and infrastructure that made using OS/2 fun at the time no longer exist, including an almost complete lack of legally downloadable commercial applications.
Since AMD doubled up their cores to compete with intel CPU+HT at same clock, intel's hit after patching now results in greater than 50% degrading of performance compared to AMD?
I agree that i7 owners finding they now have less than an i5 are indeed going to look to class actions to recover what they have lost.
Shame that the UK are not allowed the same recourse
The hyper-threading flaw is primarily an issue of data leaking from one thread to another thread on the same physical CPU. This is a serious issue in a large multi-tenant data center. Who is running Chrome OS on their multi-tenant data center? Oh wait, no one. I personally wouldn't worry too much about this on client systems, unless you're using shared VM hardware it's not likely to ever affect you because it's too hard to exploit without kernel access and if they have that on your single-OS system they have you anyway.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
