Google offers first part of its in-house M:N thread code as open source to Linux kernel Google has confirmed it plans to contribute some of its in-house threading code to the Linux kernel, but hasn’t disclosed its motivations beyond a desire to share. As described by Googler Peter Oskolkov on the Linux kernel mailing list, the tech is “an M:N userspace threading subsystem backed by Google-private SwitchTo Linux …
TBH, I hope it is much much later. Schedulers are nasty things to code. All sorts of edge cases and race conditions keep popping up especially in General Purpose workloads. These are a million miles away from an Android phone especially on HPC installations. The last thing you want is a CPU hog that you can't kill.
As long as there is a way of disabling their scheduler (should it ever be included in the mainstream kernel) I won't mind. I'd treat it much like systemd. Avoid if at all possible.
Why not wait until the code becomes available before you judge it? Google is actively involved in several open source projects and has a pretty good record for the quality of the code it contributes. If anything, critcism is levelled at it for keeping code to itself.
"Schedulers are nasty things to code."
Back in the 1970s - in the lunch-time pub - the development team came up with a scheduling improvement idea. It was so simple a concept that they did a quick successful test and coded it into the master source.
For the next 18 months support were writing patches to fix the contingency cases. When it was finally stable it was indeed an improvement.
Especially for server use, but even for personal. I've never really understood complaints about high load or memory usage etc. I paid good money for a gazillion cores and umpteen gigabytes of RAM - bloody well use them. Obviously not just for the sake of it, but if this gets more performance out of your kit, why not eh?
"[...] but if this gets more performance out of your kit, why not eh? "
IIRC some multi-core processors used to be limited by overall heat considerations. All cores couldn't run at their potential top speed simultaneously - but an individual core could have a turbo boost while others were lightly loaded.
Tradition. Open almost any Operating System Design text written during the golden age, and it was all about minimally deploying resources. This is the fundamental tenet of time sharing systems. Busy-Waiting was considered an anti-pattern.
Fast forward to where there are an embarrassing richness of cores, but a dearth of operating systems designed to maximally deploy resources, and busy-waiting has been renamed lock-free programming.
Give it a few years, it will switch poles again.
It seem that M:N scheduling was in the linux kernal prior to version 2.6 - though there's no saying how the old version compares with the code Google wants to share now; https://stackoverflow.com/a/4249245/1838819
Methinks this will be a hard sell to Mr. Torvalds to get it into the kernel
There has been a general move away from user-space threading. I think they started life in Solaris. Microsoft have deprecated their "fibers", Linux added and then removed its M:N scheduling. Apparently it is one of those superficially attractive ideas whose deficiencies only become apparent after you've invested a year or two.
This is essentially the stackless versus stackful debate for co-routines, and languages are increasingly coming down on the stackless side of the fence. As far as I can tell, the main reasons are:
If you don't know which thread a routine is going to run on, you can't safely use any OS API that depends on per-thread kernel state. If you can recode so that there are no such dependencies, you've basically done all the work needed to refactor your program as a (kernel) thread-pool feeding on a queue of work packages. If you can't recode in this way, kernel threads aren't *that* much more expensive and work as intended.
Tide in, tide out.
Popular modern programming languages, most notably Go, lean heavily on M:N models.
On the other hand C++ is looking to, ahem, standardize this model, so maybe it is doomed.
Really it is all about applicability, and avoiding kernel calls. The trap latency of modern processors is obscene, even though the architecture manuals mumble about a few cycles, that carefully dismisses making the write-fifo visible, cancelling all speculative results and branch predictors, and thanks to some benchmark-embarrassed cpu-designers, a bit of cache flushing. So after the trap, the processor trundles along like something from the 80s for a bit until it can get its funk on, only to go through the same throttling to get back to useful work. To paraphrase a brilliant processor designer "I shed a tear every time I see a syscall".
So, instead, adopt a virtual cpu model, with rapid switching between local threads on internal interlocks, and fall-back to a set of parked system threads for handling the laborious system calls. If you look at the google mutex handling code(*), it is at once as brilliant as it is frightening; especially when you realize how much of it is just to avoid system calls.
(*): https://github.com/abseil/abseil-cpp/blob/master/absl/synchronization/mutex.cc
Pretty much I'd guess, although previous experience with fibers it was something that an application largely managed rather than the operating system. This reads like it is moving it towards more of a standard OS feature where the OS can manage them instead. Although as it is something that existed previously, this is an upgrade to existing functionality rather than brand new.
I guess so, but it still feels to me that if an application has knowledge that can be used to aid the kernel in scheduling threads then it might just as well bypass the kernel altogether. I know there’s shortcomings to that approach (signal delivery, major revision to source code, etc), but in the quest for ultimate speed all sacrifices must be made. This mod feels like a half way house to avoid rewriting a lot code “properly” (granted that is subjectively judged, depends on the cost benefit analysis, etc).
I wonder if Fuchsia has this in it already?
Thanks for all the phish and here's some red herrings resulting ........ live product for blind tasting/mass crash dummy testing ...... is what immediately springs eternal to the front of engaging minds here. YMMV.
Nice one, Google. A Very IMPressive Daemon which no one sees coming ...... although it does open up a whole host of new dedicated cans of worms methinks, requiring additional superior advanced remote virtual command and control for leading practical operations ....... and that is guaranteed to have both the competition and opposition from the West to the East in the likes of a Microsoft and Apple environment/semiconductor microprocessor infrastructure, green with envy and energised to fight for the right to also wield such a might. And they'll all come a'poaching and throw big money bags at both prime and sub-prime drivers/systems movers and shakers. It is just what they can only do in order to try and prevent their very own Meltdown/Systems TakeOver and MakeOver/call it what you will in AI Narrative Pump and Dump Circles around Magical Round Tables.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
