A Linux alternative? Debian/Hurd shows microkernel Unix dream is alive

Before Linux, GNU was working on its own Mach-based Unix compatible OS. Now, in the footsteps of Debian 13, there is a new release. Debian GNU/Hurd 2025 is the latest release of the other GNU operating system. The announcement email from developer Samuel Thibault says this release includes a working x86-64 edition, thanks to …

[m4r35n357]

In the meantime . . .

The Minix site is sliding out of existence. https://blog.minix3.org/tag/news/

Also, see https://genode.org/

[picturethis]

Re: In the meantime . . .

Well, at least the copyright is up-to-date...

(probably automated)

[m4r35n357]

Re: In the meantime . . .

Tanenbaum know what he was doing, and I have nothing against BSD licensing (I have used it myself), but it looks like Minix has been permanently slurped by Intel (amongst others, I suppose). Of course they still contribute to Linux because they have to.

[Phil O'Sophical]

Re: In the meantime . . .

Well, at least the copyright is up-to-date...

(probably automated)

That's risky. Copyright protection lasts for a certain time into the future, so claiming a 2025 copyright for something written in, say, 2020 means that you're claiming 5 years more protection than you're entitled to. When updating source code for products our corporate legal team had very tight rules about when the (C) notice at the start of the files had to be changed, and when it must not be. Automated updates didn't extend copyright.

[Anonymous Coward]

Performance hit

I simply can't understand why the world won't accept a small performance hit (5 to 10%^) in return for a much more resilient, clearer and easier-to-understand and secure operating system? I mean, not everyone's a hard-core gamer are they?

There are loads of systems where security matters more than performance, like in finance, healthcare, financial, aviation and critical infrastructure embedded control. Most likely many of these are already quietly and securely humming along with a microkernel OS like QNX.

Microkernels need much more accolades for the security and reliability they bring to the world. But the sad fact is that these are simply overlooked because they "just work" and people therefore tend to forget about them. This article didn't give a shout out to the myriad of software running on microkernels either.

[Zibob]

Re: Performance hit

This is the same world and people running it that built known vulnerabilities into CPUs just to be a little faster.

Sacrifice speed now for potential what ifs in the future? Never!

[Anrtryg]

Re: Performance hit

"There are loads of systems where security matters more than performance, like in finance ... "

Yes. That's why finance (no, I don't mean "fintech") run z/OS.

[abend0c4]

Re: Performance hit

I'm not sure the performance hit is inevitable, but processors get designed for the software that exists and vice versa. Most of the 'more secure' features, such as multiple protection rings, have been abandoned because performance suffered quite markedly.

A processor that didn't use paging beneath the user level but used something like segmentation to isolate modules and whose kernel was entirely memory resident (feasible these days) could potentially run a microkernel very efficiently. Noone is going to make one in the absence of software - and, again, vice versa.

But performance is a bit of a red herring. In its earlier days Linux was not very good at asynchronous I/O using large numbers of file descriptors or at efficient high-speed networking. It wasn't a significant brake on its uptake and solutions were eventually found. The same would be true of microkernel-based systems.

If there were a significant porting issue for user software, it would of course be a different matter.

[Peter Gathercole]

Re: Performance hit

There is a performance hit, and although I've not got any evidence to back this up, I can explain where I think some of the performance hit is.

In a microkernel environment, each of the components, many of which run in userland, are scheduled as separate processes, or maybe in this day and age, threads into each component (although this may break some of the security that microkernels give).

So, what this means is that every time you need to move between modules (especially in historical systems with limited numbers of processors/execution units), you take a context switch. In a microkernel environment, where single system calls may involve multiple kernel components, this leads to multiple context switches for each system call, something that won't be such a problem with a monolithic kernel (I'm ignoring pre-emptable kernels here!)

To maintain the correct level of isolation, each context switch will cause the register set including the memory mapping registers, to be saved, and even on today's fast systems, saving the register set is expensive in clock cycles. There are mitigating techniques, such as some architectures have the concept of fast context switches, where only parts of the register set are saved (Power), or register rings (Sparc) or register renaming, or shared memory spaces used by either side of the switch protected by access keys (Intel - the thing that caused Meltdown).

These each have their problems, such as partial saves is a convention, not enforced, the register ring or alternate register sets being of finite size, and, well, flaws in the access protection on certain very popular architectures.

I think that when we go down the route of microkernels, as we probably should, we're really looking at needing massively parallel execution units in single systems, such that each microkernel component can almost be given it's own dedicated processor and memory space, so the only slowdown would be the message passing between the components, and even that could be mitigated with hardware assisted message queues.

Maybe we're almost there. In theory, nowadays it would be possible to tune execution units to the kernel component task they need to do, and this could possibly allow us to have CPUs with many heterogeneous execution units, with just the space on the die for that unit for the task they need to do, and an intelligent instruction or even thread level scheduler to direct tasks to the appropriate execution unit, much as we already do in super-scalar processors to cope with floating point, vector or BCD maths instructions.

I believe that we have seen some design architectures that are working towards this. I seem to remember there being some processor designs that had a power-of-two number of high performance cores for application work, with additional ancillary cores with different characteristics intended to execute the OS.

To my mind, this would be a very interesting development, but it would be a quite expensive experiment.

For many systems, the benefit would only be for security, bearing in mind that many computational intensive workloads spend almost all of their time outside of the kernel, by design.

[Claptrap314]

Re: Performance hit

It's been 17 years, and I was in validation, but I'm pretty confident that what I'm about to say, based on K5, K6, K7, 750, GPUL, and Cell microprocessors, still is pretty accurate. The physics doesn't change.

First, beware that there is a huge difference between threads and cores. Execution units available to a single core are shared between threads. In the designs I am familiar with, "everything" is shared out to the L1. In particular, the caches used for memory translation and the predecode buffers are shared. In the register file, the threads are kept separate by adding bits (or a bit) to specify which thread has the register. Everything thing else is just shared.

Multiple cores might share the L2 or L3. depending on the design. Of course, at the limit, you have NUMA systems, which essentially don't have main memory at all.

Since the Pentium & K5, memory bandwith has come to completely dominate processor performance. Design teams put an insane amount of effort into minimizing the amount of data that has to flow into (and, to a less extent, out of) the part. Cache architecture is therefore the major driver.

It looks to me that your idea makes really bad use of the cache. I'm also wondering just how much utilization the execution units would actually get.

[Peter Gathercole]

Re: Performance hit

I'm sorry, I let specific jargon get in the way of what I said. I was actually talking about 'threads' with no regard to the separation between hardware threads (is this just a Power term, or is it used for other architectures?) and software threads.

My main experience is on Power, which is a NUMA architecture (and contrary to your assertion, there is 'main' memory in NUMA architectures, all memory is accessible by every core, although there are progressively increasing speed penalties going through an off die, or an off MCM memory controller, which leads to the requirement to set core affinity to software threads to keep cache and memory local). I believe that what you are referring to used to be called "Distributed Memory Multiprocessors", but I don't believe that there are any mainstream system architectures of that type (unless you include discrete processors handling specific tasks as offload processors), and most modern multiprocessor systems that use a chiplet or multi-socket physical design are NUMA.

Context switches can also be damaging for cache efficiencies, as would moving software threads between cores, as either of these may end up invalidating any cached data that was in a cache associated with a specific core. Keeping specific microkernel modules on almost dedicated cores may actually increase some cache efficiencies, while preventing context switches on your full-fat cores.

I admit that there would be perceived inefficiencies, especially if you were to look at the overall core utilisation, but with specific, possibly different cores being allocated to different purposes, the idea would be to keep the full-fat cores used by the applications as fully utilised as possible. If you make the 'service' cores used for the microkernel tasks subsets of the full-fat core (how often is a core doing OS work going to need to do floating point or vector arithmetic!), it would be possible to 'spill over' from the service cores to the full-fat ones in times of extreme stress.

The point of the service cores would be to use less die space, and as a result, if they were unused for any period, this would not be seen as much of a waste. The benefit is that if you can keep the microkernel modules running in the service cores, and make them remain in those cores even when idle, you would not have to context-switch a full-fat core in order to service system calls, which was my original point.

In truth, I do not know whether it would be really practical, but it would be interesting to see a bit of a shake up in system design.

[Anonymous Coward]

Re: Performance hit

A monolithic kernel also performs a context switch when a component invokes a system call. In the microkernel there is, however, more overhead since it's based on messaging.

These hardware based limitations are IIMHO a good reason to write a microkernel in assembly language, where you have much more control over registers and (to some degree) on-chip caches. The MURTL microkernel is written in assembly, for example.

[Peter Gathercole]

Re: Performance hit

Probably a single context switch per system call for a monolithic kernel. In a microkernel, a system call may require several context switches if it involves code in more than one micro-kernel module. I don't know if this is really the case, I don't know how the micro-kernel modules divide up the work.

Depending on how the scheduling works, and how the modules interact with each other, just reading messages will probably require a context switch as well.

[grumpy-old-person]

Re: Performance hit

Long ago when electonic technology could not support the requirements for some really excellent computer architectures thre were some very interesting ideas (including Hurd!) but not feasible to implement.

Yet things like IBM's SWARD and the spin-offs were usable

Now technology exists to support all sorts of "better" architectures the past keeps us back.

Look at ERLANG VM for a software-only implementation where huge numbers of processes communicating using messages performs quite well even our current decades old architectures

[Charlie Clark]

Re: Performance hit

My understanding from talking to those who know more about the low-level stuff, is that the performance degrades only on Intel due to the way kernel and userland map onto different "rings" on x86. Infamously, it was for this reason that Windows NT went from being microkernel to monolithic. On more modern processors this really shouldn't make much difference any more. In theory, you can dedicate a single processor for kernel stuff. After all this is pretty much what happens with GPU stuff.

[Julz]

Re: Performance hit

Micro kernels have a sort of IPC style communication between the various OS actors but this in not necessarily the performance issue that you might imagine. The piece didn't mention the other main micro kernel around in the 70s/80s/90s; Chorus Systèmes offering.

As I've mentioned before, this was used in ICL's GoldRush system and specialist hardware was used to aid both intra and inter processor comms and memory management. The result was that, for it's time, it flew performance wise. Given that it used Sun UltrasSPARC IV processors, I would hope current tech would do better even without the MMU tweaks, especially given the amount of close to CPU memory that is now available. All micro kernels need is the amount of attention that the Linux kernel has had over the last thirty odd years and they would be equally performant.

Microkernels are, as Mr Tanenbaum argued (I did too but no one really listened to me ;), a much better solution than a monolithic kernel in so many ways. It's such a shame that they are languishing near the scrap heap. Let's give them some love...

[fg_swe]

They Don't

Just because the "commercial IT world" has let Chorus down, does not mean microkernels no longer exist. There are huge spheres with much less publicity, which use microkernels. Starting with A380 and Integrity 178.

See my other posts.

[Anonymous Coward]

Re: They Don't

Yep, INTEGRITY from Greenhills. Fabulous bit of software in my opinion.

I've always thought it interesting that those building stuff that really matters are often much more flexible about the tech they adopt (or at least, those that try and do an airliner's FCS on Linux tend not to get as far as production). They succeed because they don't muck about trying to make do with something mainstream / free / easy.

Whereas the mainstream software development world seems to get away with a lot of pontification about "security best practices", which usually is a long winded and expensive way of NOT saying "unless there's a bug in this gargantuan monolithic kernel somewhere", which of course there usually is. It's quite ironic when "best practice" advice includes the phrase "reduce your attack surface area" and goes on to discuss Linux settings instead of "choose a different OS".

It's horses for courses, of course, but as the Linux kernel keeps growing it's presenting more potential avenues for attack.

[Zolko]

Re: They Don't

unless there's a bug in this gargantuan monolithic kernel somewhere

Bug or backdoor. What are the chances that some binary blobs in the kernel contain NSA-supplied code ?

[BinkyTheMagicPaperclip]

Re: They Don't

Zero if you're running OpenBSD and your definition of 'binary blob' is 'opaque binary code executed by the kernel'.

Non zero if you mean 'opaque blob supplied by the manufacturer or processor vendor and uploaded to the device as firmware or to update processor microcode', but frankly good luck with attempting to build any modern system without that. If you're really into tin foil hats you can pay Raptor Systems an arm and a leg for a POWER9 based system - and you'll likely *still* have an issue if you want a working modern-ish GPU, as most of the AMD cards will want firmware loaded.

[Anonymous Coward]

Re: They Don't

You're incorrectly implying that a "binary blob" is something that cannot be analyzed or scrutinized. In the end it's all assembly language and there are plenty of people who can read it, including yours truly.

[Sil]

Re: Performance hit

is the performance hit really only 5-10%?

any benchmarks or real life examples to back it up?

I would be astonished if such a low performance hit were the main reason for the lack of adoption of microkernels,except in performance critical domains .

[thames]

Re: Performance hit

From what I've read from people working in OS development, the main problem with microkernels is that they are much harder to write and debug and that for the same amount of effort you could get more development work done on a conventional monolithic kernel.

Basing an OS on a microkernel doesn't remove any complexity from it. Instead it just moves the complexity from the code into the interfaces between the modules. Existing software development tools are great at writing, analyzing, and debugging a large code base. They aren't so good at analyzing and debugging the interfaces between message passing modules and so that work is thrown back on the developer to do the hard way.

The result is that microkernels are "simple" in theory, but very difficult in practice once you get beyond very simple cases. This is supposedly why microkernels have been relatively successful in embedded or special purpose applications (e.g. QNX), but far less so on desktops or servers which encompass a lot more features and use cases.

[fg_swe]

No

Mikrokernels have *much* smaller "effective" attack surfaces, easily three orders of magnitude less.

An exploit in the TCP stack sinks the Linux ship, but not a Mikrokernel. All you need to do on a Microkernel is to restart the TCP process and your secrets are NOT exposed. The worst effect of the TCP exploit is a TCP DOS event, which you might mitigate by firewalling.

See this chart: https://sappeur.di-fg.de/L4gegenueberLinux.html

[Anonymous Coward]

Re: No

AFAIK Linux is unique in having the TCP stack inside the kernel. Neither Windows nor FreeBSD nor MacOS do, it's all user-land libraries for TCP. Neither Windows or FreeBSD are microkernel either. Linux - already a monolithic kernel - simply got bigger again.

TCP in the kernel is a bit of a performance issue these days. A few years back El Reg carried an article about how the BBC had significantly improved the throughput of their iPlayer servers by putting in a pass through driver for the NICS and doing TCP processing in the iPlayer server software stack.

[Anonymous Coward]

Re: No

AFAIK Linux is unique in having the TCP stack inside the kernel.

IIRC Solaris also has it in the kernel.

[Anonymous Coward]

Re: Performance hit

From a programmer's perspective there's little or no difference in developing for microkernel operating systems. All operating systems virtualize calls to hardware.

When you program QNX you just include a C library where you call functions. The implementation of those functions format messages and send them to the kernel. You implement a callback for receiving results and status updates.

[Charlie Clark]

Re: Performance hit

BeOS and later Haiku demonstrated that there is no contradiction between microkernels and desktop OSes and BeOS API was considered one of the easiest to work with.

[grexe76]

Re: Performance hit and BeOS/Haiku

Just wanted to say exactly that.

While not strictly a micro kernel, BeOS and its open source successor Haiku have a kind-of microkernel, so even drivers run as add-ons but share the kernel memory space so can still crash the system.

All other processes run as application level servers, much like in The Hurd, and communicate with the kernel and each other using an efficient and highly performant, elegant messaging infrastructure that's easy to use. Way ahead of its time in 1997 and still unmatched today by mainstream OS's.

[Charlie Clark]

Re: Performance hit and BeOS/Haiku

The cost of context-switching on x86 is one of the reasons for the implementation. I've rarely experienced a crash but, when they do happen, at least you can play a game of hangman! :-D

Still don't really understand why Google doesn't switch to Haiku as the basis for Android, etc. Except maybe the NIH (not invented here) phenomenon that seems to be pretty strong there.

[Liam Proven]

Re: Performance hit and BeOS/Haiku

> Still don't really understand why Google doesn't switch to Haiku as the basis for Android, etc.

Well, it _was_ working on Fuchsia for that...

[bazza]

Re: Performance hit

> but very difficult in practice once you get beyond very simple cases

It depends on tools, and how one goes about it.

If you use your common or garden debugger (like GDB), debugging multi-process systems is quite hard. It's difficult for such a debugger to give you the control you want over specific processes / threads. VxWorks solved this problem very nicely - or at least its dev tools on Solaris did; you'd run an instance of GDB for each task you wanted to debug. This was really nice to use, and was not repeated when they moved the dev tools to Windows.

The other aspect is the design. If one considers how IPC is typically used, you're writing a byte stream down a pipe. The act of writing is decoupled from the act of reading, the pipe storing data in transit. This is not unlike a TCP socket.

The result is that this is an "Actor Model" system, and these have some specific problems (deadlock, etc). Apart from trivial cases, it's impossible to prove either analytically or through testing that these problems have been avoided. What can go wrong is that when the IPC system gets a little busier than normal, a circular dependency can suddenly rear its head and cause deadlock.

An evolution of Actor Model is Communicating Sequential Processes is a good way forward. In this, the act of writing and reading are linked; a write does not complete until the recipient has read the sent data; and "execution rendezvous". This is rather transformative; now, you can analytically prove a design (there is a Process Calculi that comes along with CSP), but the other quality is that architectural problems like deadlock rear their head every single time (so, testing is far more conclusive). This actually makes system (either application or operating system) development pretty straight forward; mistakes can still be made, but you soon learn of them. The downside is that the execution rendezvous takes up time; you need more signals (electronic signals) to flow to-and-fro between sender and receiver to coordinate the transfer. That's fine in a single CPU, but over a network or inter-CPU interconnect, the sender is inevitably going to be kept waiting for a little while. The other really nice aspect of CSP is that there's no hiding of inadequacies; if you've not got enough recipients of a message to keep up with the source of the messages, you're not hiding this fact in buffering up messages in IPC pipes; you learn pretty quickly that you need more recipients. [BTW, CSP - a late 1970's invention - has made a comeback in Go, and also in Rust].

I don't know if the microkernels out there are Actor or CSP systems. "Copy" data (which is what's happening when you use IPC) sounds like a bad idea, but these days I'm not so sure. From a point of view of the microelectronics, copying data between cores isn't so very different from accessing data on another CPU and having a load of cache-coherency traffic flowing between CPUs / Cores. The real answer is to have CSP built into the hardware, which is what the Inmos Transputer did.

What Go is doing is implementing a CSP / NUMA system on top of an SMP environment that is itself synthesised by CPUs that do not have SMP memory. Where Rust is interesting is that with its view of data ownership, it would be possible to take Rust source code and compile it / auto-parallelise for both an SMP environment (real or synthetic) or a real CSP environment.

With good tools, debugging CSP systems isn't very difficult (especially with the multiple debugger approach adopted by VxWorks). The execution rendezvous inherent in data transfer allows one to know a lot more about how processes / threads are getting along together, much more so than in Actor Model.

[fg_swe]

Parallel Program Execution Problems

You certainly need a proper theory/concept of parallel program execution. But there is no shortcut by either means of "shared memory" or "message passing". Both approaches have their pros and cons.

As always, aim for KISS, as overly complex approaches are at least initially hard to get properly running. Control complexity.

Then accumulate experience and you will become a seasoned engineer of parallel programming. Not really novel, the same you did(on a meta level) when you "learned walking" with serial programs.

[NickHolland]

Re: Performance hit

well, a lot of reasons.

1) people don't care about security. Tell me I'm wrong all you want, I'll have no difficulty showing you real world examples of how people love security as long as it gets in the way of literally nothing else. Security is unimportant to the vast majority of administrators, managers and architects. (I wish this wasn't true. I'd love for you to prove me wrong)

1a) if people cared about security, OpenBSD would be what people use.

2) Like it or not, many of your system administrators and system architects come out of the hard-core gaming world (and yes, it does show). When you got an understanding of your Linux or Windows hammer, all the world looks like the appropriate nail.

3) I don't think the security benefits of the microkernel idea are quite as well proven /in the real world/ as we'd like. IF you have a lot of control over the hardware (remember back when we said, "pick your application, the app picks the OS, the OS picks the hardware?), AND can keep the feature creep under control, then a microkernel might really win-out for security. But on modern diverse hardware and the expectation of lots of features, I've seen some pretty persuasive arguments from people who have demonstrated secure programming skills, that microkernel OSs can become a big mess of races and dependencies. Really, stripping features out of the kernel and putting security first (really first) probably matters more for security than a microkernel architecture could ever deliver.

[fg_swe]

One Example

"just" a Bluetooth stack is in the order of 300 000 lines of code these days. Waiting to be exploited by drive-by adversaries. It runs inside the kernel with full authority in Linux and Windows.

Very bad from an engineering and security point of view.

[DS999]

Re: Performance hit

If you run a full modern OS with all the trimmings you are going to have a ton of security holes. That's the nature of the beast just based on LOC. The difference between a more secure and resilient kernel matters less - yes it can stop attackers from getting kernel level permissions but they don't need that to obtain root/Administrator level rights which lets you get up to a ton of mischief even if you can't get the next level beyond root and break out of a VM. That's why hypervisors are kept small, essentially a microkernel in design.

The article mentions that Apple's Mach based XNU kernel imported a bunch of BSD stuff to address performance concerns. That was done by NeXT back when it was running on a 25 MHz 68040, so kind of necessary at the time. With the massive performance today maybe that's less of a concern. Apple has been slowly making changes that appear to be working towards moving networking and the filesystem outside the kernel to run in userspace, so they're trying to make their kernel a bit more "micro" bit by bit.

The thing about performance hits is that you can't point to a certain figure and say "this is your hit". It depends on what you do. If your load involves running stuff in userspace with few system calls the performance hit would be undetectable. If your load involves tons of I/O and thus many system calls you could easily cut your performance by over half! So for Apple's market which does not involve servers a true microkernel might be more amenable than it would be for Linux or Windows, both of which see a lot of use in servers.

Another way to mitigate the security concerns of microkernel vs not-so-microkernel is by putting the stuff that really matters for security into a different CPU running a different kernel entirely. i.e. stuff like Apple's Secure Enclave (which runs on an seL4 microkernel) and the equivalent on Intel and AMD CPUs (though they've unfortunately given them a dual role which includes not just security but also system management and DRM which makes them rather unpopular at El Reg for obvious reasons)

[Doctor Syntax]

Many in the GNU/FSF world seem to view the BSD unencumbered licence as anything between misguided and evil for allowing code licenced with it to be used in the way Intel has used it. How ironical, then, that code from BSD itself is used in Hurd. I'm sure there's a lesson there for the zealots. It's called pragmatism. Of course, being zealots, they won't learn it.

[Charlie Clark]

And Stallman's bloody ego.

[fg_swe]

No

There are operational reasons for microkernels, especially if a computer is connected to sensors and telecommunications networks. Soldier's lifes and your freedom, your security sometimes depend on it.

[Gary Stewart]

I view the BSD unencumbered license as a way to hide anywhere from useful to extremely useful additions/extensions/improvements in BSD and BSD licensed software behind a proprietary wall. To some people that is evil, to me it just slows down development of BSD software and using hardware that hides behind that wall (looking at you Apple), a problem that for the most part Linux doesn't have. There are of course exceptions like Linux driver blobs (looking at you Nvidia) and that is why I don't buy any hardware that uses blobs if there are any reasonable alternatives available. In most cases there are. So if you want to use the BSD license for your code go right ahead, I'll just have to look elsewhere for an alternative.

[Charlie Clark]

I don't think the licences have ever held the BSDs or Postgres or Apache back. FreeBSD was held back by a massive legal case with AT&T, Postgres just keeps getting better and better, partly because of the open licence: companies can easily contribute components without worrying about an audit.

[Missing Semicolon]

if Hurd is modular

Can you strip it down enough to boot a prompt from a floppy, like you used to?

[Blackjack]

[An all-FOSS microkernel for x86-64 PCs that can run nearly three-quarters of the packages in Debian is a great achievement. ®]

Yeah it is.

There are probably servers running on it, because if there is a Linux/Unix like tthing out there people will try two things, run a server and make Doom work on it.

It probably has a Doom port, pregnancy tests have Doom ports

[Liam Proven]

> There are probably servers running on it,

Unironically... I wrote this article last week. Most of the day, many GNU project servers were down and I had to do a tonne of extra legwork on the Internet Archive and so on.

There was a time, about a decade and a half ago maybe?, when there was a new Hurd release, they proudly demonstrated this by hosting the Hurd site on it, and it got Slashdotted into oblivion in minutes... and due to lack of load balancers or whatever it took _days_ to get it back online.

I don't think it is there yet, but I do think it might be possible to specify some tiny subset of a server's duties, make a hardened config of that, and deploy that in prod. Maybe.

Something that serve flat no-JS HTML pages from a RAMdisk over HTTPS and does nothing else, maybe.

[gerryg]

something I have never quite understood

Is there a material difference in systemic risk between a usable system based on a macro kernel and a micro kernel?

Everything I read and think I understand suggests micro kernels throw risk over the fence for others to deal with, whereas macro kernels plough on trying to handle everything. But I don't get any idea that a system based on one or the other is less risky.

All explanations gratefully received.

.

[fg_swe]

Re: something I have never quite understood

Macrokernels have an enormous attack surface and a single exploit sinks the ship of your computer. See my other post.

[Claptrap314]

Re: something I have never quite understood

Microkernels push work into user space a lot. Monokernels don't.

So, microkernels gain a lot in terms of having a much smaller kernel in privileged space.

HOWEVER, as mentioned, that's not the entire story. Microkernels rely on a lot of message processing and context switching. So, if you can compromise user-space code to pass a bad message, you might well be able to forge your way deeper into the system.

Which still requires a lot more work than "whoops--that printer driver isn't secure--you're p0wned!".

[fg_swe]

Security Point Of View

Microkernels have a *much* smaller attack surface. Thousands lines of code as compared the 40 000 000 of Linux(Windows similar or worse).

https://sappeur.di-fg.de/L4gegenueberLinux.html

A single exploit in these 40 Mio loc will sink the ship, as opposed to the Mikrokernel(seL4 here) frigate with plenty of compartments.

The L4 folks even tried to prove correctness mathematically, to a certain degree.

It's used for government and defence applications mainly.

(X11 included in my diagram, as it usually runs with UID=0)

[K555]

Re: Security Point Of View

"A single exploit in these 40 Mio loc will sink the ship"

Would that be dependent on it being a line of code in a mod that's loaded, cutting it down a bit from 40M is so?

[m4r35n357]

Re: Security Point Of View

There is no machine in the universe running anywhere near all those lines!

Come on guys, can we drop the "40M LOC" bullshit, because there are impressionable souls out there that believe it.

(moved to reply to correct post!)

[fg_swe]

Re: Security Point Of View

So it is "just" 10 mio loc versus 10000 loc for the seL4 kernel ?

Does not change the character of my argument.

[K555]

Re: Security Point Of View

No, but it reduces the hyperbolic element which might help it come across as less fanatical.

[Paul Kinsler]

Re: Microkernels have a *much* smaller attack surface.

I would have thought that it would be better to say that microkernels move most (or a lot of) the attack surface into userland; which might indeed solve or reduce the many problems introduced by that attack surface, but not -- I presume -- all of them.

[fg_swe]

Re: Microkernels have a *much* smaller attack surface.

Its a very big reduction, by up to three orders of magnitude loc.

An exploit in Bluetooth, in IP stack, in a filesystem, in USB stack, in a device driver etc. no longer sinks the ship. The exploit is contained in a subsystem.

All of which is great, security-wise.

[Phil O'Sophical]

Re: Microkernels have a *much* smaller attack surface.

If by "sinks the ship" you mean "crashes the whole system" that's possibly true, but most security exploits aren't intended to do that. They are more directed towards data access and exfiltration, or subverting the system for other purposes. For that it really doesn't matter if your exploit is in kernel or user land, it just needs to affect the part of the overall system which manages the resources it is targeting. Indeed, it may be easier to exploit a bug in accessible userland code than in the kernel.

[fg_swe]

Wrong Assumption

With "sinking the ship" I mean "extract all secrets from the target system OR commandeer the system". Both of which might be catastrophic in banking, policing, defence and other security applications.

For example: an exploit in Bluetooth will only affect "black"(enciphered) data, while the "red"(plaintext) data exists only on the application+ciphering process. Red data is protected and all the attacker can do is to disable the Bluetooth stack at worst. After moving out of danger area, a bluetooth process/stack reboot will achieve availability again.

Example regarding "Commandeering", a Cash Machine based on Windows or Linux will eject all of its cash upon successful kernel exploit. A microkernel based cash machine will only stop working upon an exploit in the TCP stack, but it will not spill its cash to the attacker.

[DS999]

Re: Security Point Of View

The L4 folks even tried to prove correctness mathematically, to a certain degree

Actually seL4 has been formally verified. Apple uses it as the OS for its Secure Enclave, which is exactly the sort of place you'd want a formally verified OS.

[fg_swe]

Re: Security Point Of View

Afaik, the seL4 correctness proof only claims that no memory errors can occur.

[fg_swe]

Integrity 178

An important mikrokernel for aerospace and special applications: https://www.ghs.com/products/rtos/integrity.html

[IGnatius T Foobar !]

...and nothing of value was lost.

This is fantastic. Hurd can absorb all the filthy luddites who hate Wayland and systemd, and all the tantrummy children who insist on saying "GNU/Linux", and maybe even all of the Che-shirt-wearing Rust programmers, and let the Linux world finally live in peace.

[fg_swe]

Re: ...and nothing of value was lost.

Beer helps ;-)

[keithpeter]

Re: ...and nothing of value was lost.

@iG

Couple of somewhat selective quotes from OA...

"More people can cooperate more easily working on smaller, simpler, cleanly-separated modules of code..."

and

"Hurd has a port of the Rust language..."

I was thinking that such a system would make changing the core language much easier. One module at a time. Interfaces are (presumably) defined message protocols. No need for arguments &c.

Icon: What we actually need is an Ark Fleet Ship B icon.

[jaypyahoo]

NetBSD for the win.

[fg_swe]

No

Not a microkernel, unlike seL4, Integity 178, Mach, Chorus.

[m4r35n357]

Re: No

So, with all of those microkernels, plus Minix, why has nobody yet created an open source desktop system?

Genode put their "money where their mouth is", everyone else is spouting hot air.

[Taliesinawen]

SystemD free ..

“The Shepherd is a service manager written in Guile Scheme that looks after the herd of daemons running on the system.”

[breakfast]

Re: SystemD free ..

I love that this works as a description of a part of an operating system and a story prompt.

[Jon Bailey]

Props to those still finishing the work

The Nouveau project was chasing "impossible" goals, until it wasn't.

Attempting a binary-blob-free RPi firmware that works is an "impossible" goal, until it isn't.

HURD, too, remains "impossible", until it isn't. :)

[My Coat]

> we still read Windows fanboys deriding Linux as an amateur OS.

So the authors do read the comments here ;-)

[CRConrad]

Not all of them.

Not all of the authors, that is; pretty much only mr. Proven, AFAICR.

And I wouldn't swear that even he reads all of the comments.

[Ian Johnston]

Since it uses NetBSD drivers, can we all insist that it's called NetBSD/Hurd?

[martinusher]

Hurd and Harmony

I don't know enough about the system to be able say this definitively but I have this suspicion that Harmony is a re-implementation of Hurd. It may not be exactly the same, especially in initial incarnations (Harmony's focused on servicing a product line's needs, Hurd is more academic) but the overall design goals of the systems seem similar.

Has anyone thought about this?

[steelpillow]

Other OS than Linux

This eclectic adoption of OS makes it all the harder to understand why the SystemD-ebian hegemony actively threw out all the other Linux-compatible init architectures. Did they ever explain why?

[m4r35n357]

Re: Other OS than Linux

They are hardly likely to admit to pride, vanity, hubris, and megalomania.

[Displacement Activity]

Yawn...

35 years and counting. Windows, which is apparently also an OS, has released 20+ versions in that time. Linux is now on v6.

Perhaps not so much an exercise in OS research, as an exercise in licensing.

[fg_swe]

Re: Yawn...

There exist highly successful microkernels, such as seL4, QNX and Integrity 178. Some of them might run in your car.

Oh and Minix runs inside your Intel CPU, without telling you.

[fg_swe]

Plus

Your security and freedom sometimes depends on secure microkernels, as your police, armed and security forces use them to communicate sensitive commands and intelligence. Which in turn protects YOU.

[CRConrad]

That's a Minus, for many people.

“Which in turn protects YOU.”

Well, not everywhere. In many places it's more that it protects the police and security forces FROM “you”. Places like North Korea, China, Russia, and nowadays the USA.

[Ilgaz]

Re: Yawn...

Next time look to Windows version itself, not the consumer facing name. E.g. NT4, NT5. At least they are sincere in that versioning of the *kernel*. For now.

[alisonken1]

Re: Yawn...

Windows, which is apparently also an OS, has released 20+ versions in that time. Linux is now on v6.

Well, to compare apples to apples, Windows had released 20+ versions in that time. Fedora Linux (part of Red Hat Linux originally released in 1994) is now on version 42.

I believe Windows is running NT Kernel 5/6 and Fedora is running Linux kernel 6.15

[Anonymous Coward]

Alternative Descriptions

Quote: "... Linux was good enough..."

Liam, surely you meant to write " Linux IS good enough"........

But I have a problem with "good"...........

When it comes to choices (you know....next car....next election......next restaurant....) I always think about "least bad".

So, for me, when it comes to Windows or IOS, "Linux is the least bad choice".

Just saying!

[Liam Proven]

Re: Alternative Descriptions

s> Liam, surely you meant to write " Linux IS good enough"........

No. I meant was.

The point is that what stifled development of alternative kernels, such as Hurd, early on was that Linux was there and good enough, in that critical period in the early 1990s when there was significant OS engineering occurring in the mass market: Win NT, Win95, OS/2 2, BSD on x86, and so on.

Linux killed off all the other FOSS Unix experiments. It was technologically unambitious but it did the job. That nearly killed the BSDs, it did kill OSF/1, and all proprietary Unixes, and the Hurd, and Chorus, and Sprite, and Amoeba, and HeliOS, and L4... the only survivors lived by fleeing into niche markets. That is what DR did when Intel's final production 80286 killed Concurrent DOS/286: it fled into a niche market, in DR's case that being realtime OSes.

Realtime also saved QNX, L4 and seL4. It didn't save Symbian or anyone much else.

Windows survived. Apple nearly died and only NeXT saved it. NeXT too had to move to x86 and not only did NeXT save Apple but Apple also saved NeXT.

I'd love a glimpse into some alternate universes...

* Where DR shipped a multitasking DOS-compatible Concurrent OS on the 80286 and thwarted the development of OS/2 at all.

* Where OS/2 did happen, but v1 was a native 386 OS, and Windows 3/NT/9x were never needed

* Where GNU on BSD shipped in the early 1990s and Linux never happened

* Where Acorn saved its workstation line with a all-solid-state cool-running laptop in the 1990s, and as a result, it had enough money to buy Be and make a BeOS-powered desktop multi-ARM-powered media workstation.

[FIA]

That means lots of separate little code modules that must talk to each other, sending and receiving millions of messages every second to coordinate their activities. Because of the sheer volume of communications, their performance becomes a significant burden.

To be pedantic, it's not the passing of the messages that are the problem (you still need to get the data in them between things regardless), it's that the messages are passed between user programs. In a completely microkernel approach this means things like reading from a disk may involve interactions between several user processes (the block layer, the file system driver process, the process requesting the data at least). Each one of these requires a context switch, which is the real killer.

QNX gets round this to some extent by allowing the target of a message to use the remaining slice time of the sender. So if program A sends a message to program B and B is ready to receive it will be paged in and allowed to use the remainder of As time slice to process the message. This is faster than just waiting for B to be scheduled as and when.

[Liam Proven]

> QNX gets round this to some extent by allowing the target of a message to use the remaining slice time of the sender.

Fascinating. Thanks for that insight.

[DrXym]

I remember Hurd from its announcement

I was subscribed to usenet groups when the thing was announced and it's funny that it's still not 1.0. Technical issues, governance and politics all conspired to doom the project, especially when Linus got fed up with the pace and chose pragmatism over correctness. A skeletal but functioning kernel in people's hands is better than something broken and in pieces.

I don't even think that micro vs monolithic kernel is entirely to blame although it can't have helped. Micro kernels do exist - I was programming QNX soon out of university and it ran remarkably well. If Hurd had thrown out a functional but imperfect microkernel a lot sooner then it might have been more popular. If Minix had been GPLd it might have taken a life of its own. But that's the pragmatic approach and I don't think pragmatism has ever been in GNU / FSF's DNA.

[CRConrad]

Re: I If Minix had been GPLd...

If Minix had been GPLd it might have taken a life of its own. But that's the pragmatic approach and I don't think pragmatism has ever been in GNU / FSF's DNA.

So you're blaming GNU / FSF for Minix not being GPL...?

[frankvw]

It's not just technical issues

One major reason why the adoption of microkernel architectures over monolithic ones hasn't made great strides is not the pros and cons of monolytic kernels vs. microkernels. A major hurdle is the natural inertia of the existing code base. There's a ginormous pile of production code out there that has to be either rewritten or scrapped, and that causes a certain amount of lock-in that will take a lot of work, investment and determination to break. That's just not been happening, nor can I see it suddenly start to happen anytime soon.

There's also a cultural hurdle: seasoned devs who work on stuff like this are (of necessity) good at what they do, but they also tend to be reluctant to simply ditch everything they know in favour of something entirely new. The recent brouhaha around the use of Rust in the Linux kernel is a good example of that.

All of this has given microkernels a bit of a bad rep in Linux land. If, the reasoning goes, this is such a good idea, why has it been around for a quarter of a century (give or take) without amounting to anything mainstream? Let's face it: work on the Hurd started in 1990 and by now it's "made progress", but is still highly experimental and nowhere near being production-ready. Successful commercial microkernel architectures tend to be limited to highly specialized niche applications (typically industrial, embedded, or where security is so important that performance is considered expendable). So nobody is holding their breath waiting for microkernels to take the market by storm. There's simply too much investment in monolithic architectures and not enough projected benefit in switching to microkernel architectures for anyone to be in any kind of hurry to adopt it.

[fg_swe]

"Niches"

Just because you are a datacenter engineer and have limited vision to the borders of PC and server technology, means little.

For an aerospace control system engineer, datacenters might appear dull. Full of non-realtime machines with enormous RAM, but questionable security. For him, the world revolves around Integrity 178, seL4, QNX etc.

In the automotive world, AUTOSAR Classic is the go-to mid size(In the order of 1MByte RAM and 1MByte Flash) OS standard. We use Windows as a development platform, but could easily do the same work on Linux or BSD, if the Vector tools were available there.

[DrXym]

Re: It's not just technical issues

I think Linux won simply for pragmatic reasons - it existed and Hurd didn't. By the time Hurd eventually showed up in a functional form Linux had already been ported to run against every system under the sun and was a proven platform.

Since you mention Rust in passing, Redox is a microkernel written in Rust and arguably that is attracting more interest than Hurd is at this point. I wouldn't call Redox complete by any stretch either but I could see it occupying a similar niche as QNX does.

[fg_swe]

QNX on RPI

https://www.hackster.io/news/blackberry-s-qnx-seeks-hobbyists-and-makers-with-free-non-commercial-license-raspberry-pi-image-7c53320cac11

Looks very much like any other Unix, except that the compiler runs on the development machine, as the target usually is too small for a compiler.

[m4r35n357]

Re: QNX on RPI

Just some random thoughts . . .

They can keep their dangling carrot - who is going to blow their personal data at things like this these days?

VS Code - no thanks!

Using the Pi4 as a captive "target" is dishonest and manipulative - a Pi5 can run any compiler easily. Even a Pi4 can do it if you are patient.

[fg_swe]

Re: QNX on RPI

Well, you can always find the elefant's a$$, if you search hard enough.

But - QNX is a proven realtime OS, with lots of use cases:

https://www.firmenpresse.de/pressinfo210759-jeder-nutzt-qnx-30-beispiele-zum-jubil-um-der-firma.html

Also, it is Unix-like, which makes it interesting for all software engineers with POSIX experience.

Maybe you kindly look at the elefant's trunk and its phantastic capabilities ?

[harekrishnarama]

seL4 is the answer

The problem with Hurd is the microkernel overhead.

seL4 microkernel is all dressed up with no place to go.

All we have to do is port that in, and we can dump Linus Torvalds the tyrannical covid-19 vaccine pusher for Bill Gates, and Linux forever.