Upgrading Linux with Rust looks like a new challenge. It's one of our oldest

The Rusting of Linux proceeds apace. Of course there are problems, some technical, some very human. Last week saw one of the leading Rusties sign off from the project, quoting "non-technical" barriers to progress. That'll be people, then. Frustrated software developer Rust for Linux maintainer steps down in frustration with ' …

[Paul Crawford]

Why a new language?

The thing that always gets me about computer scientists is when there is a problem, the first thing they do is invent a new language to try and solve it in!

Take C versus Rust, the bone of contention here is the existing C developers having to learn and maintain a C-to-Rust API that is static so the Rust folks can write things that interface to the MASSES of existing kernel C code. And for a project that famously and possibly by-design has an API that is fluid-ish and defined by the C structures used.

Now had the issue of memory safety, etc, in C (or the lack thereof) been addressed by creating a sub-set of C and/or a few features to mitigate some issues like use-after-free then it would have been fairly trivial to do this, just set some flag on the compiler to new code that prevents careless/dumb mistakes that are oh so common, and the remaining aspect of compiler interfacing to the wild bad world of traditional C is not an issue. Same for existing C code, try compiling with the mythical safe flag and see if it can be tamed in a small way (great) or is too hairy and best left alone (maybe common). But no, a new language it had to be...

[prandeamus]

Re: Why a new language?

"Now had the issue of memory safety, etc, in C (or the lack thereof) been addressed by creating a sub-set of C and/or a few features to mitigate some issues like use-after-free then it would have been fairly trivial to do this..."

If wishes were horses...

I hear ya. But consider that the memory-safety aspects of C have been known since day 1 and no doubt there's chapter-and-verse somewhere out there in K&R or some Bell labs memo. Subsets of C have been created but C is so deeply dependent on certain things being done via pointers I don't think anyone can have something which is both a subset of C and capable enough to build kernel or drivers. Or maybe there is, but for some reason it didn't have traction.

[Paul Crawford]

Re: Why a new language?

Ultimately all variables (except those that can be kept in CPU registers by the optimiser) are accessed via pointers - machine addresses that map to a location in RAM, etc. The issue for language safety is how to you make sure that access is to memory safely set aside for such use? In all languages I know of common variables are done by the compiler and that is safe, for arrays or dynamically allocated storage there be dragons...

The issue with C is not when you do array[idx] sort of work, that could be transparently wrapped by check-code, etc, by the compiler to be sure you don't overstep the bounds at modest speed penalty, or maybe a good compiler/lint tool will be able to verify that idx is always inside the declared size of array[] for example, in a fixed for() loop, and so skip the overhead of dynamic testing.

The issue for C as I see it* is when you get manipulation of array as a pointer as if it is any sort of variable. Yes, that is exactly how the CPU works in most cases, but it leads to the harder to spot memory bounds errors. Same with use-after-free, that could be blocked by the same sort of trick as the electric-fence library where allocated memory is invalidated on free() so you get a segmentation fault if it is then abused. Yes, your program bombs out which might be an issue (but its broken already, ain't it?) but it avoids the security implications of a jump in to program space via such an oversight.

[*] pun intended. Yes, I won't be winning any joke awards here.

[frankvw]

Re: Why a new language?

I remember a phrase from the Borland Turbo C 2.0 manual (yes, I'm that old and crusty) describing a key aspect of C: "It gives you plenty of rope to swing with -- or to hang yourself." This, to me, describes C accurately. It's a power tool that will move mountains and take off both your legs at the knees with equal facility. Bluntly put, C has no safety net. Which is fine if you're the kind of tightrope walker who will never fall. The problem is that the vast majority of tightrope walkers can fall any moment, but they choose ignore that risk with confidence. The result is code that may lead to buffer overruns, use-after-free issues, out-of-bound reads and writes, and what not. Working around that fundamental problem with additional pre-compiler checks to vet code health is like, say, accepting an operating system that is insecure by design and then trying to fix that with an entire industry worth of antivirus software. (You know the one I mean.)

Rust is more safety net than tightrope. Used properly, it won't let you fall even if you try. But it also requires you mess around with a truckload and a half worth of entirely unfamiliar safety gear that you absolutely must wrap yourself in to the point where you feel you can't move before you can even begin to ascend the ladder on your way to the tightrope. This, as has been pointed out repeatedly, is the main problem: Rust puts the bulk of its changes in both attitude towards programming and in language and coding right at the front.

To a seasoned kernel developer who grew up on C and never felt a need for anything else that means a challenge not unlike a patient having to go through months of rehab after an accident just to learn how to walk all over again. It's the sort of thing you only do when you absolutely have no choice but to bite down hard and deal with it.

[nijam]

Re: Why a new language?

> Rust is more safety net than tightrope.

It's the kind of "safety net" that operates mainly as a strip hazard.

[prandeamus]

Re: Why a new language?

A more elegant phrasing is found in the BCPL (strong influence on C) reference documentation

"The philosophy of BCPL is not one of the tyrant who thinks he knows best and lays down the law on what is and what is not allowed; rather, BCPL acts more as a servant offering his services to the best of his ability without complaint, even when confronted with apparent nonsense. The programmer is always assumed to know what he is doing and is not hemmed in by petty restrictions"

https://archive.org/details/richards1979bcpl/page/n11/mode/2up page 5.

[martinusher]

Re: Why a new language?

There isn't anything in K&R about memory safety, its just assumed that memory unsafe operations are 'bloody stupid' and that no programmer would knowingly allow them to arise. Wide use of this language for purposes it wasn't designed for leads to problems as the need to get code out quickly conflicts with the need for deliberate and careful design followed by extensive testing.

The problem of producing a programmer proof language is almost as old as computing itself. I'd guess that ADA was a bona-fide attempt to make a language that no matter how hard a programmer tried they couldn't screw things up. I'll have to leave a discussion about whether this is true or not to people who work with it regularly; my guess is that it certainly ensures code safety but -- unfortunately -- doesn't do anything to help with logical screw ups (the code is 100% syntactically correct but doesn't work properly....). Automated software design tools don't help us much with this logic problem, ultimately its going to come down to "people who know what they're doing", anathema to our corporate masters that like to think in terms of interchangeable work widgets but, regrettably, a fact of life- -- skill's products can be canned for reuse but skill itself is a bit more difficult to replace.

[Dostoevsky]

Re: Why a new language?

Ada is a good language, and definitely an inspiration for Rust. It's just so little used...

[Anonymous Coward]

Re: Why a new language?

... a new language it had to be...

Indeed ...

Finding reasons for this can probably make a long list.

But the first one I can come up with is that the new generation finds the tried and true C language too hard / time consuming to learn how to master it.

ie: do not think it worthwhile, so they come up with the idea of a shortcut.

Not being a programmer/developer, it seems to me that asking existing C developers to learn and maintain a C-to-Rust API so that the Linux kernel can be written in Rust is basically the same as pretending that those C developers do the work for the Rust crowd.

And even if this came to be, be certain that at some point there will be a need for people who actually know how to write in C.

When they cannot be found, someone will probably come up with a brilliant idea: let's write a Rust-to-BS API.

Sign of the times we live in.

.

[JoeCool]

Re: Why a new language?

In the entire history of "high level" language evolution, the innovation has nearly always been to make the language "more" : more featurefull, more rigorous, more regular, more abstracted, more like math, more like speach, more like thought, more like messages.

So, I don't think complexity or difficulty of the old language is the issue; I am certain Rust is quantitatively "harder" because it is trying to solve bew or diffierent problems, or offer new solutions to old problems; it's not a shortcut.

[Avfusion]

Re: Why a new language?

If Rust was a shortcut or "easier to learn than C", I'm sure we'd have far less people dropping it at their first borrow checker holdup. The Rust forums are 50% "How do I make the borrow checker like me" posts by weight.

In my experience, Rust code is rarely shorter or easier to write than its C counterpart. It takes a lot of discipline to write anything sufficiently worthwhile in it.

[R Soul]

discipline

"It takes a lot of discipline to write anything sufficiently worthwhile in it."

Unlike like writing code in any other programming language.

[42656e4d203239]

Re: discipline

>>Unlike like writing code in any other programming language.

ADA would like a word....

[Dostoevsky]

Re: discipline

Bro has never heard of COBOL.

[nijam]

Re: Why a new language?

> It takes a lot of discipline to write anything sufficiently worthwhile in it.

It takes a lot of junmping through hoops to write anything sufficiently worthwhile in it.

[containerizer]

Re: Why a new language?

> In my experience, Rust code is rarely shorter or easier to write than its C counterpart. It takes a lot of discipline to write anything sufficiently worthwhile in it.

This statement is probably not true if the definition of "sufficiently worthwhile" includes a memory safety guarantee.

[nijam]

Re: Why a new language?

> In the entire history of "high level" language evolution, the innovation has nearly always been to make the language "more" ...

Except when the industry collectively turned it's back on Algol68 in favour of ... well, anything really.

[martinusher]

Re: Why a new language?

Algol68 was a bit tricky to implement so a practical subset, Algol68R, was built. I really liked that language, everything that followed seemed a bit weak. Purists looked on 68R with disdain, of course.

IT was for me an early introduction to language snobbery.

[Drakon]

Re: Why a new language?

> Not being a programmer/developer, it seems to me that asking existing C developers to learn and maintain a C-to-Rust API so that the Linux kernel can be written in Rust is basically the same as pretending that those C developers do the work for the Rust crowd.

The Rust maintainers were going to take on this responsibility.

[jake]

Re: Why a new language?

"The Rust maintainers were going to take on this responsibility."

Were?

[Richard 12]

Re: Why a new language?

They explicitly weren't.

The request was for the C maintainer to add "maintain a stable Rust API" to their tasks.

And the implicit demand was for the C maintainer to also maintain the Rust code in the near future.

[TheRealRoland]

Re: Why a new language?

"We'll save the day. Oh, and clean up after us, ok?"

[two00lbwaster]

Re: Why a new language?

Part of the issue with that is that Rust is not just about addressing memory safety. A subset of C would probably be a new language too, and anyway, they only just moved for C89 to C11 a little while back so hope long would it take before a version of C with memory safety would grace the light of day

[Dan 55]

Re: Why a new language?

We're at C17 with C23 in the works.

[Lipdorn]

Re: Why a new language?

There is a C17 standard with C23 in the works yes. But as of now (2024/09/10) Linux itself uses GNU C11.

https://www.kernel.org/doc/html/latest/process/programming-language.html

[Anonymous Coward]

Re: Why a new language?

Paul - 100% agree with you except one thing: it shouldn't necessarily be a compiler flag but an improvement to lint. All the work that went into Rust could have gone into lint but that's less sexy than writing a new language, as you say.

[F. Frederick Skitty]

Re: Why a new language?

Lint is a static analysis tool, essentially the same as compile time analysis, and there are a lot of memory bugs that will only be encountered at run time with C or C++ code. Hence Rust having its "borrowing" approach to memory management since that means it can be analysed at compile time. Retrofitting that to C would still require a huge change to existing C code in order to take advantage of it.

As pointed out above, there are libraries like Electric Fence that can make memory bugs at least non-exploitable, but they have overheads and may not be appropriate for kernel code. I used Purify to find memory bugs in C code I worked on, but this had too much overhead to be enabled on production code. It also required a level of unit testing that again may not be easy with kernel code.

[Anonymous Coward]

Re: Why a new language?

> Lint is a static analysis tool

That's a fair point and I can see that I should have been clearer.

lint doesn't have to remain a static analysis tool - it could be extended to do run-time analysis as well. Sure it's a ton of work and the instrumentation APIs need to be present but far more than that level of effort has gone into developing Rust.

I feel that analysis tools should be analysis tools and the compiler should be the compiler. Not just because of the Unix philosophy of programs that do one thing and one thing well, but also because a side-effect would be to allow people to develop and offer skills as "software security analysts". I feel it would be a lot easier to persuade a company to allow a review of their code for security issues using tooling, versus allowing free access to the source and the "corporate secrets" that go with it.

A second, rather obvious advantage is that Rust can only improve code re-written in Rust, whereas enhancements to lint could confirm that existing C code is safe and therefore doesn't need re-writing.

[containerizer]

Re: Why a new language?

The whole point of using C is for the compactness and efficiency of the code. If you start adding stuff like this on it defeats the purpose. And unit testing can't give you what compile time memory tracking gives you.

[bombastic bob]

Re: Why a new language?

I like that - a lint-like external tool for 'C' that evaluates code for "code smells". Make it compiler independent, even better. Just add it to Makefile as part of the .c.o rule, perhaps "lint-like-tool cc [args]" as the command line. works for me!

Like a BETTER MOUSETRAP

[Mage]

Re: Why a new language?

Upvoted.

But C was obsolete by 1983. Why did Linux use it? Loads of choice by 1987.

Linux kernel was first released in 1993.

[Paul Crawford]

Re: Why a new language?

1) C is a very good choice for writing some of the lowest-level of an OS as it lets you do all-sorts of dirty things with memory allocation and hardware access. What it is not so good for is writing very large projects with lots of code that really has no business doing dirty things with memory, etc.

2) C compilers were/are available for practically every CPU, which is still not the case with many languages.

Basically the C language is the universal assembler!

[Mage]

Re: Why a new language?

"C is a very good choice for writing some of the lowest-level of an OS"

Yes, but in 1970s, not in 1980s and 1990s

[jake]

Re: Why a new language?

"not in 1980s and 1990s"

Personally, I found C to be better than any of the alternatives in the '80s and '90s.

Still do, in fact. And it would seem that I am far from alone.

::shrugs::

_{If we can agree to disagree, I'll be happy to buy you a beer.}

[Snake]

Re: Why a new language?

"Basically the C language is the universal assembler!"

That is indeed a good property. But the Rust community, plus this article's author (thankful) note something important: That property is no longer good enough. With the incredible complexity of modern software & multitasking hardware, memory safety has now become important and a goodly-number of C programmers are refusing to acknowledge this new reality in the guise of maintaining their hold on the legacy benefits of C (universal system accessibility and close-to-metal programming power). As was quoted by frankvw in an earlier post here, C "[It] gives you plenty of rope to swing with -- or to hang yourself." But pride isn't allowing some C programmers to hear all this; it's easier to think that you are not the one to ever make a mistake or even a simple oversight in your coding, so why believe that we need anything like Rust??

[containerizer]

Re: Why a new language?

I don't recognise the idea that memory safety only recently became important. People have been banging on about C's limitations in this regard since the language first became available.

It's not like the industry has been frozen in aspic. Outside of specialist fields (kernels/device drivers, low-latency software) C/C++ have been replaced with Java, C-Sharp and Python, and Go is making some inroads in the systems programming side.

The remaining software which continues to be in C is that which cannot be easily migrated to any of these languages. For most cases, the cost of continuing to use C is lower than the cost of replacing codebases with Rust. The workarounds for C - static analysis, stack smashing protection tools etc - are deemed "good enough" most of the time.

I can well understand why some kernel developers might see that all of this is a solution in search of a problem. And the Rust crew aren't the first to make this case - there have been attempts to push C++ on the kernel in past years too, and the same kind of reaction when it was blocked - that the devs are a bunch of luddites trying to hold back advancement. It's not that simple.

If the world really is right, and the kernel devs are wrong, then this problem will solve itself in a different way : people will write a from-scratch, Rust-only kernel which is compatible with Linux at the system call layer and can therefore be dropped in to existing distributions. If that idea seems too far-fetched, then it telling us something about the cost/benefit of using Rust.

[JoeCool]

Calling this bluff

Care to name those "lots of other choices" ?

C89 disproves that ridiculous and meaningless characterization.

[nijam]

Re: Why a new language?

> But C was obsolete by 1983.

Or, by most measures, C is not obsolete yet.

[bombastic bob]

Re: Why a new language?

Have you ever

* written a device driver or kernel module

* coded for a microcontroller

* used assembly language for speed

* hand-optimized inner loops for speed/efficiency

If not, THIS is why you do not understand why Linus chose 'C'. It's also true that UNIX and BSD kernels are written in 'C' so the BSD network stack (for example) could be easily ported to Linux.

[Bebu]

Re: Why a new language?

"But C was obsolete by 1983. Why did Linux use it? Loads of choice by 1987."

Back then I followed the development of C standardization - the evolution of K&R C to ANSI C (C89) [ISO]

I actually obtained a copy of the ANSI standard - two things stand out for anyone that has dealt with standards documents viz the C89 standard document is short, rational, pragmatic and readable. C89 is probably unique in this. ;)

In the 1980s the commonest systems were comparatively low powered, with small primary [ram] and secondary [disk] memories which were comparable with the smallest contemporary embedded systems.

Self hosted development precluded most of the "non obsolete" options. If you were running a Z80 CP/M-80 system you pretty much had a choice of assembly or a C compiler (a few choices.)

Modula-2 compilers were available and arguably a safer language but never quite as good a fit when dealing with low level hardware.

I also remember that most of the "developers" were more likely to be engineers or scientists with minimal or no exposure to computer science (CS100 ;) but who could digest K&R in one sitting - try that with Ada. :)

While trying to revise or rewrite history is currently very much de rigueur it is pointless - it is a fact C dominated software development from the 1980s and to some extent still does - pretending otherwise is not sanity. Whether C remains dominant is a choice that can be made now and in future.

[Anonymous Coward]

Re: Why a new language?

Rust is a classic case of not invented here, also the lack of a formal definition would seem to indicate it is being developed by people who have little real understanding of what a programming language, probably because they did not study Computer Science…

Not saying that Ada is the solution to everything, but I suggest the Rust community take a look at the reimplementation of Linux in Ada; they will learn something.

[Dostoevsky]

Re: Why a new language?

It has a formal definition, just not a stable ABI. Its design was consolidated from many sources, most of them academic.

[that one in the corner]

Re: Why a new language?

> The thing that always gets me about computer scientists ... the first thing they do is invent a new language to try and solve it in ... Rust

You are aware that Grayson Hoare created Rust as a side-project whilst working as a Dev at Mozilla?

He wasn't an academic Computer Scientist.

CompSci certainly creates plenty of new languages - it is a good way to make your experiments[1] concrete and demonstrable, especially when publishing (and creating a thesis!). So, yes, of course CS invents new languages (or modifies existing ones).

But very, very few of those are ever even intended to "go mainstream", certainly not as-is. The *concepts* are hopefully going to end up somewhere practical, but like the bulk of science, what Joe Public ends up using day to day looks little like the original publication.

On the other hand, the languages you actually *see* as a Dev grunt[2] tend to be - rather more pragmatic. PHP, Perl, Python, JavaScript - yes, even C and most definitely Rust.

Curiously, you can tell - by looking at the language specification. Or the lack thereof.

To get your CS thesis best marks, you have a specification, and you use comparisons of this to point out where your nifty ideas are new and different.

To get your personal side-project out there, you fling together all the example programs and the compiler/interpreter executable, then try to keep up with the various totally random demands[3] that (re)shape your baby. After a while, if you (and everyone else) are lucky, there is enough interest that (other) people start work on a proper language specification, for pragmatic reasons.

[1] as in, the actual "science" bit.

[2] except for the lucky few who actually get paid a salary, not just a sequence of grants, to work on production code in an "obscure" language, such as ML.

[3] like JavaScript changing its outer appearance to look more Java/C'ish whilst underneath... ok, JS is a rather pathological example...

[that one in the corner]

Re: Why a new language?

>> The thing that always gets me about computer scientists

> You are aware that

>> But no, a new language it had to be...

Or, more succinctly:

Just use the doll and point to where the Computer Scientist touched you.

[Anonymous Coward]

Re: Why a new language?

Interesting he is on record as saying he prefers Swift and thinks Rust will be superceded ie. It isn’t the silver bullet many Rust devotes think it is.

He is probably in a good position to understand the limitations and omissions of Rust, but can also see the important contribution Rust has made to the development of programming languages.

[Dostoevsky]

Re: Why a new language?

Graydon's design for Rust had a GC and green threads. It was Go, before Go was popular. Obviously it's been out of his hands for a while now.

[StrangerHereMyself]

Re: Why a new language?

C is a Systems Programming Language and therefore doesn't require or need bounds checking or memory protection. The lack of these isn't a design fault it's a FEATURE!! The error we keep making is that we use a Systems Programming Language to write application software!!

We should invent a new Application Programming Language which has a the same familiar C syntax, but with aforementioned protections built-in.

[Richard 12]

Re: Why a new language?

That's such a great idea that it's already been done - and given the slightly ironic name "C++"

[MrBanana]

Re: Why a new language?

Er, no. C++ is mostly a bunch of syntactic sugar on top of the same old memory problems that C has, but only worse if you want to do clever things.

[StrangerHereMyself]

Re: Why a new language?

Not a chance. C++ is a superset of C and therefore the same limitations (no bounds checking or memory protection) apply.

Yes, you can add these to C++ objects but you can also choose not to, which makes it on the whole just as unreliable as C.

[Richard 12]

Re: Why a new language?

It isn't.

There are quite a lot of things that have a completely different meaning, or are even straight up prohibited in C++.

The confusion likely arises because most C++ compilers have a C mode and automatically switch languages based on file extension.

[StrangerHereMyself]

Re: Why a new language?

So how does this work exactly?

I program in C++ all the time and haven't seen any protections against buffer overflows, memory leaks or use-after-free.

[JoeCool]

B O O S T

also, std::string is pretty robust. you can choose to use the safe apis, or not.

[matjaggard]

Re: Why a new language?

But Rust has proven that a Systems Programming Language as you call it can and should have safety features - they just can't be implemented at runtime if you need performance. That is why a completely new language was required, you can't add compile time checks for memory safety for C code because as soon as you call a method, especially one that passes data between threads you can't prove who owns the data and has the right to free it for example. That set of concepts is what people find hard to learn about Rust and that is exactly what's required for provably safe code.

[StrangerHereMyself]

Re: Why a new language?

Even if that were true then Rust should only be used for Systems Programming (i.e. operating systems, device drivers) and not for developing application programs (and that includes tools like Unix utilities).

The learning curve for Rust is simply too great to force it on application programmers. What we need is a compiled Application Programming Language with safety features and a familiar C-like syntax. C# could've been that language was it not that Microsoft chose to make it a runtime-bound language.

[James Anderson]

Re: Why a new language?

Languages for courses.

C - if you are updating an existing large low level set of code written in C ( e,g. The Linux kernel ).

Rust — if you are writing a new isolated feature that requires low level access to the hardware.

GO — if you are writing a high performance multi threaded server app. Incidentally the existence of GO negates the old dog can’t learn new tricks argument — it was written by old guys!

Php — if you want a sever application up and running quickly and performance is not a major issue.

Java — if you work for a Dow jones listed company you have no choice.

Python — if you are doing something new and expect the project to evolve — but not if you need it to perform.

JavaScript — stuck with it if you want to run in a browser.

Swift — if you can’t stand JavaScript.

[bombastic bob]

Re: Why a new language?

"The thing that always gets me about computer scientists is when there is a problem, the first thing they do is invent a new language to try and solve it in!"

New/shiny and "It is OUR turn now" are a large part of that.

Arthur C. Clarke's "Superiority"

[Herby]

Superiority

Great (short) read. Very worthwhile.

[containerizer]

Re: Why a new language?

> Now had the issue of memory safety, etc, in C (or the lack thereof) been addressed by creating a sub-set of C and/or a few features to mitigate some issues like use-after-free then it would have been fairly trivial to do this

"Come on boffins, get of your backsides and make C safe!"

[jake]

There is no doubt in my mind ...

... that eventually, C will no longer be used as the language for the Linux kernel. It is inevitable. Things change over time.

However, I have many doubts that Rust is the language that will be the one to take the place of C.

I rather suspect that it won't be the next language du jour, either. Nor the next.

What it'll take is a major paradigm shift[0] in programming that all (or at least vast majority of) kernel coders can get behind. My gut feeling is that the seeds of this new way of looking at programming haven't even been laid as yet. And no, before anyone says it, so-called "artificial intelligence" will not be a part of the answer.

Thus the bulk of the Linux kernel will probably still be coded in good old C long after I am gone, and probably well past my great-grand kids programming careers (assuming).

_{I hate that phrase just as much as you do ... but I think it actually fits there.}

[that one in the corner]

Re: There is no doubt in my mind ...

> However, I have many doubts that Rust is the language that will be the one to take the place of C... What it'll take is a major paradigm shift

All this argument over Rust is the sound of people trying to make a paradigm shift without disengaging the clutch[1]

[1] or even having built a finished clutch in the first place, to overstretch the analogy and strain the joke.

[Dagg]

Re: There is no doubt in my mind ...

However, I have many doubts that Rust is the language that will be the one to take the place of C.

Been there heard that except it was "However, I have many doubts that Java is the language that will be the one to take the place of C."

[Anonymous Coward]

As much as the rustfappers like to market it as a better version of C, it really isn't. It's just yet another niche language with different syntax and no discernable benefits.

The reason why embedded developers like C is BECAUSE it allows easy and direct reading and writing of memory. It's not an undesirable behaviour that we want curtailed.

It gives you the ultimate freedom to write the code in whatever way you find most optimal without the langauge trying to force you into any particular structure or paradigm.

Rust doesn't offer any advantage over C in those kind of projects.

Projects that don't require that, probably shouldn't have been written in C in the first place. Rust might be a good language for those, but then so might any other high level langauge.

In 10 years time, nobody will remember that Rust even existed, but C will still be going strong.

[zimzam]

"It's not an undesirable behaviour that we want curtailed."

This is the problem. You want to write code the way you want to, but it's not about you, it's about the maintainers who come after you who have to try to parse the idiosyncrasies of how you write.

[jake]

If the maintainers know how to properly parse C, there should be no problem.[0]

If the maintainers do not know how to properly parse C, they shouldn't be publicly mucking about in the mainline kernel.

Substitute C with any language, and the kernel with any project.

_{[0] Assuming no intentional obfuscation, of course, which is one of the major sins of shared coding.}

[An_Old_Dog]

Rust to the Rescue (Not)

1. Rust will not fix or prevent those ideosyncrasies.

2. You can submit a patch, mod, or new feature for the kernel, but it will be reviewed before it is accepted. If your coding style is wildly ideosyncratic or otherwise difficult to deal with, it probably will be rejected.

3. Ideosyncratic is not necessarily the same as "hard to understand", or "hard to maintain."

[Doctor Syntax]

"it's about the maintainers who come after you who have to try to parse the idiosyncrasies of how you write"

If maintainers have a problem it's likely to arise not from the language the code was in but in the lack of documentation. If something's idiosyncratic leave a note as to why.

[zimzam]

They should, but they don't. Documentation is also subject to the idiosyncrasies of the author. I'm sure we've all read comments that seemed perfectly clear to the author but left "obvious" points out.

[Anonymous Coward]

> They should, but they don't ... I'm sure we've all...

Which is why serious projects, like the Linux kernel, have so much review - and no objection at all to rejecting stuff until it does describe/explain "the obvious" to the wider audience.

[Androgynous Cupboard]

Twenty hours after you posted and no-one has picked the obvious hole? C gives you the freedom to do what you want, I agree, and someone else pointed out that burden falls on the maintainers.

I’m here to note that it’s the users that have to live with your decision. For Linux, there are billions of ‘em. So it’s necessary to fetter your freedom with tools and techniques (and, maybe, languages) that will prevent a moments inattention on your part from doing wide ranging damage.

[DrBobK]

C is easy to learn. It is easy to do amazing things with it. Pointers make interacting with hardware straightforward. But, very big but, it is very hard to write safe C.

[Michael Strorm]

> "very big but"

You make that sound like a bad thing, but Sir Mixalot would disagree...

[Anonymous Coward]

depends on how much of a crap programmer you are.

thanks for playing, stay away from the kernel

[Spazturtle]

"depends on how much of a crap programmer you are."

So apparently all of the kernel maintainers are crap then since they have all written code with memory bugs in it.

[Mike 125]

"The tragedy is that the sum total of knowledge, experience, and perspective acquired over decades of being highly productive in a complex environment is not bound to a particular language."

...

"It is extremely difficult to not only preserve but make good use of such valuable perspectives when the ground rules change. Corporate culture is exceptionally bad at this"

e.g. 99% of embedded job requirement specs:

1) C/C++

errr.. that's it.

[Howard Sway]

Upgrading Linux with Rust looks like a new challenge

It shouldn't do. This brings back distinct memories of a company I knew of a couple of decades ago who decided that they'd have to rewrite all their successful C++ applications in Java in order to keep up with the new cool kids in their new web application world. Hey, the languages looked almost the same, but Java was "memory safe", so had to be even better! How hard could it be?

It turned out to be disastrous. Features their programmers were used to were missing from the new language, and the whole application design paradigm needed to be rethought from scratch. Add in the fact that the rewritten application was of course the first project undertaken in the new language, with all the mistakes and bad design decisions that entailed, and they were left with an error plagued, badly performing mess of an application that failed in the marketplace, as did the company soon after.

[hammarbtyp]

When I was in my 20's, I learned Erlang

For about 5 years after that I was a Erlang Fanboy, telling everyone they should use it, and being rude to those that didn't

I am still a fan of Erlang, but 20 years on., my opinions are more nuanced

I understand that now the people who refused my advice, were not stupid or stuck in the mud. They had there own issues to deal with which basically involved getting the next release out. Probably they had there own fanboy moments and now knew better. They appreciated the arguments, but also understood the challenges, which as someone who was still relatively inexperienced didn't

Its like when your kids accuse you of the same thing you did to your parents.

Erlang is still going strong, but had to find its own niche. Rust is the same. There is always this search for the holy grail of languages which will do everything. Rust is just a long line of these.

If you stick around long enough, the world comes back to the same point. Unfortunately it also means the same mistakes keep being made

[Mage]

re: Erlang and other languages

Before Linux project was even started we had Modula-2, Ada, C++, Oberon and Occam.

Obviously we also had other languages, but standard Pascal was only good for teaching. Lisp, Prolog, Smalltalk, Coral 66, Fortran, Basic, Forth, Comal, Objective-C, Eiffel, Erlang, Chill all existed long before Linux, though most of those are not suitable for writing a kernel.

https://en.wikipedia.org/wiki/Timeline_of_programming_languages

I've written several RTOS and designed two languages as well as a pseudo language parsed for the Lego Mindstorms (Ver 1). I did a platform game engine in plain x86 Modula-2 that performed CGA, EGA, Hercules and VGA graphics via run-time assignment of virtual functions for the drivers that performed as well as or better than Assembler coded platform games on DOS. Only disk access was single threaded and it was the only DOS API calls. Drivers in Modula-2 for keyboard, graphics, PWM sound on PC beeper, soundblaster and Midi etc.

C was a lazy choice. It made sense for UNIX, but that was started in 1969 and released publicly TWENTY YEARS (1973) before the Linux Kernel was released.

The problem isn't choice of language, though there are safer ones than C, but the programmers. I've seen "C++" that might as well be C.

[Anonymous Coward]

Re: re: Erlang and other languages

> released publicly TWENTY YEARS (1973) before the Linux Kernel was released.

Which gave it plenty of time to have multiple tried and tested[1] implementations across assorted platforms, applied to a wide variety of domains.

Oh, and to have a published specification that took into account all of the above.

Meanwhile, certain much newer languages...

[1] btw, "tried and tested" does not mean "perfect" but "known and characterised, functional within these bounds"

[that one in the corner]

Re: re: Erlang and other languages

> C was a lazy choice

For Linux, C was a very sensible and pragmatic choice.

Now, if you (or anyone else) had been able to tell Linus how large Linux was going to grow, *and* given him the money to buy the compiler and kit to let him start with the better choice for his personal project that wasn't going to be a big and professional OS like Minix...

[James Anderson]

Re: re: Erlang and other languages

C was the sensible choice. Given that bcc was free and open sourced. Practically the only language that had a mature, free and open sourced set of tooling.

If Linus had required contributors to buy a compiler licence from Borland or whoever LINUX would not even have made the “historical footnote” stage.

The open source community culture around LINUX was and is the key to its success.

[Anonymous Coward]

still not got rid of this bad writer?

[trevorde]

Don't need this newfangled technology

Rust is just molly coddling this young generation of lazy whipper snappers with all this fancy memory management and such. If Linus thought it would be a problem, he would've invented something better before he started kernel development. Now get off my lawn!

[JoeCool]

Very thoughtfull opinion, with only two fatal flaws

#1) The conception of the foundational technical problem

C is an extraordinarily good fit for quickly evolving, resource constrained environments. It is less so where robustness, security, and maintainability are coupled with frankly overpowered hardware and a massively shared environment.

Security is a function of bugginess; fewer bugs means fewer Security bugs. Ergo Security is a function of code base maturity, and the code development process.

The language is not relevant to that. Switching C to Rust may resolve one class of bugs, but if those have already been eliminated, such a Language shift is almost guaranteed to increase bugginess, including Security bugs,

#2) The Conclusion

You state a human problem of intellectual "stubborness" ( which I would point out is sometimes a critical factor in scientific advancement ) ...

If you're a Gandalf-grade C wizard, putting on novice's robes to learn new spells might seem incompatible with one's dignity. One's wily subconscious will find plenty of reasons to make this so.

And you almost have the right answer

The tragedy is that the sum total of knowledge, experience, and perspective acquired over decades of being highly productive in a complex environment is not bound to a particular language. Techniques vary, but the underlying problems and frameworks of thought do not

A Gandalf wizard is a wizard of productivity; C is simply the choosen tool that dictates the techniques and level of high productivity.

If a better tool came along, I think a lot of Wizards would be willing and able to pick up that tool.

But to echo a Stroustrup comment on Java - if it offers nothing new, no better way of expressing what you want to design, then it is of no interest.

The Correct conclusion is that : for all of the human, pride-based motivations that the article develops so well, if Linux becomes a Rust program (and is still Linux ) it will not be substantially for technical reasons, but for human ones.

[You aint sin me, roit]

Re: Very thoughtfull opinion, with only two fatal flaws

A Gandalf level C wizard will have progressed beyond the stage of "It's cool that I know C so well I can condense 20 lines of code into a single unintelligible line that takes a maintainer a day to attempt to unpack".

Maintainability and readability are fundamental to good code. If this is not reflected in your code then you aren't a good programmer let alone a wizard.

See also: "Not documented, not done"

[Fruit and Nutcase]

Cures for constipation

the cultural constipation of proprietary options loosening and becoming dislodged by the tonic of Torvalds

Prune juice

Syrup of figs

Tonic of Torvalds

[Anonymous Coward]

Some people just can't do it.

Reading the comments and how defensive they are, akin to zealots defending their religion or their 1956 Ford pickup from an EV, I can only conclude people are nervous of Rust, or more importantly not-C.

Rust is just harder to write, and that's still an understatement. It's not afraid to take your code that is logically sound and "fail" it in big red ink and tell you to start over. I think that scares the old guard of developers who lack the ability to deal with the loss of the full control C lets them have. If your career was built on that, change is terrifying. I get it.

However, whether it's Rust or something else, C is likely to be superceded soon. Times change and we evolve to meet it. You will too.

[An_Old_Dog]

Loss of Full Control

As a programmer who has seen auto-magic things fail in subtle ways, I get a nearly-overwhelming desire to take a mace and smack it upside the head of any-and-all dipshits who tell me/insist that I should "just trust the magic" without knowing how the magic works.

It's one reason I've not allowed myself to make or puchase a mace.

I can trust magic I understand. I won't trust black-box magic, and will avoid it if I can reasonably do so.

[Bent Metal]

Re: Loss of Full Control

Never trust anything that can think for itself if you can't see where it keeps its brain

-- Arthur Weasley

[Abominator]

Re: Loss of Full Control

As someone who flor good reason has to fiddle with and create custom allocators in C++, I went looking through the Rust code base and lo and behold...all that shit is unsafe as fuck.

[matjaggard]

Re: Loss of Full Control

You're wrong in your assumptions about loss of control and magic. That's not how Rust works. You are in full control, there's no magic* but you are limited to only doing things that are provably safe.

*Actually all languages have magic. C will magically handle passing arguments to functions using registers and putting things on the stack where needed. Library functions are magic until you read the code but mostly we don't, we rely on a human description.

[Justthefacts]

Re: Loss of Full Control

But memory-safe, does not mean correct. The problem is that focusing on memory-safeness, and obfuscating the language to ensure it, can *reduce* the overall probability of correctness.

If memory safety is 30% of bugs, but the Rust obfuscation doubles the complexity of what you write (and that’s conservative), and more lines of codes means more errors…..then you are left with 2x70% errors = worse code.

You know what would really strengthen the Rust kernel guys case? Since they are using something that contains formal verification by design, and re-implementing known-production-quality kernel modules from C to Rust….hacking the Rust compiler around a bit should allow you to use its formal methods identify a list of cases where the Rust produces different output from the C. And then we could check whether it is the original C code in error, or the new Rust module.

This would be actually *useful*. Their claim is that there are more bugs in the C than the Rust. My claim is that they will find there are dozens of cases where the Rust gives different output to the C, of which only a couple will be memory-safety bugs in the C. The rest will be bugs of other types, very heavily skewed onto the Rust.

The irony of course, is that this will be done *whether the Rust advocates want it or not*. There are plenty of black hats, and it’s really a very obvious point. If you want to find a cheap and rich source of zero-day bugs, the obvious place to go is find a new mine of source code, where somebody has helpfully provided a handy line-by-line crib to point up security holes. The Rust modules absolutely *will* be used in that way. The Rust maintainers better hope they get there first, is what I’m saying.

[LionelB]

Re: Some people just can't do it.

C has been likely to have been superseded soon for the past five decades.

File under industrial-scale nuclear fusion and AGI.

[jake]

Re: Some people just can't do it.

"I think that scares the old guard of developers who lack the ability to deal with the loss of the full control C lets them have."

Scared? Nah. It's just ones and zeros.

That level of control is useful when coding things like drivers and kernels, and us old-guard know it. You youngsters will too, eventually.

[JoeCool]

you, my friend have a bright future in manglement

Never lose sight of the new shiny, Ever.

[A.P. Veening]

HR

No matter what HR may suspect, developers are human too.

Developers are, I have my suspicions about HR.

[AdamWill]

I've got one

"Linux kernel types can do that with their niece's Steam Deck and their astrophysicist brother-in-law's supercomputer galactic evolution model. Find another job like that."

Plastics engineer.

[Claptrap314]

Re: I've got one

A nice demonstration of the lack of critical thinking involved in writing the article.

[((,)*(*))~]

Programming languages are a bit more than coding languages

What goes on in your head when you are writing code, in whatever language, should be thinking straight about what code to write down.

It is a massive irrelevance, mere noise, (sometimes tiresome) what syntax you have to express the code in.

In my experience, one starts with wishful and vague thinking, and it is only when the code doesn't behave as expected,

in debugging, that forensic, rigorous thinking gets involved. You're lucky if it is just a crash. the guy who is unlucky is the one

who has to deal with downstream crashes, or mysterious misbehaviours at a higher level.

It may not be a popular idea, but after several decades as a programmer, I'm convinced that mastering exact ways of writing

specifications at various levels that are consistent with each other is actually programming. These things can actually be written down.

Fiddling about with whitespace or suchlike noise isn't programming, just coding.

(Never even seen a rust program. Maybe rust is actually The Solution?)

[A.P. Veening]

Re: Programming languages are a bit more than coding languages

It may not be a popular idea, but after several decades as a programmer, I'm convinced that mastering exact ways of writing specifications at various levels that are consistent with each other is actually programming. These things can actually be written down. Fiddling about with whitespace or suchlike noise isn't programming, just coding.

After several decades as a programmer, I completely agree.

[Matthew "The Worst Writer on the Internet" Saroff]

Screw this, Let's Rewrite Everything in Cobol

I just won troll of the day.

[that one in the corner]

Re: Screw this, Let's Rewrite Everything in Cobol

COBOL?

Nah, let's go for Forth.

And do it *properly*: define layers of meta-compiler words[1] and keep going until we run out of different ways we can express brackets/parentheses/braces[2]. Then the real meat of the code can be expressed compactly, no need for multi-page functions!

[1] bonus points for defining punctuation words in layer n, using them to create layer n+1 then re-defining them to start layer n+2, because then the code always familiar and neat.

[2] hey, you can get Unicode Forth now, so proper paired speech marks, in various languages, can be used. And those funny upside down pling and question mark. Maybe we can find some pictoglyphs that sort of look like left/right pairs to an English speaker. Ooh, ooh - smiley and frowny face emoticons! We can use a school bus to be a bag of values that is driven into wall to do a merge sort...

[nijam]

Re: Screw this, Let's Rewrite Everything in Cobol

> Nah, let's go for Forth.

Coward! APL is the answer.

[Matthew "The Worst Writer on the Internet" Saroff]

Re: Screw this, Let's Rewrite Everything in Cobol

I am an APL fanboi since the 1980s. I approve.

[An_Old_Dog]

Re: Screw this, Let's Rewrite Everything in Cobol

"Programming in C: like walking through a pitch-black machine shop.

Programming in Forth: like walking through a pitch-black machine shop, while the machines are running." -- a Forth-lover.

Forth is actually reasonable, if-and-only-if you overcome your inner urges to show people just how god-damned clever you are. Most people don't have sufficient self-discipline to succeed at this.

[LionelB]

Re: Screw this, Let's Rewrite Everything in Cobol

> ... if-and-only-if you overcome your inner urges to show people just how god-damned clever you are.

See also C++ template metaprogramming.

(I once wrote a template metaprogram for super-efficient arbitrary-length bitstrings. It was, if I may say so myself - and I do - extremely clever. QED.)

[eric.verhulst(Altreonic)]

... no straight thing was ever built. Not even a kernel?

Well, that's not really true. Our own embedded RTOS (now called VirtuosoNext) was formally developed from scratch using Leslie Lamport's TLA+. It has a history before it. VirtuosoNext (previous name was OpenComRTOS) was a 5th generation development. The previous 4th generation (that has roots in Hoare’s CSP) one made it into the open-source Zephyr (after WRS acquired the IP). The issue is less one of using C versus other languages (like ADA, Spark-ADA and RUST). The first domain to tackle is architecture. Monolithic vs. modular. With separation of concerns. That includes semantics. This is where a formal design delivers. The VirtuosoNext kernel is 5 to 10 smaller than it predecessors because it has no duplicated code. It's kBytes vs. Mbytes. It also supports fine-grain space partitioning with real-time fault recovery in microseconds. This is on the technical side. A major issue is market acceptance. Most software done is coding, not engineering. People reuse what they know and often this is what they learned at school. Economic factors are important. Most often people prefer a quick and dirty solution rather than one taking more time but then being more trustworthy. Hence the need for vast third-party libraries and tools (the lack thereof is why Modula-2 failed). We have looked several times to rewrite it in not-C, but in the end the reason to stick to C is the availability of compilers (for embedded targets).

[StrangerHereMyself]

Demand

Linux has shown me that there's an enormous demand for an open-source operating system people can tinker with and base products on without having to bother paying anyone or begging for the source code. Linux isn't the "best" operating system (far from it IMHO), but it's "good enough" for most purposes. That's why it can be found in everything that's connected to the internet, from a fridge to a supercomputer and everything in between.

I would've liked to have seen a future when Minix or some other microkernel operating system thrived, but due to Tanenbaum's ill fated decisions not to license his OS Linux took over that role.

The advantage of a microkernel is that its core system services are so small that they can effectively be written in assembler, making the overall system much faster and responsive.

[An_Old_Dog]

Microkernel OSes

The theoretical advantages of microkernel systems are largely nullified by the difficulty in getting them installed and running. The GNU/Mach/Hurd/-however-it's-properly-called ("By reference!" *ba-dump* *ksshh*) Debian version recommends you run it in Qemu, rather than on real hardware. As with Plan 9 / 9Front (not a microkernel design, just another voice-crying-in-the-wilderness OS) there's only an i386 version, no AMD64.

L4/Fiasco has no download site I can find (though as a non-Deutsch-reader, I may well have missed it).

"Anyone? Anyone? Anyone?"

[StrangerHereMyself]

Re: Microkernel OSes

There are many microkernel OS'es out there, many written by hobbyists. RedoxOS is one that comes to mind, MMURTL is another. I've seen Japanese graduate students write a microkernel OS. QNX is also still around and used in many products. Minix runs on every Intel processor as a stealth OS.

I believe HarmonyOS used in Huawei products is also a microkernel. Then there's Google's Zircon (used in Google Fuchsia, which could be the underpinning of future Android versions).

Microkernels aren't as visible as Windows or Linux but they are there.

[Abominator]

Used Rust a bit, still don't like it. A bit of a stinky language. And the fucking positivity of the zealots. To me that's a warning sign.

[Jim 59]

Well written article.

[squizzler]

Linux is dying, lets work on successors

Meanwhile in other news Redox 0.9 has just been released. Not only is that written in Rust, but you get a microkernel, Rust based Cosmic desktop and ease of porting Linux drivers and so forth.) Redox is not the only game in town, we also have Theseus and HarveyOS/r9

The failure of Rust in Linux is a good thing that enables creative destruction. It creates new niches and frees up Rust programmers for these new systems, many with features that could not be back-ported easily. And if you are a Rust programmer, wouldn't you rather work on a younger code base that allows you to grasp the bigger picture rather than sprawling Linux?

The greatest empires at the height of their powers will have already started the rot that will bring them down and operating systems are probably no exception. Linux and its surrounding scene are so bloated there is room for something more manageable as this publication's Liam Proven has noted many times.

[An_Old_Dog]

Re: Linux is dying, lets work on successors

Starting anew on a smaller -- and hopefully, more-comprehensible -- codebase will probably raise coder enjoyment vs Linux, yet fail in an important way.

What's the point in solving the same problems yet again?

We need to find an effective way of extracting the valuable lessons learned in the construction and evolution of previously-written software in order to not waste valuable time, and to raise the state of the art.

[An_Old_Dog]

Empires at their Height, Bloated Linux Codebase, etc

So you start again, small. How do you eliminate the Katamari Damashi effect which turns your formerly-small operating system into a huge, agglutinated ball of features and code?

Everybody has just one tiny bit they want to add. "Would you like a mint? They're wafer-thin ..."

[Claptrap314]

Is this the horse for this course?

I spent eight years writing in assembler (x86 & ppc) & really hating it on an hourly basis. I did so because that was the only was to solve the problems my employers wanted me to solve.

C solves the problems that OS developers need to solve, and it does a decent job. (It would do a much better job if it exposed the C & O bits).

Rust solves ONE problem (more-or-less) that C isn't so great at. (however, if those C & O bits were exposed, it would directly address a lot of it).

Does rust actually do a better job than C for the kernel as a whole? Or is does it solve a problem that a certain class of maintainers want solved? Namely, they don't want to have to be careful about memory themselves.

I've not followed this closely, but I seem to recall the promise that Rust in Linux was not going to requires the core devs to convert. That promise is massively broken.

Good riddance.

[Phil Lord]

Re: Is this the horse for this course?

It isn't going to require core developers to convert. But as a Rust API is likely to have less undefined behaviour and certainly will place different requirements on the users and implementors of that API, it is going to require more clarity about the way things work.

When ever you rewrite something, try to add a new language or just have a new developer come on board, you are going to uncover corner cases, or unwritten assumptions that you didn't know or care that you were making. I think that this is all that happened here. Whether core developers want to engage with that process or not will depend on how much the value the inclusion of Rust. Some just won't like it; otherwise might be more worried about something else.

It's hardly surprising that in this process there are going to be some personal falling out.

[Anonymous Coward]

Although it was like pulling teeth at the time, getting Slackware up and running back in the 90's was tremendously instructive. I'm frequently dismayed at how ignorant today's kiddlywinks are at what is going on under the hood of an OS.

[Fido]

The difficulty I see is that more bugs are likely when simultaneously using two languages in the kernel compared to using only one. Thus C mixed with Rust will have more security issues than only C, just because of the mixing, even if Rust might be better.

I also find it surprising that Redox OS has been in development for 9 years and still not available for production use on any cloud platform, but that's a different story.

[usariocalve]

From what I understand, the dispute is actually about work - specifically, who maintains the rust-side interfaces.

In the specific case of the filesystem the Rust dev wants the current maintainer to maintain the interface, and the current maintainer is all "why should I take on more work (integration, testing, maintenance) just so you can use your shiny new toy."

To me, that's a completely valid question.

[Herby]

Once told to me

"You can write FORTRAN in any language". This truism can be related to many languages. It just goes to show that everything old is new again.

That being said, the "safe" features of any language can/will be bypassed to "make it work". If this happens we come back to the starting point, and something even newer will be put forth.

Some languages survive for long times. FORTRAN is but one of them. I'll leave the list of others for someone else to generate.