Microsoft is busy rewriting core Windows code in memory-safe Rust Microsoft is rewriting core Windows libraries in the Rust programming language, and the more memory-safe code is already reaching developers. David "dwizzle" Weston, director of OS security for Windows, announced the arrival of Rust in the operating system's kernel at BlueHat IL 2023 in Tel Aviv, Israel, last month. "You will …
"Beyond the presumed safety improvement, performance is said to be 5 to 15 percent faster for Shaping (substituting) glyphs with OTLS (OpenType Library Services). That's all available to developers Hackers now." ... I would be so happy if my comment is total wrong but these days I'm getting malware deliveries every hour, every day now.
I'm thinking there's been a sea change at Microsoft.
For several decades most of MS's intellectual property has come by absorption from outside products and projects.
In the last few years there has been an apparent openness to sharing and be participatory rather than controlling.
I will welcome this while still harboring a sense that I'd better be checking the rear-view mirror fairly frequently.
Well I still have to work hard when I want to buy a new laptop to make sure it will install Linux. I need to make sure I can format c: the virus and put Linux on. And I don't get a refund of the M$ tax. I'm sorry, but IMHO they always have been, are, and always will be cunts.
Oh god, better get standardising Rust before the scope of a variable defined in a simple for loop(*) becomes code-breakingly ambiguous.
(*) One of the more egregious of many ways Microsoft tried to make C/C++ better in the bad old days
that has always driven me crazy. WTF was MS thinking to put gaming software on and by default turned on and Enterprise OS. Its like, they thought - hmm this is not exploitable enough, lets put something on these to make businesses IT staff scream.
W11 is the worst so far - why do the Pro / Enterprise versions need all the same pointless shit as they inflict on home users? Business deployments would be a lot easier if a load of time didn't have to go into working out how to de-crap Windows as much as possible as part of the deployment.
I haven't worked on deploying Windows 11. Partly because my employer isn't deploying yet (although it *is* coming), and partly because it doesn't run on Mac bare metal.. Previously, my responsibility was taking our existing Windows image and adapting it for, and deploying it to, Macs (for dual booting). Long term, we *may* deploy as a VM running on Parallels. But long term, we are switching (where possible) from on prem Windows machines to Azure, so I suspect any Mac user that needs a PC will either have to have a PC laptop, or will get a VM running on Azure.
If you run it on Paralleks on an ARM Mac, I think you'll have to use the ARM version of Windows which is probably going to be even more "fun" - I've not tried it, but we all know what new-ish products from Microsoft tend to be like for a number of years...
If you run it on Paralleks on an ARM Mac, I think you'll have to use the ARM version of Windows which is probably going to be even more "fun"
I run Parallels on both Intel and Apple Silicon - yes, you *do* have to use the ARM version of Windows on Apple Silicon (the Rosetta2 layer obviously doesn't work on virtualisation). As far as I can see, there's very little difference between the ARM and Intel Windows (apart from the software available of course).
On Server 2016 the XBox services were set to start by default too! Who on earth thought that was good design is a mystery - even by MS standards it's ridiculous. One of the first jobs when setting up a new VM with 2016 was disiabling those services. So far as I recall 2019 has them disabled by default - and they aren't there at all on 2022.
As the person led the charge to suppress games (actually my team had already done some of that), Take A Tour, Movie Maker and lots of other crap from Server 2003, and rebrand its OOB, it drives me nuts too. Until then it was basically XP with a different process scheduler. Shout-out to Bill Veghte for insisting we do it.
"Rewriting Windows in Rust probably isn't going to happen anytime soon," said Weston, "so while we love Rust, we need a strategy that also includes securing more of our native code."
And once it is written, they will be spending the rest of their time writing rust-centric bindings against other C (or C++ or bindgen sys-*) middleware. It will be endless.
Personally I feel that building safety upon C is still the way to go. I made an attempt to prove this during my PhD (as a semi related distraction...) and the results of one approach (basically tombstones on steroids) have been very positive.
https://www.thamessoftware.co.uk/forge?repo=stent
(This simple test summarizes the idea: https://www.thamessoftware.co.uk/forge?repo=stent&path=src/tests/dangling_ref_copy.c)
You can see it in use (fairly brutally) in a "safe" websocket proxy I challenged myself to write in a weekend here: https://www.thamessoftware.co.uk/forge?repo=wsstent&path=src/wsstent/Client.c
However, it will be impossible to get developers to uptake something like libstent. A new language always seems more exciting even though it drags in the same old problems of bindings. I am just not convinced by Rust, even though it does seem to be seeing some success currently in kernels where package manager bandages to the bindings problem (crates.io) are less available.
I believe this is feasible, but only at the cost of performance. And I can tell you right now most developers are unwilling to trade in even the tiniest amount of performance for safety.
That's why Rust was invented. It's basically a C compiler which disallows unsafe code. If you write dangerous code it simply refuses to compile it. Under the covers it generates the same code as a C compiler.
Performance is obviously an issue on both sides. For example Rust's vectors check memory access *each* time an element is indexed.
With something like stent, all of these expensive checks are in the debug build but then stripped out in the release build. Yes, dynamic arrays will change (and checking each time is safer) but the index out of bounds should have been encountered during debugging and fixed before stripping in release and in many situations, you don't want that overhead anymore.
> Assuming you can catch malicious input in debug and strip protection for production is a pretty dangerous assumption isn't it?
You can keep the safety in if the performance hit is acceptable. For certain network facing servers I do recommend that.
> Why bother with the memory production at all if you can safely assume malicious input can be successfully and exhaustively tested against in development?
Being able to test memory properly in development (i.e via Valgrind, Asan) is a *big* industry. It feels wrong to trivialize it like this.
You are also perhaps assuming that Rust isn't still calling into unsafe code (the entire base operating system). You can't test this any better than i.e libstent.
But yes, Rust bindings developers *could* (and probably should) leverage libstent underneath. But then it is even more layers.
Oh, another one!
Correctness FIRST.
Runtime checks on arrays/vectors are done to prevent Array OUT OF BOUNDS bugs, you know, those rarely heard of, once in a blue moon mistakes that have no impact…
If you’re indexing an array using a variable that you’ve set at runtime, you have to check the array size at runtime, not compile time.
Yes, please give me a “performance hit” and stop me causing a great big security incident
It’s actually super obvious now it’s been pointed out, hey?
Absolutely. The question is if you really need to test it at runtime *and* in the release build.
For many things (i.e a software rasterizer). I think not. If the pixels fit in the debug build, they will be guaranteed to fit in the release build (unless solar radiation and all that intervene).
Just to clarify. libstent tests array access at runtime in the debug profile.
> Correctness FIRST
This may sound pedantic but checking an array each access isn't "correctness". It *is* still important but is classified as "safety" instead.
Yes, Rust does encourage correctness; I find C++ (even the modern stuff) to be lacking in this department. However correctness with C is possible and safety then tends to follow from this. For example OpenBSD has fantastic code quality and safety due to an attempt at "correctness".
libstent is merely another tool in the box (alongside asan, valgrind) to help write correct code. The difference is it is 100% portable because it sits ontop of ANSI C; it even works on a crusty megadrive compiler.
Yes, pedantic and also wrong.
Show me the bit on your Venn diagram where safety falls outside correctness… that’s a bug
If the only argument is it’s inefficient, then come up with something better and we’ll use that. We’re not talking pen and paper vs computer, we’re talking one computer vs a slightly slower one….
I cannot see the position you’re taking, our perspectives are so different. This is credit card info, nudes, real people’s lives, not a fairy land of neat and tidy theory.
The definition is well presented here in a succinct and easy to digest manner:
https://www.educative.io/answers/what-is-correctness-in-software-engineering
In summary, "know" the algorithm is correct rather than bounds checking to retroactively catch errors in your code.
Both aid in safety but "correctness" is already very well defined (i.e in many published engineering journals) and does not need to be argued (and certainly not with me).
But as I mentioned, I am happy to strive for both in tools like libstent.
The problem is programmers are human. Smart, capable, professional humans who knew their algorithms were implemented perfectly, were wrong. Array bounds overflows occur, and companies get hacked.
It seems like you’re promoting your own library, that you wrote, which has not been audited, battle tested or accepted by anyone else. Recommending that global companies use your random tool vs Rust is arrogant and very very likely to be a worse or incomplete solution - please see: don’t roll your own crypto.
Submit a pull request to some major repositories, set up a website illustrating the use of your tool and when that’s accepted you’ll get global recognition and people will ditch Rust in favor of your superior solution. A YouTube video and a PHD does not put you anywhere near the Rust foundation in terms of capability, correctness or trustworthiness at this point
> Smart, capable, professional humans who knew their algorithms were implemented perfectly, were wrong. Array bounds overflows occur, and companies get hacked
Then the code is called, "not correct". Array bounds checking that panics (or throws an exception) can be "safe" but the code is still "not correct".
Put simply, would you call code that accesses an invalid index of an array correct?
But lets not dwell on this because as I mentioned, the terminology was possibly being a little pedantic.
> Recommending that global companies use your random tool vs Rust is arrogant
I recommend global companies use (and build upon) standard C rather than making and discarding new languages each decade - also see: don’t roll your own crypto.
Certainly valgrind, asan, pvs, my tool, etc, etc can help. In a decent engineering workflow (for any language) you would use a whole range of these kinds of tools. Arrogance would be thinking all the rest of these tools can be replaced rather than joining forces to make a stronger test suite.
> anywhere near the Rust foundation in terms of capability, correctness or trustworthiness at this point
It doesn't quite work like that. Far more commercial companies use i.e PVS studio [https://pvs-studio.com/en/pvs-studio/] than Rust and I imagine you haven't even heard of that? (Apologies if you have, especially if code safety (incl safety critical) is your area, but I am pretty sure you can agree though that most have not!)
This isn't youtuber style popularity, this is looking at tools to be used within the industry that can leverage existing code, homogenous codebases rather than the fantasy of rewriting everything again in the current popular language of the day. Rust even has an active marketing department and budget and this is not something that I will be pursuing. It wouldn't even make sense for a small memory testing tool.
You are absolutely right though, putting this in more publicly visible projects is the way forward. This just should have been done decades ago so I shouldn't even have had to write something like this in the first place.
Bound checking isn't even really a performance hit on modern processors. Before using Rust I'd gotten in the habit of preceding all C++ vector accesses with assert statements to check bounds. The idea being in a release build those would be off and not affect performance.
The thing is turning those assertions off rarely made any appreciable improvement to performance, and on a surprisingly number of occasions it made tight loops slower. (When I investigated this it tended to be attributable to processor branch prediction rather than the compiler using my range check to simplify the code).
Okay in a super-tight loop the extra instructions can matter a lot, but then the Rust compiler has much better chance of eliminating the check anyway, and there are ways of coaxing it to do so when it doesn't. Writing clearer code is a good starting point.
Not a secure solution. Ok, with 100% testing, and fuzzing, and wotnot, the debug build will trigger an error on a vulnerability. But with always-on checking (for free with Rust) means that even an imperfectly-tested app will quit at runtime, instead of allowing data leakage or arbitrary code execution. Denial-of-service, but not compromise.
C has had valgrind and friends for ages, to perform these checks. Heck, I wrote a horror that overrode malloc() and new() in the late 90's that used the Windows allocator (originally PharLap286/386 on DOS) to detect array overruns and use-after-free.
> Heck, I wrote a horror that overrode malloc() and new() in the late 90's that used the Windows allocator (originally PharLap286/386 on DOS) to detect array overruns and use-after-free.
Its a good point. On platforms with mmap, mprotect functionality you can lock the pages. On DOS, DJGPP's default extender also does support a rudimentary mmap (pleasantly unexpectedly). However there are many platforms where Valgrind, Asan, mmap aren't supported. Older platforms (Plan 9, Playstation) and newer platforms (WebAssembly tooling like Emscripten was lacking asan for years). libstent is another tool for the memory safety collection, albeit a 100% portable tool.
Rust is good (a memory safe systems language took an embarrassingly long time to appear. The closest we has was Ada but that was far off; SPARK is improving things) but if C is made more safe, it might just be enough to avoid having to invest in writing your own FFI code to call into native system libraries (a process involving a lot of unsafe code!), or pulling in potential technical debt in terms of bindings from crates.io (and hoping their unsafe sections remain well maintained).
but if C is made more safe, it might just be enough to avoid having to invest in writing your own FFI code to call into native system libraries (a process involving a lot of unsafe code!), or pulling in potential technical debt in terms of bindings from crates.io (and hoping their unsafe sections remain well maintained).
I'm not sure how C could be made safe enough for that without becoming "Rust, but with less ergonomic syntax" but, supposing it can get part of the way there, it'd certainly make writing Rust FFI less unsafe, which would be nice.
(Rust FFI is unsafe because C doesn't provide enough information about its invariants for the Rust compiler to guarantee that your FFI code is memory-safe. The more information C code can offer to a tool like bindgen, the closer to safe Rust the resulting generated code can be.)
Agreed. Some aspects of FFI lose type safety. I think something like libstent can still be useful against this by having a primitive form of RTTI to allow casting from a void* to a i.e struct Employee*. Actually, this is better than RTTI because it doesn't require a vtable and can work on raw void*.
If you have time, please check out this libstent test as an example of how this range of type errors can be caught:
https://www.thamessoftware.co.uk/forge?repo=stent&path=src/tests/invalid_void_cast.c
I feel this would also be useful for idiomatic C GUI libraries where they pass around void* user data too liberally for my liking.
The overhead is non-trivial (albeit probably quite acceptable for GUI work) but stripping this out during release minimizes that and in some ways this use-case will get a fairly good coverage of potential invalid casts.
In short, if i.e Gtk+ was all completely written via a libstent centric C, it would flag so many potential bugs. This would be an easier task than a rewrite in Rust. But also the Rust bindings could benefit from this too.
But also the Rust bindings could benefit from this too.
GTK is a poor example for this because they already have their GIR (GObject Introspection) interface definition language, which provides enough memory safety information that PyGObject is able to dynamically load the GIR file and synthesize Python bindings at runtime.
My understanding is that gtk-rs is already also relying on GIR.
That is true. Though looking through some gtk-rs (the generated sys- binding component); there is a number of gpointers pointing to Rust "book keeping" data. This kind of userdata is potentially fiddly and could do with some more tracking.
I probably see most issues appear when Gtk is used from C directly (a combination of API updates and / or rotting codebases). For example see many warnings appear in stderr stating that some "widget is not a GtkScrollbar". I maintain an old GtkRadiant level editor tool and these warnings were rampant!
The Glib g_object_weak_ref is a very nice but very manual affair, relying on the programmer to increment / decrement the refcount as things move around. This could perhaps be a candidate for some tooling.
Since the libstent websocket proxy, I am finishing up an MS-DOS/OpenWatcom gameboy emulator using libstent but then will try to tackle a GUI library. Probably integrating libstent into FLTK ANSI C bindings. Small enough for one guy to maintain but may yield some actionable results.
Iterators in Rust are quite restricted compared to C++'s wild-west style (considerably unsafe) iterators. They are more akin to the foreach syntactic sugar in C++.
libstent also provides a safe foreach wrapper but honestly indexing is usually more useful/flexible and optimising compilers often generate the same machinery anyway. In libstent the foreach can decompose to pointer iteration in release mode. Pretty much the same as Rust and C++ (but safe in debug like MSVC's "safe iterators").
"Yes, dynamic arrays will change (and checking each time is safer)"
How 'safe' does it need to be inside the kernel though? When properly designed, kernel functions will always get valid pointers, so they do not need to be checked and re-checked over and over.
One could argue that going 25MPH on an open highway is "safe". But people do not LIKE going slow. People want AUTOBAHN speeds!
Same with computer performanced. Overly-unnecessary 'paranoid' pointer safety is another concern with using Rust (for me, behind garbage collection)
Exactly, people keep on making the same old memory safe VM languages like Java and .NET!
The evidence is there that these don't really work in the long run. Porting a i.e .NET VM to a new platform is a lot of work.
.Net was never meant to be cross platform so it's not surprising that it's not a great exercise in simplicity to retroactively port it to everything under the sun.
A better example, the Java VM probably is a lot of work too, but the point is its shifting cross platform concerns to the VM. So that work only needs doing once per platform.
You would have to compare the cumulative time porting all C based applications (or similar randomly sampled line counts at least) to different platforms vs porting the VM once and having applications suddenly work with no extra investment.
> You would have to compare the cumulative time porting all C based applications (or similar randomly sampled line counts at least)
> to different platforms vs porting the VM once and having applications suddenly work with no extra investment.
From a practical point of view, I only half agree. Looking at specific examples, porting Java or .NET to i.e web browsers. It took Unity3D 4 years to get .NET running well enough on the web compared to UE4 with just C++ (+ garbage collector). In the end they transpiled from bytecode to C++ (Burst). So this is a good example of large VMs being difficult and time consuming to port. Yes, .NET has loads of existing software that can now run on the .NET VM on the web, i.e Blazor, but it still lost early adopters of a platform 4 years. Emscripten as a C/C++ cross compiler has existed even longer before that.
This is also assuming that C (or C++) needs porting. 99% of it is just as portable as Java.
If we go back further to Dis (the forefather of JVM/.NET). The Inferno Dis platform isn't 64-bit safe after decades. Anything written for this via Limbo, is dead in the water. C (and to a lesser extent C++), avoids this by having individual vendor compilers. It distributes the risk of 1) a central VM and 2) complexity.
> .Net was never meant to be cross platform
Ironically many .NET VMs are easier to port to different platforms than the JDK. Less legacy baggage.
Plus Mono was designed to be cross platform and that is still a dog to port. It is more the sheer complexity of it all that I see as the problem. Check out this single file (warning, may freeze browser due to size).
https://github.com/dotnet/runtime/blob/main/src/coreclr/gc/gc.cpp
But this aside, I don't really feel there is any argument against C being the most portable technology today. I think where there is disagreement is if its lack of inbuilt safety is worth it. This is the angle that my libstent approach was trying to tackle.
There is the language though, and the API's. A lot of pure C code is of course portable, but when you add on the layers of complexity to make the code useful on modern operating systems that all suffers and you start having to port more and more applications.
Porting a (fairly well designed) VM is front loading all of that effort to a single, consolidated effort is my point, networking, console handling, UI, etc.
Of course it's moot for device drivers. Mono is of course a long running project, but it remained fairly niche until Microsoft took over it all.
I don't disagree with most of what you say in the reply though.
The problem with C is it allows you to make a lot of assumptions, which can come back to bite you when you port. I remember dealing with porting C code that hard-coded the size of an Int, for example. And many system and library calls vary from system to system, which complicates porting. If porting C were easy we wouldn't need abominations like autoconf that generate huge piles of #ifdefs.
On the surface this may seem to be the case but it is possibly a little more misleading than this. For example:
The sort of code you write using i.e Rust, Java, etc (lets call it "application" code) can be written just as portably with i.e. C (i.e no #ifdefs).
It is the low level code (lets call it "systems level" code) that needs a lot of #ifdefs. We only perceive this to be an issue caused by C because only C can typically really write this type of required code in the first place. Rust and Java delegate this "messy" part to C via jni, ffi and bindings. One example I can give is POSIX sockets vs Winsock (or Plan9 dial(3)). Rust can't use these core libraries directly so it calls into C which does then use #ifdefs to abstract between then both.
Yes, it gives C a bad name but if you have ever had the "pleasure" of writing bindings or maintaining them (including the *-sys libraries that bindgen churns out), you might likely prefer the perceived elegance of C and #ifdefs.
Yes, but isn't it fun to watch a monolithic software company come to a near-halt screeching tires and heading off at a 90 degree angle with foot to the floor?
(in my mind I see a tall 'over-compensation' building nearly toppling over in the process)
But give it to Micros~1 to re-re-re-re-write "New, Shiny" because "New, Shiny" regardless of common sense, actual improvement, or profitability.
My biggest concern is the same as it is for Linux kernel using Rust: garbage collection
(it does not belong in code ANYWHERE and is so often abused nowadays that mail readers and browsers left running will ultimately allocate EVERY FREE BYTE OF RAM, causing performance stutfers when you switch to something else, because 'hang onto RAM' for THAT is the ONLY application worth running on YOUR computer, at least in the devs' less than brilliant minds)
It is certainly interesting.
Honestly I was so disappointed by the WinRT APIs in C++. They completely lacked safety (raw pointers / references with dubious lifetimes everywhere (albeit not owning thankfully)). Throwing in a bunch of lambda callbacks, does not make a safe API.
However for bindings to other languages (Microsoft call these language projections for some reason), their Rust API is going to have to be butchered to allow extern C-style calls, let alone ABI stability.
C++ -> C++/clr -> C++/cx -> C++ (winrt) -> Rust... It looks like Microsoft isn't happy with their modern tech stack and keep changing. Hopefully they can settle on Rust for at least a while longer than the others.
My biggest concern is the same as it is for Linux kernel using Rust: garbage collection(it does not belong in code ANYWHERE and is so often abused nowadays that mail readers and browsers left running will ultimately allocate EVERY FREE BYTE OF RAM, causing performance stutfers when you switch to something else, because 'hang onto RAM' for THAT is the ONLY application worth running on YOUR computer, at least in the devs' less than brilliant minds)
Rust doesn't have a garbage collector and the only garbage collection it has is the same kind C++ has: an opt-in reference-counted pointer type (std::shared_ptr in C++). Ownership and borrowing is just C++-style RAII with reliable compile-time support for rejecting code that would create dangling pointers, no legacy array type that decays into pointers, and gating things like pointer arithmetic behind an unsafe keyword. (Something that can't be feasibly retrofitted onto C++ because of the vast swaths of existing code lacking the necessary information about the programmer's intent.)
What's more, the language is designed so that using reference counting is uglier than not using it, so programmers are nudged toward single ownership and architectures where it's hard for accidental data lifetime extension to lurk.
...OK, technically, Rust has two reference-counted pointer types. It splits std::shared_ptr into Rc (no atomics) and Arc (atomics) and makes it a compile-time error to send an Rc handle to another thread rather than switching use of atomics on or off based on whether libpthread was linked, so you've got more control than in C over whether you're taking the performance hit of using atomics.
(That's also not magic. Things which are safe to send to other threads implement the Send trait (interface), compound data types automatically implement Send if all their members do (but you can opt back out), and APIs for sending stuff to other threads impose a Send requirement on their arguments.)
Windows encompasses hundreds of millions of lines of code and rewriting those is Rust is not something that's feasible. However, they can get the most bang for the buck rewriting core Windows NT libraries in Rust, as they seem to be doing now. I also believe they will eliminate many, many bugs doing so, thousands of which may not have been discovered yet.
Rust is the standard for low-level programming henceforth. Although languages like Zig (which have a completely different but interesting take on memory management) are interesting, they will not supplant Rust or even compete with it.
"... I'm sure they're under pressure from their engineers to adopt Rust."
"If you're building an application today that's either performance critical or low-level, then Rust is a no-brainer at that point."
These two statements were together in the article. The first has been said about so many whiz-bang 'advance's for decades. The second has been demonstrated for very few of those 'advances', and then only gradually over time.
How do we distinguish irrational exuberance from provable facts warranting enthusiasm?
Computer technology is the greatest generator of slippery claims known to the human race. Why should we believe in this shard of the philosophers stone?
"If you're building an application today that's either performance critical or low-level, then Rust is a no-brainer at that point."
Here we go again.
Depends on the definition of low level. I do a lot of bare metal stuff where dynamic memory allocation are very much a no-no [1] and for clarity C is still best for those applications. For arrays, it really doesn't matter if they are dynamically allocated or not as an overrun is an overrun regardless of how an array was declared.
I highly commend everyone to number 5 of the The Ten Commandments for C Programmers reproduced here:
5. Thou shalt check the array bounds of all strings (indeed, all arrays), for surely where thou typest ``foo'' someone someday shall type ``supercalifragilisticexpialidocious''.
I have picked up code directly driving hardware where no error checks were performed (see number 6 at the above) written by supposedly excellent software engineers; ion one case I added a status word to the code and as each part of the initialisation was done successfully, I cleared the appropriate bit. If everything went ok I got a '0' at the end; more importantly, I could tell exactly where in the sequence things had gone wrong by finding the first non cleared bit.
Rust makes sense in certain things (courses for horses and all that) but in many applications, C will remain on top. It has its issues but they can all be dealt with.
1. On some occasions it is necessary to use malloc() to get the alignment correct. I had that requirement for a DMA descriptor setup.
> "Although there are fewer people [with extensive Rust experience], there's a lot of engineers who are interested in trying to learn it," he said. "And the sheer difficulty of writing code which is safe, it's an order of magnitude easier in Rust."
One of the things we should all fear is "lots of engineers who are interested in trying to learn it."
In the blue corner: 5 engineers each with 15+ years of C and C++ coding experience. In the red corner: 5 engineers interested in trying to learn the language no one heard of 2 years ago.
Which team did we want to put on the OS kernel, again?
If Microsoft had made C# compile natively from the start there would be much less of a need for a language like Rust. Almost all programs can run perfectly with Garbage Collection (GC). There's no need for deterministic memory management for the majority of use cases.
Although I love GC languages, the runtime dependency sucks and users will always choose a program that doesn't require one if they have the option.
I hate GCd applications, they always stutter at the wrong moment.
It's really annoying, and effectively impossible for the developers to avoid as they have no control over when the GC will come around.
There's also the fact GCd languages tend to encourage memory leaks. Telling developers "you don't need to worry about X" usually means they stop thinking about it, and hold onto a reference forever by accident.
After all, a GC can't release unless it can prove there are no references. It does however (usually) avoid fragmentation, which is useful.
I'm very glad Rust does not use a GC. RAII is provably superior for most uses.
However, Rust's runtime error checking is just useless. My application should not simply die if something would write outside bounds! Most errors are recoverable, give me a chance to recover.
"In particular, because I expect Microsoft to reuse the existing compiler, I hope that a side effect of this will be better PDB [Program Database] support,"
Yep, working on it:
* https://reviews.llvm.org/D148761
* https://reviews.llvm.org/D149367/new/
Disclaimer: I am a Microsoft employee, and those are my PRs...
... any chance you've got the 'thing thing' from Long Covid - you know, the err ... thing ... when your err ... thing ... seems to not work properly any more?
For nearly 18 months I thought I was in cognitive decline. Wasn't till this time last year that it finally started to improve, increasingly rapidly.
Don't feel bad - I'm 83 now & long retired but still have a deep love of all things computer, but all of these newer languages leave me scratching my head - the terminology has gone beyond my brain but still i can't resist peeking at these articles. Who would have thought in the late 1970's that this realm would become so huge nobody can keep abreast of it all ! Fascinating ! The world is moving so fast now, if civilization is still functioning in another 50 years it will be again majorly different. I'm sad I cannot be there, but i will be watching if it's possible !
One of the problems that most pre-retirement techies like me in their 50s are finding is that recruiters and some dopey HR depts still think it's 1992 and we can cover 75% of PC tech with just 2-3 people, they seem to have no idea that IT and tech is now so vast that the jack-of-all-trades people like me can barely scratch the surface, meanwhile those who specialise in 1 or 2 things have to be super careful their chosen speciality doesn't become obsolete too fast and they have time to skill up when the new things come long.
It's not 1988 anymore and "We're a long way from Kansas now Toto!", retirement can't get here fast enough for me so I can finally cut my loses and get off the treadmill.
There's (rightly) a lot of hype at the moment about rust and lots of people rushing to learn and use it, but don't forget rust is not an application development language; it's just a memory-safe alternative to C or Assembler for use when building operating systems and ultra-high-performance code.
If you want to build web sites or mobile apps you're well serviced by some great languages nowadays, but don't be dragged into the rust bandwagon.
Correctness FIRST. Performance is nice, please see Spectre and the fact that CPUs are a biiit faster than they were in the 80s.
Runtime checks on arrays are done to prevent Array OUT OF BOUNDS bugs, you know, those rarely heard of, once in a blue moon mistakes that have no impact…
Cool complaint bro
Computers (CPUs anyway) are 4-5 orders of magnitude faster than they were in the 1980's. A baseline of 1MHz / 8bit has become 4GHz / 64bit / 4 cores, even in a phone.
System RAM is perhaps 1-2 orders faster, if you include wider bus widths. Hence multiple cache tiers required inside the CPU.
And yet, somehow we seem to have squandered all this. I suppose much of it has gone in pretty graphics, animation and video playback, and thick layers of libraries to support all that. I would trade some of that for safety and security any day.
You don't *need* all that performance for a workable graphical UI. The Macintosh LC II had a 16/32-bit 16MHz processor (32 bit internally but 16 bit system bus), 4MB RAM (expandable to 10MB), and Color Quickdraw in 256KB of ROM. And I bet it booted up more quickly than a modern PC.
> The Macintosh LC II had a 16/32-bit 16MHz processor.... And I bet it booted up more quickly than a modern PC.
No, because SSD.
(My 2015 PC booted VERY much faster when SSDed. I recently setup a 2018 mini-PC for Dad and was flumoxed how slow it was to semi-boot and then digest the Registry, because rotating rust. Lesser but similar speed changes in all applications.)
If all you want is runtime array-bounds checking, several C compilers have compile options to insert that. Intel ICC supports this, for example. It’s also perfectly possible for compiler to insert runtime *pointer* bounds checking within a program unit, which drops out to a defined exception handler. Contrary to a lot of erroneous statements, there is *nothing whatever* in either ANSI or K&R which prevents this, and in fact several compilers do support it. It’s simply compiler-specific behaviour. You do however need to RTFM of the compiler, which few people bother to do.
What is true, is that both GCC and clang have refused to implement this, for religious reasons. In fact, way back GCC used to support runtime array bounds checking option, and they *removed it*.
None of this is any reason to invent a whole new language. There may be other reasons to use Rust, but every reason I’ve been given so far, borrow-check included, could be and *is* implemented by some compiler, on top of ANSI C, without rip-and-replacing the language.
Bingo.
Few people RTFM. So give them Rust, pay them 30k and my credit card info will be waaaay safer than doing the same in C.
Refute that.
They won’t be using unsafe Rust if they haven’t RTFM, will they. They will be using unsafe C UNTIL they RTFM.
Welcome to the modern world, programming is not done by expert ninjas, we ain’t fixing that but we can make their nunchucks out of foam, so let’s do that.
I would also like foam nunchucks as I am accident prone, sleep deprived and distracted, from time to time
None of this is any reason to invent a whole new language. There may be other reasons to use Rust, but every reason I’ve been given so far, borrow-check included, could be and *is* implemented by some compiler, on top of ANSI C, without rip-and-replacing the language.
To me at least, Rust's greatest strength is unavoidably tied to it being a new language... the requirement that code be annotated with richer information about what invariants the compiler must uphold.
Retrofitting this has been tried with C and C++ on various occasions in the form of tools like splint, but they failed to get uptake. In Rust, they manage to get people to accept that by clean-rooming a syntax for native code that makes it less onerous than if retrofitted onto C or C++, and by making it part of the process of writing bindings, akin to TypeScript definition files.
That aside, C would also look very different if you extended it to have as rich a type system as Rust. Rust is closer to being an ML-family language disguised in a coat of C++ syntax, and one of its biggest strengths is how many invariants you can encode in the type system, to be proven at compile time.
Dijkstra was actually arguing for stronger type systems and more formal proving when he wrote this:
Program testing can be a very effective way to show the presence of bugs, but it is hopelessly inadequate for showing their absence.
It’s not actually mandatory - if you would prefer to do a particular bit of array indexing without the bounds check in Rust (presumably because you already know you are within bounds and are in a performance-critical bit of the code), you can. Also, if you iterate over the array, the bounds check only happens once at the top of the loop, same as the “i < n” check in a C for loop. It’s really not an unreasonable approach.
1. It's recommended that you use the iterator APIs where feasible. They only need to check the array length once (same as with an optimal C for loop) and the API is designed so that going out of bounds is prevented at compile time without additional checks.
2. ReadySet wrote a blog post named How much does Rust's bounds checking actually cost? where they tried compiling their database engine with a version of the Rust toolchain that had been patched to remove the bounds checks and found no significant difference, indicating that, at least in their codebase, CPU branch prediction and memory latency were making bounds checks zero-cost.
3. Rust's assert remains present in release builds (it's debug_assert that's equivalent to C's ASSERT) and, if you assert the length of the array at the beginning of the relevant code, it serves as a hint to the optimizer to strip out the bounds checks in the indexing operations. This effect can also be achieved by iterating in reverse.
4. If you profile your code and actually find bounds checks in a hot loop being significant, there's the get_unchecked and get_unchecked_mut methods which must be called inside an unsafe block... but remember what Knuth said about premature optimization:
There is no doubt that the grail of efficiency leads to abuse. Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil.Yet we should not pass up our opportunities in that critical 3%. A good programmer will not be lulled into complacency by such reasoning, he will be wise to look carefully at the critical code; but only after that code has been identified.
I agree. Rust is only needed for use cases that require deterministic memory management or high-performance under tight latency constraints.
OTOH using languages like Rust can save huge amounts of electrical power, since its much more efficient than say JavaScript or Python. And in this day and age of climate hysteria this a welcome trait.
It's entirely possible that MS will eventually begin moving things like DirectX over to Rust, especially if they also introduce Rust into the guts of the Xbox. Under those scenarios you will probably see more uptake amongst game developers.
That would be a major change, but likely a way off yet.
That doesn't sound like a logical argument at all. Game developers don't care if the underlying firmware is Rust of C++. They have other concerns, like how much effort is required to port a AAA title, or how much users does the platform have. XBox is clearly lagging in the latter as PlayStation is outselling it 3 to 1.
Those with a very long memory will remember that Microsoft introduced FAR pointers to C to enable the porting of Unix (Xenix) to the intel 80286 processor. before the i386 Intel memory was segmented into 64k chunks, and far* allowed C programs to use segment registers to the extend addressing to the massive 20-bit address space of 8086 and 22-bit 286
The industry basically responded with "that's a good idea.. we'll have that". It wasn't until the fight for survival with IBM and OS/2 that Microsoft took a proprietary turn. Modern MS knows that "embrace, extend, and extinguish" wouldn't work even if it wanted it. We'll know that MS is really serious about Rust when:
1) It adds Rust tooling to Visual Studio
2) It adds Rust interop to .NET for safe AOT compilation
A minor correction. 286 had 24-bit addressing with 16MB memory (https://en.wikipedia.org/wiki/Intel_80286). But thank you for the reminder.
Embrace, Extend, Extinguish ended with Windows Mobile/Phone, and I think having had the trauma of that, MS have moved on with the departure of Steve Ballmer, who must have been around the OS/2, Win NT transition, and departed with the Nokia merger.
Had OS/2 2.1 and 3
Also Windows Phone 7, 8 and 10 so equally backed the wrong horse every time.
The 286 processor supported 24-bit addressing in protected mode, but since it was incompatible with the real mode (8086), few operating systems supported it (OS/2 had a horrible kludge that reset the processor to switch from protected to real mode programs). Through experimentation, Microsoft found a 286 could address 64 segments in real mode (22-bit addressing) - Windows 3 was able to address 4Mb of memory without the very-slow switch from protected mode, beating OS/2.
It's not meant to be the final stage. It's intended to get you from C that compiles to semantically equivalent Rust that compiles so you can transplant your test suite and then start incrementally rewriting it to take advantage of Rust's stronger type system without a big gap where you can't be running your tests.
With regards to chatgpt, it's not good enough no. It frequently gets concepts and code wrong. I find myself correcting it upwards of 90% of the time. When I ask a question but I'm the one that has to correct it, it's not a good sign, as I might as well not have asked to begin with. I also see a lot of new people come in with concepts that they learned from chatgpt, and I (and others) have to correct them, because chatgpt got it completely wrong, or misled them in a subtle way.
Chatgpt can be useful, but there are limits to it and one has to know the limits.
With regards to GitHub copilot, I've never used it. But I would expect it to be much better given that its entire purpose is to get it right in the first place (I don't remember, but I'm not sure that at the time of posting copilot even supports rust or not)
Yes, they're too different structurally to port automatically. At the very least you'd need a fair amount of manual porting to get it to compile.
Having said that the sheer amount of C/C++ code needing to be ported would likely benefit from semi-automatic porting using AI.
It is silly to use Rust for memory safety since it is easy to make C or C++ perfectly memory safe yourself.
And all you do by using Rust is to expose the memory safety methodology to all the hackers, who then know exactly what the Rust weaknesses are.
When you use C or C++ and do your own memory safety, then no hacked knows what you did.
There is no way to use Rust for everything either, since sometimes you do have to do unsafe things with memory, between the operating system and the hardware.
Actually, rust can write those unsafe portions too. Rust allows you to also write unsafe code, but that unsafe code still has a lot of the same checks (e.g. borrow checking), the only thing that isn't checked are raw pointers.
There's already entire experimental operating systems and kernels built entirely in rust. So yeah, it actually can do all of that
What you're saying is demonstrably wrong. It's extremely difficult to get memory management right in very large C/C++ programs (if not impossible). We're seeing this demonstrated almost on a daily basis.
I'd say that you're illustrating one of the biggest problems: overconfidence of many C/C++ programmers in their own ability. THAT is why we need something like Rust.
It took a while for the WTF moment to manifest, but MS is rewriting Win32 GDI in this heavily oxidized programming tool??? Wow! It seems good ole GDI is still the way to go if you need a native Windows GUI*, huh? Time to see where the Petzold and Richter Win32 programming books are. (Largely outdated now, but those were probably the most useful books on programming I have ever encountered: Charles Petzold on the GDI and Jeffrey Richter on the Win32 API).
* What are my choices? QT - nope! WPF - nope! UWP- NOPE!!!!, GTK - maybe (if I can find the time to learn the framework). I've settled on embedding GNU Libmicrohttpd into apps and dishing out HTML if I need a GUI — seems to be a sensible choice.
It depends. In the era of Qt 3.x and GTK+ 2.x, it was a good all-round choice.
Now, with GTK 3 and 4 apps being increasingly alien on non-GTK desktops as non-negotiable GNOME-isms creep into the toolkit, it depends on whether I care more about making something feel Windows-native or KDE-native (I'm a KDE user).
Here's hoping the rumblings I remember hearing about a revival of the wxQt backend bear fruit.
Time to see where the Petzold and Richter Win32 programming books are. (Largely outdated now, but those were probably the most useful books on programming I have ever encountered: Charles Petzold on the GDI and Jeffrey Richter on the Win32 API).
Thanks for those recommendations. Back when I was young, I jumped from QBasic to Visual Basic 6 to wxPython to Linux, bypassing the actual Win32 APIs and, now that I'm tooling up and kitting out to do some of the hobby programming I lacked the tools for back then, I've been looking for some good MFC, ATL, and COM materials as a lead-up to Dino Esposito's Visual C++ Windows Shell Programming.
(Utilities and UI enhancements have always been my thing. On the Windows 3.1 hobby programming side, one of the big things I need to do is figure out how those "custom icons for Program Manager" things work. So far, my best guess is that, when I start investigating, I'll discover that they're hooking the event loop and rewriting the requests to build and modify the UI in a way conceptually similar to something like Proxomitron or Privoxy for HTTP.)
"Growing industry support for memory safe programming"
What a good idea. Although relying on defect free code is not a very secure idea. Such defects needs to be fixed at the hardware level.
“Where is the horn that was blowing? They have passed like rain on the mountain, like wind in the meadow. The days have gone down in the West behind the hills into shadow. How did it come to this?” Théoden King of Rohan
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
