Memory safety is the new black, fashionable and fit for any occasion Memory safety, a longstanding concern among serious software developers, has finally met with mainstream stardom. Consumer Reports, an 87-year-old nonprofit focused on product testing, this week published a report on memory safety. The publication acknowledged its unexpected coverage of the topic by explaining how it came to …
This is clearly the way forward, as it avoids the necessity to fundamentally rethink one's development practice and it also facilitates conversion of existing code.
It's sad that almost nobody ever got their heads round writing memory safe code in C (it can be done - I've done it), but this is probably the next best and more feasible option (provided there are no serious hidden vulnerabilities of other kinds in the rust modules).
Well, IMHO, it is a poor design when humans are expected to overcome deficiencies rather than allow the [already existing] automation to do the job more efficiently, every time.
In other words, IMHO, the problem is that the coders are expected to protect memory in C rather than spotting this out, and disallowing it, in the compiler. Of course the problem is legacy - back in the Ye Olden Days, when C was originally created, giving that level of power to the coders was seen as a benefit (just like how much power they gave [unnecessarily] to JS) and compilers weren't powerful enough to do the job, either. But programs back then didn't have to deal with today's hardware and software complexities; there was no predictive branching, memory space was massively limited, multiple cores sharing conjoined structures, etc. C was kept on, unmoved, as programmers loved the power of the available API's but couldn't deal with the increasing inter operational complexities that are possible in today's systems. C+ and C++ was created in hopes of dealing with some of it but not completely.
It needs to be 'intrinsic' in the system, and multi-layers, because humans are [very] fallible. If hardware was perfect (which it will never be), the hardware memory protections would be sufficient. But they aren't, so we need multi-tiered protection.
I personally have believed in Rust since first hearing of it, I can hope that it will really taper down vulnerability attacks that we hear about in the future.
C is a systems programming language, its what you write other languages and language support libraries in. Apart from assembler its the closest you'll get to the metal and like assembler you really have to know what you're doing to use it properly. So just as you can screw up royally writing in assembler you can easily screw up using C. Its not a problem with the language, its just that the wrong programmers are using it for the wrong purposes.
A language like Rust is (to my untutored, ignorant) eye just a wrapper around C that prevents careless programming errors.. In that sense its a good thing, the latest in a long string of attempts to protect computers from programmers. But ultimately Rust has to be written in something and will produce assembler, just like any other language**. Its main strength is replacing numerous library modules, some of great vintage and perhaps dubious quality, by a small set of properly tested components. But especially if it has a C API -- or when it makes system calls -- its just another language wrapper, not something totally new and improved and Earth shattering.
(**You could make 'Rusty hardware" but this is where you get into the notion of microcode and who writes it. Its all layers, after all!)
1.) All Turing-complete languages, including Rust, can be used to write a compiler for compiling the language itself. Has been done for Pascal, Algol and partially for Rust. Rust uses llvm because the optimizer and code generator of llvm is lots of work to reproduce in Rust.
2.) Many operating systems have been written in non-C languages. E.g. HP MPE (Pascal), ICL 2900 (Algol), Marte(Ada and some C), Oberon(Oberon), ASOS(Ada), Singularity(C#), RedoxOS(Rust).
3.) Ideally, as many parts as possible of a system are realized in strongly typed and memory safe languages. Many popular C based libraries were chock-full of exploitable bugs, including Pcre, OpenSSL, libpoppler, libwget, libcurl and many others.
4.) Finally, memory safety and strong typing are not a Silver Bullet. Software Engineers and Managers still need to have proper requirements, system architecture, scanners, parsers, proper object models and sufficient test cases from unit to system level. For reasons of "economics", testing is often insufficient to the extreme.
"because humans are [very] fallible"
This is the key observation. Even the most seasoned and best-educated software engineers will create bugs then and now. It happened to VxWorks, HP-UX, Linux, Windows, Apache, all Office packages, Yacc, Pcre, Flash Player, all sorts of PDF readers, all types of web browsers. In embedded control units(from ABS to software-stabilized jets), bugs will be found in "expert engineer" code by means of static analyzers, unit tests, module tests and system tests. Human error is the norm and must be countered by technological and organizational measures.
Memory Safety and Strong Typing (as opposed to JS, PHP, Python) are two of the most powerful tools to limit and contain the damage from software engineer's error.
Finally, this is not a new observation, Tony HOARE and Niklaus WIRTH have been saying this for decades. Algol Mainframes from ICL/Fujitsu, Lebedev Institute and Unisys have been providing memory-safe execution environments for decades, but they were much pricier than Unix. The cheap approach "won".
Risks provide us with drive to do things and when headed teach us valuable lessons. Classic knee-jerk complacency. "Oh no, C and C++ are dangerous 'cos I could make mistakes. Boo hoo!". Holy heck, I could cut my finger off with a bread knife and bleed to death before the week is out, oddly I haven't thrown all the sharp things away yet! I'm not in the least bit interested in C or C++ anymore but I respect that they have a place and like chainsaw, they can be bloody dangerous in the wrong hands but that's life. You pay decent money to a professional then you put faith in their abilities to mitigate such things.
Don't get me wrong, devs need to be efficient and I see solid languages that will aid in helping coders concentrate on logic and squirral away the nasty things that could go wrong but that's the reason being a coder is so well paid, in the wrong location and the wrong code you get nuclear meltdown and rockets going bang. Most of the time an app crashes and some data has to be reloaded.
The curmudgeon in me is desperate for a rant I guess about how the world is too comfortable and risk free, from cradle to grave no one will take risks anymore and we're losing the ability to learn from our mistakes and make ourselves better for it.
I think the real benefit is because there aren't enough skilled and careful software developers to meet demand. Heck there aren't enough of any ilk around.
So the software development industry has to cater for the less able to the downright careless and incompetent. We have to be able to utilise the second-, third- or fourth raters so we need whatever tools we can come up with to try and protect everyone else from their mistakes.
Software development cannot be restricted to the best of the best - not across the entire gamut of development. If you're developing airplane control systems then sure - you weed out the chaff and your customer pays for the quality. But if you're just trying to produce a new streaming box or amazing digital-calendar-with-weather-station app for a phone then you take anyone you can find and you're grateful.
C'est la vie.
Even the most educated, capable and seasoned software engineers are under economic pressure to "deliver something working". That implies bugs. Example: The widely used Yacc compiler generator had a bug, which resided for more than 30 years undetected. There is no such man as a "perfect software engineer", but only various levels of "fallible".
That includes core software components for embedded systems. It also includes Boeing Co, who killed 250 people in a rookie software conception mistake("737 MCAS").
I’ve worked in consumer electronics (TVs to be precise). The pressure to get stuff out of the door is enormous
BUT I never released any software I didn’t think was totally correct. And if I was managing the projects I worked on, in works be very unimpressed by anyone who would consider such a thing
I'm quite happy to amend my comment to:
I think the real benefit is because there aren't enough skilled and careful software developers to meet demand and even those that exist are under pressure to discard their principals and 'just get something out of the door' due to management pressure.
I'm not saying it's right (certainly not shoddy code due to management pressure). I'm just suggesting that expecting all programmers to work to a high standard all the time is unrealistic. That's 'ivory tower' thinking. In the real world most programmers have to cut corners at least some of the time and we need tools to act as a safety net for all of us.
Then have hand-holding languages for under-trained engineers and let the rest of us use the tools we've developed for our own preferences.
This is what we do in every other endeavour : we have training where it's needed.
Why is programming any different ?
Someone mentioned Max. That wasn't a language failure. It was a management failure and a specification failure. It would have relied on faulty sensors whether the language was memory-safe or not.
While it may be beneficial to newer and better(?) tools for new projects, there are legacy systems in place nonetheless. So what about them?
The focus on memory safety should not be wasted in "Rust vs. C++" debates. The energy should go into addressing the causes for memory-unsafe code, and to devise ways to mitigate these.
We do this all the time.
People are inherently selfish, hence we enact rules and guidelines to help us creating a reasonably well functioning society.
Think about your city's traffic, if there weren't any traffic rules.
The rustaceans should do their thing, and the C++ people should strive to achieve memory safety on their part.
And if C++ isn't inherently memory safe, then some framework is needed to ensure that the resulting code is.
However, this does not work well with copy&pasting code from the internets or doing away with testing and Q&A teams.
Hence, I get my coat.
"C++ people should strive to achieve memory safety ".
There are very basic reasons, why this cannot be achieved efficiently:
1.) No concept of thread-local and thread-global data in the type system. This creates inefficiencies (any smartpointer must be threadsafe ?) and dangerous race conditions, which can destroy the heap and thereby the entire process.
2.) No automatic detection of raw arrays, raw pointers or vector::operator[](). All of which is a potential memory bug.
3.) No concept of detecting "pointer in the heap pointing to the stack".
4.) No concept of avoiding crazy casts such as
int x;
RadarTrace* radarPtr = RadarTrace*(x);
5.) Not standard way of stopping a stack overflow before it damages other modules.
$ valgrind ./unsafe_c_program
This will resut in a memory safe program execution at a penalty of 100 times more runtime cost. But it will detect all memory errors. Due to the slow execution, some bugs will be completely masked and appear to not exist (especially the multithreading bugs). With Rust or Sappeur the penalty will be in the order to 1.5 to 5(e.g. matrix multiplication).
Threads always have been and always will be a dumb idea because:
1) There is no well defined interface between the interacting components.
2) The hardware & compiler has to try and fake a flat memory model (no physical memory is perfectly "flat" in the time domain, never has been, never will be).
3) The programmer has to try and come up with a reliable scheme for ensuring the threads don't clobber each other.
At the end of the day anything that scales reliably ends up using a message passing/CSP model at some level - most hardware has been operating in that way at some level for *decades* to present the fiction of flat shared memory to threaded apps.
Going out on a bit of limb here, but any argument or assertion that presumes threads are either desirable or necessary is inherently invalid.
"Asked about C++ creator Bjarne Stroustrup's assertion that ISO standard C++ can be memory safe when rules are enforced with static analysis, Aas expressed skepticism."
Perhaps he should speak to the squillions of C++ devs who only worry about memory leaks when interfacing with C code.
Frankly, anyone who can refer to "C/C++" in a discussion of memory safety is not worth listening too. (And yes, I'm familiar with Russinovich and his work and I do not know what he was smoking when he made those comments. Perhaps using Visual C++ as a glorified C compiler is common with MS. It would explain a lot.)
I guess it depends on whether you consider C++ to be a completely separate language, if only philosophically, or just "a better C". I suspect that Russinovich falls into the latter category. In the Windows world it's been quite a while since you could buy a C compiler that didn't also do C++. So they're just the same, kind of.
-A.
I'm not sure how to answer this argument that modern C++ solves memory problems. The argument normally comes with a trivial example as proof. And the example is typically already easy to identify & fix with linting or static tools even before modern C++.
Those are not the actual kind of coding errors that leak memory that the approach rust takes is intended to address.
From my experience the real problems lie in thread / hardware module interactions (often not fully defined even on existing x86 hardware), or the use of an API that requires a guarantee that the coder failed to meet in 1% of cases. Or where someone just simply screwed up and the tools never ran on that part of code due to error.
In my experience C and C++ requires the coder to have an ungodly understanding of the entire code base and operating system and sometimes even hardware specifics to be sure the code they are writing is correct.
In some ways this is why explicit ownership enforcement is the best paradigm to address this problem.
"real problems lie in thread / hardware module interactions"
I have been using pthread threads, mutexes, semaphores and never had such mysterious problems. Works nicely on SPARC, Intel x86, AMD x86, Apple M1, PowerPC, Elbrus, HP PA. OSs Windows, Linux, Solaris, HP-UX, xBSD, MacOS 11.
Can you post an example of your problems ?
"Memory errors occur when computer code tries to access an area of memory that's undefined"
Is that what happens when you install some suspiciously cheap Chinese DIMMs that turn out to have less addressable storage than claimed?
Seriously, I'm guessing that particular description came directly from Consumer Reports. I wouldn't expect such gibberish in a tech publication.
-A.
It's a terrible definition, but composing a good definition is pretty difficult. For example, let's try to create one in terms of Turing Machines:
In each state, the current instruction is intended to read a subset (which may be empty or non-contiguous) of cells of the tape, and to write another such subset (which may be identical to the read subset, and may be empty or non-contiguous). Reading or writing any cells outside of those subsets is a memory error.
The problem there is obviously the "intended". What can we do about that? Perhaps:
Here "intended" is defined in terms of the program being executed by the TM. A program may be understood, without restriction, as a reification of a set of algorithms (non-empty if the program has non-zero length). "Intention" can then be defined as the set of consequences described by the idealized descriptions of those algorithms.
That's rather Platonic and abstract. It certainly doesn't get us any further in terms of a systematic procedure for classifying some proper subset of errors as memory errors.
There may be a clever way out of this, but ultimately, I think, the definition of "memory error" has a subjective component because it relies on some ground-truth notion of "intention" for the program, which will be determined by the observer.
That doesn't mean we won't often be able to come to consensus on particular errors being memory errors or not, but in at least some cases we'll be left with Stewart's Definition: "I know it when I see it".
So, it seems I'm being asked to trade in my sleek racing bike for a junior bike with training wheels because it is safer. I've got a better idea, for developers to learn the skills required to use dynamic memory allocation safely. I've been doing that for over 30 years and not had a memory-related crash in that time. Let's stop trying to save 'copy-and-paste' coders from themselves.
The real threat are the undiscovered bugs, also in YOUR code, which will be exploited by a cybernetic attacker over the network or other channels. If the attacker is an expert, you and your users will never even notice the subversion.
E.g. https://www.theguardian.com/world/2022/apr/05/apple-iphone-pegasus-spyware-nso-group-israel-jordan
I think that your analogy is wrong. If you compare C to Python, you are being asked to trade your sleek racing bike in for a sturdy commuter. The latter is easier. simpler to use, safer but not as fast. Comparing Rust to C, I would say you are asking to trade your sleek road bike for another sleek road bike but with electronic gear shifts.
Of course, my analogy is a bit biased. Electronic gear shifts are getting pretty embedded now and will probably take over. Rust is rather less so. It's still in that stage where everyone is looking and thinking, cool, but will it work in competition use.
If you are a bike mechanic, getting the skills for all of these is probably wise. People will still be commuting. Club riders will be using a mix of electronic and manual for some time to come.
Damn it, I was supposed to be talking about programming languages....
"In a very theoretical sense that ignores the practicalities of the real world, that might be true," [Josh Aas from ISRG] said. "It might very theoretically be practically possible to write memory safe C++. But it's just not how things work in our world [emphasis added]. There are just better ways to do that. C++ was not designed from the ground up to offer memory safety."
In other words, in this guy's "real word", the average coder is too stupid, too ignorant, too lazy, too incompetent, too undisciplined, or just can't be arsed to do a proper job of software development when it comes to managing the resources needed to do a particular job. (That is an inclusive 'or'.)
The dumbing down of the world, and of the "practitioners" of this art and science in particular, continues unabated. I harken back to the middle 80's, when doing interviews I found that some PhD graduates I was interviewing had no idea what a register was. <sigh/> Nowadays, these putzes probably don't even know what a heap is.
Gee, I wonder if ChatGPT would be able to write memory-safe C or C++?
Heaps are easy. Those are the things the that you configure in the JVM command line such that they consume the entire address space of the machine - just in case it runs out of memory printing Hello World. Of course real pros run it under a VM because it's safer that way when it takes the whole OS down with it.
No mention of SQL injection attacks.......might just take place at the web/HTML level......
....no need for C programmer skills at all.....memory......what's that?
....and of course there's the Ken Thompson attack....SolarWinds and so on.........
Please.....let's just get away from C, C++, C#, Rust.........there's PLENTY of opportunity elsewhere!!
Link: https://www.invicti.com/blog/web-security/sql-injection-cheat-sheet/
Link: https://wiki.c2.com/?TheKenThompsonHack
One of the activities I spend a lot of time on is mitigating / fixing / risk assessing vulnerabilities. The memory based vulnerabilities still pop up but the vast majority are related to poor design / implementation of Java libraries. For example Jackson needs regular updates to extend their blacklist of classes that they should not handle.. Even log4j keeps springing new ways to be owned, and then of course there is Spring... JavaScript based stuff is just plain rotten and unfixable due to the vast tree of interdependencies... Case in point I tried out an example hello world static web page... it needed 1471 npm modules (not even kidding here) to build it. Good luck validating that lot.
"Endorsing Russinovich's mandate, security firm Chainguard on Wednesday said Wolfi, a memory safe Linux "undistro" – designed for producing secure container images – has incorporated the memory safe Rustls TLS library and HTTP via the Rust-based Hyper library in the curl network request tool."
Looking at their site, are they really claiming to have made a memory safe distro because they packaged hyper and rustTLS for curl? The things people do for marketing. Redox is more true to being a memory safe OS.
The very same Hyper library that waves its hands in the air and gives up if a Content-Length header threatens it with too much input:
Rust projects open to denial of service thanks to Hyper mistakes
Well we're all saved then. Hallelujah.
From the horse's mouth (Julian Reschke, contributor to HTTP spec):
The length is defined by the Content-Length header field (or Chunked Encoding, or closing the connection).
If the length doesn't "match" this means that the extra bytes belong to the next message. If these extra bytes do not represent an additional HTTP message, that's a communications error. Just close the connection and report an error.
This was also explained in the cited article (7th paragraph).
I don't see aborting and bringing down a client or server as following the spec.
I said that there are worse things you can do in. There are better things as well, and this is no doubt why they are or have fixed it.
In the end, Rust as a language is designed to protect against a number of things, including memory safety. Jumping and saying "look, here is a bug that is not about memory safety, it's all a waste of time" is hardly a strong riposte.
1.) Do not use C or C++ to implement internet-facing systems.
2.) If you need maximum performance in technical or scientific applications (e.g. Matrix Multiplication, FFT,...) and your input can be considered "safe", then C and C++ might still be the right languages. This becomes tricky, if your enemy could potentially fake a radar or sonar echo and expose your algorithm to this fake signal.
2.2) A Hybrid Approach of "C++ for low level signal processing" and "memory safe scanner, parser, validator, application logic" might be ideal.
@fg_swe
Quote: "...Do not use C or C++ to implement internet-facing systems..."
Me and my buddies have a peer-to-peer application implementing messaging (using email, typically Gmail) .
The code is ONLY installed on clients.
The application implements private encryption/decryption.
The code is written in C....clang/gmp/GTK3..........
Please explain to me why we should worry about vulnerabilities in C?????....or because Gmail is "internet-facing"?????
Maybe this particular setup is "secure" because "no potential attacker knows the email adresses".
Maybe not and the attacker will send a crafted message, which exploits a bug in your crypto endpoints. This will become likely, as soon as you have lots of users and one of them is hacked. And of course, if you are an interesting target. Finance, politics etc.
Why do you use gmail as the "router" ? Why not your own little TCP based router program ? That would cut google out of the picture. Whitelist the allowed IP addresses as a Defence In Depth.
@fg_swe
Good suggestions.
(1) Email is a universal transport.....one less thing to worry about, especially since the message content is encrypted.
(2) Gmail and others provide an "app password", so the peer application can send and receive on its own -- no email client needed.
(3) The TCP suggestion is a good one....but I've had a look at programming around NAT...too much trouble (item #1 is MUCH easier).
(4) The TCP suggestion also has the problem that broadband providers (like BT) regularly change the IP address -- more admin -- too much trouble!
(5) "...attacker will send a crafted message..." OK....but highly unlikely.....since every message uses a different random key which is never published and is immediately destroyed (see Diffie/Hellman).
......but thanks.....I'm not used to helpful responses!!!!
You possibly think that your crypto layer protects the code which displays the plaintext (text, audio, video, HTML, GIF, whatever). This is a dangerous assumption, as one of your communication partners could be captured, be a mole or be hacked.
Then a properly ciphered data stream, containing an exploit, could enter your "display code".
I assume this is a theoretical threat for you, but this is not true for all users. See
https://en.wikipedia.org/wiki/Operation_Vengeance
https://www.theguardian.com/technology/2020/jan/21/amazon-boss-jeff-bezoss-phone-hacked-by-saudi-crown-prince
https://de.wikipedia.org/wiki/Hans-Thilo_Schmidt
You can write memory safe C++ code. You can write SIL4 safe code in C.
My approach is...
Use the tools available... Continually learn your craft...
Write tests that can be automated.
Check coverage. Ensure sufficient coverage by writing more tests.
Use static analysis tools.
Use the Core guidelines.
Turn all warnings on. Make them errors.
Avoid raw pointers and arrays.
Use valgrind and sanitisers.
Build with more than one compiler.
Automate your build, test, analysis pipeline.
Document your code and provide useful comments.
Take a defensive programming stance.
Twenty odd years ago I had an Admiral complement us on a particular large piece of software. He said it was the most stable application his people had used. After 5 years in service there were less than 10 bugs registered. All were essentially change of requirements. The application was written in C++ for performance reasons. Test coverage was higher than 85%, the missing bits were error reporting and recovery.
If you need to do SIL4 code you or someone else will be looking at the output of the compiler, i.e. the assembler with optimisation turned off to ensure what you wrote in C will be correctly expressed in assembler and thus machine code. Either way the level of testing and validation is huge...
... isn't the language itself, it's the build structure that is being encouraged to be used. Same with "go".
Is this a thing now?
Sure, many programs rely on other libraries from third parties - no point inventing the wheel, but it's now ridiculous. I attempted to install a relatively simple rust program recently, and it had 137 dependencies. - The majority of them were crate distribution files smaller than 10K !
I wouldn't be surprised if people are linking to third-party live development snapshots, like seems to be the cool way now with things like "node". et. al.
Yes. What we've seen over the past several years are that public code repositories are pretty much all toxic, and the rush to open-source code reuse for every trivial thing hasn't improved code quality or security. The constant parade of attacks on packages and package repositories, and of severe flaws in popular packages (which are often poorly maintained) is ample evidence of that.
Having everyone reinvent the wheel wasn't a good approach either, of course; but public repositories and package managers have very much failed at providing a better solution.
Some of that is fixable. Much better security controls and provenance tracking in package repositories would help, for example. Some of that is fixing obvious vulnerabilities like the account-takeover bugs that have been fixed in GitHub in recent years. (For that matter, not using GitHub would help. git's a decentralized change-management system; stop centralizing it.) Some would require significant changes to how commits are made, possibly along the lines of proposals such as CHAINIAC. Discouraging feature creep and bloat would help. Discouraging direct pulls from public repositories as part of the build process would help; commercial software, at least, should be built using staged packages from an internal, locked-down repository that gets updated using a tracked, controlled process, by people specifically authorized to do so, who are required to perform due diligence.
And the move by some big customers to requiring SBOMs with transitive-dependency reporting may help, simply because it will require tighter control of imported packages, and creates an economic incentive to reduce the number of packages being used.
On the whole, though, the current trend in software development to pull in an enormous graph of poorly-understood packages from many sources, often for little or no good reason, is a disaster. And while we can improve the situation (as we did with, say, TLS and PKIX) with various tweaks, it's always going to be severely fundamentally flawed.
I started following Rust out of interest to see what this could offer.
I was disturbed to find the language isn't defined yet. This looks like another Python in the making, with a constantly changing API.
Write up a definition that's good for as long as the C standard and you'll get somewhere. Until then, it's for experiment only.
Although not really having an real knowledge of Rust, it seemed, even in this fairly early stage of its evolution, to be rather large and increasingly complex to the point I wouldn't discount its surpassing C++ in both.
If programmers require both a deep and broad knowledge of the language to produce correct (which isn't quite the same thing as safe) code, I am not so confident this will be such a great advance. Perhaps the availability of industrial strength components (libraries, tools) and restricted domain specific patterns will address this.
I was wondering about tools for (formal) verification and/or validation of Rust code and was pleasantly surprised, after a few clicks, to discover an apparently well developed ecosystem.
Should be interesting...
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
