Posts by fg_swe 1478 publicly visible posts • joined 20 Nov 2021
"real problems lie in thread / hardware module interactions"
I have been using pthread threads, mutexes, semaphores and never had such mysterious problems. Works nicely on SPARC, Intel x86, AMD x86, Apple M1, PowerPC, Elbrus, HP PA. OSs Windows, Linux, Solaris, HP-UX, xBSD, MacOS 11.
Can you post an example of your problems ?
Maybe this particular setup is "secure" because "no potential attacker knows the email adresses".
Maybe not and the attacker will send a crafted message, which exploits a bug in your crypto endpoints. This will become likely, as soon as you have lots of users and one of them is hacked. And of course, if you are an interesting target. Finance, politics etc.
Why do you use gmail as the "router" ? Why not your own little TCP based router program ? That would cut google out of the picture. Whitelist the allowed IP addresses as a Defence In Depth.
1.) Do not use C or C++ to implement internet-facing systems.
2.) If you need maximum performance in technical or scientific applications (e.g. Matrix Multiplication, FFT,...) and your input can be considered "safe", then C and C++ might still be the right languages. This becomes tricky, if your enemy could potentially fake a radar or sonar echo and expose your algorithm to this fake signal.
2.2) A Hybrid Approach of "C++ for low level signal processing" and "memory safe scanner, parser, validator, application logic" might be ideal.
The real threat are the undiscovered bugs, also in YOUR code, which will be exploited by a cybernetic attacker over the network or other channels. If the attacker is an expert, you and your users will never even notice the subversion.
E.g. https://www.theguardian.com/world/2022/apr/05/apple-iphone-pegasus-spyware-nso-group-israel-jordan
$ valgrind ./unsafe_c_program
This will resut in a memory safe program execution at a penalty of 100 times more runtime cost. But it will detect all memory errors. Due to the slow execution, some bugs will be completely masked and appear to not exist (especially the multithreading bugs). With Rust or Sappeur the penalty will be in the order to 1.5 to 5(e.g. matrix multiplication).
"C++ people should strive to achieve memory safety ".
There are very basic reasons, why this cannot be achieved efficiently:
1.) No concept of thread-local and thread-global data in the type system. This creates inefficiencies (any smartpointer must be threadsafe ?) and dangerous race conditions, which can destroy the heap and thereby the entire process.
2.) No automatic detection of raw arrays, raw pointers or vector::operator[](). All of which is a potential memory bug.
3.) No concept of detecting "pointer in the heap pointing to the stack".
4.) No concept of avoiding crazy casts such as
int x;
RadarTrace* radarPtr = RadarTrace*(x);
5.) Not standard way of stopping a stack overflow before it damages other modules.
Even the most educated, capable and seasoned software engineers are under economic pressure to "deliver something working". That implies bugs. Example: The widely used Yacc compiler generator had a bug, which resided for more than 30 years undetected. There is no such man as a "perfect software engineer", but only various levels of "fallible".
That includes core software components for embedded systems. It also includes Boeing Co, who killed 250 people in a rookie software conception mistake("737 MCAS").
1.) All Turing-complete languages, including Rust, can be used to write a compiler for compiling the language itself. Has been done for Pascal, Algol and partially for Rust. Rust uses llvm because the optimizer and code generator of llvm is lots of work to reproduce in Rust.
2.) Many operating systems have been written in non-C languages. E.g. HP MPE (Pascal), ICL 2900 (Algol), Marte(Ada and some C), Oberon(Oberon), ASOS(Ada), Singularity(C#), RedoxOS(Rust).
3.) Ideally, as many parts as possible of a system are realized in strongly typed and memory safe languages. Many popular C based libraries were chock-full of exploitable bugs, including Pcre, OpenSSL, libpoppler, libwget, libcurl and many others.
4.) Finally, memory safety and strong typing are not a Silver Bullet. Software Engineers and Managers still need to have proper requirements, system architecture, scanners, parsers, proper object models and sufficient test cases from unit to system level. For reasons of "economics", testing is often insufficient to the extreme.
"because humans are [very] fallible"
This is the key observation. Even the most seasoned and best-educated software engineers will create bugs then and now. It happened to VxWorks, HP-UX, Linux, Windows, Apache, all Office packages, Yacc, Pcre, Flash Player, all sorts of PDF readers, all types of web browsers. In embedded control units(from ABS to software-stabilized jets), bugs will be found in "expert engineer" code by means of static analyzers, unit tests, module tests and system tests. Human error is the norm and must be countered by technological and organizational measures.
Memory Safety and Strong Typing (as opposed to JS, PHP, Python) are two of the most powerful tools to limit and contain the damage from software engineer's error.
Finally, this is not a new observation, Tony HOARE and Niklaus WIRTH have been saying this for decades. Algol Mainframes from ICL/Fujitsu, Lebedev Institute and Unisys have been providing memory-safe execution environments for decades, but they were much pricier than Unix. The cheap approach "won".
Russia has a population in the order of 140 millions. For simplicity we assume a similar age structure in NATO.
If they do all out conscription, then Ukraine and Poland don't have sufficient manpower. At least England and Germany would have to start conscription, too. Sweden and Finland will be happy to defend themselves.
Maybe this must be done, I dont have a crystal ball, either.
And this time we will not make an exception. Draft the journalists and bureaucrats first, they are typically the worst warmongers. Those who claim disabilities must serve in command centers/logistic centers/arms dumps, always ready to take a CM or ballistic hit. Age 18..60, no exceptions.
Of course we could escalate:
1.) Take out critical infrastructure deep in Russia, using cruise missiles, stealth.
1.2) Sea blockade of Russia; serious sinking of Russian surface fleet. Submarines must be dealt with, too.
2.) Fight a nasty infantery/tank/missile war in Poland, Baltics
2.2) Conscript men in England, France, Germany, Spain, Italy, USA, Canada to get the required manpower vis a vis Russia's conscription
3.) Absorb quite a few cruise missile hits on London, Berlin, Warsaw and maybe even Paris.
In other words, not as convenient as our childish warmongers want it.
Do YOU want to be conscripted as a MILAN anti tank gunner ? We have would have 200 000 openings TOMORROW.
The question of "who belongs Ukraine to" was an open question a long time ago. It could have been answered by "neutral like Finland." But diplomacy failed to make this happen, so the weapons make the decision.
From the Russian POV, there are three empires: Moscow, Anglosaxon(from Seoul to Vilnus), Beijing. The Moscovites see the Anglosaxon empire encroaching on theirs and they felt a need to make the expansion stop, as talking yielded nothing substantial.
We now have the fire burning, how to turn it off ?
After that, the Dear Leader and his Generals will have all their nice villas wiped out, the airforce destroyed, all surface ships sunk and so on. His huge army with outdated cr4ap will be hammered into pieces in three weeks.
If they dare to use nukes, well, the Americans have thousands, all of them well tested all round.
If Russia were to ever face all of NATO, it would be very asymmetrical. Russia already showed the Nuclear Card around because they know their weakness. Unlike Hitler, Putin has not inherited an industrial powerhouse. The difference in industrial power is easily 20x or more. That is, if you can quantify the technological gap in many technologies.
The main "weapons" of Russia are propaganda and cojones. That won't help against mass attacks of cruise missiles and state of the art electronic warfare. It won't help against a VW factory which is converted to low cost drones.
Cojones, we have, too.
Regarding the "burn down the world" utterings, we should strongly request the Moscovites to put the dear leader into hospital for six weeks. Or does Moscow really want to commit suicide ?
Given that both sides have had serious damage and horrible death counts; given that both sides have rational arguments (Ukraine: self-determination; Russia: strategic security) it is time to look for a Korea-style solution.
Freeze the war at the current frontline.
The alternative is a deadly idealism, a continuation of the meatgrinder.
Was it Mrs NULAND or was it Mr PUTIN ? Nobody really knows anymore.
I get it, I am wrong in theory. But not in practice for the OSs/environments Sappeur currently targets. I specifically said so from the beginning. Windows, Linux, BSDs, MacOS, Solaris, HP-UX, AIX - they will all reliably generate a SIGSEV(or equivalent) when accessing a NULL pointer. That's deterministic behaviour as required for Memory Safety.
The other environments (small embedded systems without an MMU) I currently do not target.
I never claimed the "address 0 invalid space" exists in embedded systems, rather I specifically claimed this to exist for all sorts of modern Unix(es) and Windows. This invalid address space exists, has been existing for a long time and will detect NULL pointers reliably and at zero runtime cost.
ALSR will randomize addresses outside the "invalid space" and does not matter here.
Again, please post a demo program for Windows, Linux or MacOS, which will prove me wrong.
In this case the software engineering organization must PROVE this cannot happen. This usually means the process cannot allocate or deallocate heap memory(except during boot-up and stand-down), as the heap is too unpredictable for hard realtime systems.
Of course the generation of this Proof might be hard to do by hand. If the language support non-nullable pointers the proof is easier. A source code inspection by experienced engineers combined with all the testing of the V model might(or not, if code is too complex) be able to generate this proof. Any relevant source code change must trigger a regeneration this proof.
Also see SPARK Ada and similar.
Please see my other post with the test program in C. The invalid memory space starting from 0 is actually many Gigabytes in size on 64 bit machines. This means that any NULL pointer to an object smaller than that will generate a SIGSEV exactly where the bug is. A debuggable core fil will be dumped. Sappeur Arrays are a not affected by this limit.
Conclusion: for all remotely sane (object size lower than 1000 000 000 octets) programs NULL pointers will generate a SIGSEV.
If you do not believe me, please perform your own tests and prove me wrong.
Please use the following test program to see that in practice your concern is not an issue. Apparently the "memory guard space" is in the order of 140 735 371 892 940 octets. (Linux 64 bit).
On MacOS, it seems to be about 6000 000 000.
This also aligns with my practical experience writing software in C, C++ and Sappeur.
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
struct MemTest
{
char buffer[650000];
char* str;
char buffer2[6500000];
};
int main(int argc, char** argv)
{
int x;
char* str1 = malloc(100000000);
printf("adresse von str1: %lli\n",(long long int)str1);
printf("adresse von x: %lli\n",(long long int)&x);
strcpy(str1,"abc");
printf("%s",str1);
free(str1);
str1 = NULL;
struct MemTest* mtp = NULL;
mtp->str = malloc(10);
return 1;
}
As I wrote above, I have never seen what you describe happen in the real world. (On Unixes and Windows)
And of course I had plenty of cases of forgetting to initialize a pointer during a development session and always got the determinsitic SIGSEV exactly at first dereferencing.
Your are right though that it is a problem in contrived examples.
During my 20 years of software engineering experience on HP-UX, AIX, Solaris, Windows, MacOS and Linux I never had the problem of a NULL pointer not leading to a deterministic crash, exactly where the FIRST pointer dereferencing happens.
This is because these operating systems by default allocate "invalid" MMU pages from address 0 to something like address 64000. In the example above, the planet struct would have to be bigger than 64K to lead to an undetected error.
This is not the case in embedded systems, though. Sappeur currently targets "only" all kinds of Unixoid and Windows OSs with MMUs. From Solaris to ELBRUS Linux. One key assumption of Sappeur is that its smart pointers are initialized to NULL and will create a deterministic SIGSEV (or Windows equivalent) on dereferencing a NULL pointer. This assumption is important for performance, safety and security reasons.
So you are "right" that this mechanism is not safe and secure for "huge" Sappeur classes.
In 10 years of programming in Sappeur I never had such a case. Creating a class with 64K size per instance is rather unusual and I never needed this. In case of arrays, which can of course be much bigger than 64K, the SPRArray._sz member is at the beginning of the data structure, well within the first few dozens of octets. Each array access will be preceded by accessing _sz for index checking. This will then generate the deterministic SIGSEV and a debuggable core.
So in theory you have a point, but not in the real world I have seen. In a future version of Sappeur I might consider adding a check that classes cannot be larger than 64K, thereby eliminating the problem in principle. Larger classes would then require non-nullable pointers, something also to be added to the language.
Again: you have a programming error, which can lead to RAM exhaustion. An attacker comes along and triggers a SIGSEV from that. Program stops, core is dumped. Then you, the senior engineer, attaches gdb and finds the error location. Very soon you have found the bug, fixed it and compiled the new program version. System running again after 33 Minutes.
No information disclosed to attacker, no effectors manipulated, downtime 33 Minutes. Great.
planet* P = new malloc(sizeof(planet);
p->weight = 1E30;
That will fail deterministically, debuggably and SECURELY(SIGSEV plus core dump) on all IT operating systems I know of, if memory is exhausted. Of course you can (and sometimes should) check the return value of malloc, but if you do not do it, you will not have a security problem. That was my point.
A determinstic(read: easy to debug) crash from a DOS attack is benign as compared to Malware Injection and Reconnaissance For Months.
The only exception to this would be military communication systems, where downtime could mean losing a war. As far as I know, they have their software engineers "embedded" or on very short call. Their most important systems are created by "themselves", which means they can fix any DOS issue on very short notice.
If you create an http server for a "known population of clients", then maybe you do not need to care about DOS attacks. Note that this is not true for subversion opportunities, as you must always assume one of your intranet machines being compromised.
Actually, this is how things like Oracle are operated - they are locked behind a firewall as they would be easily hacked if exposed to the wild world of internet or even the entire intranet.
The management of RAM allocation, database connection numbers, file handles, number of threads etc must be managed by the application programmer. There is no sensible way an automatic runtime mechanism can do this for the app programmer. Except, of course, stopping the thread or program upon resource exhaustion.
So - the application programmer must think about all the resources he allocates in his program. For example, an http server must reject too many parallel requests(Code 429 Resource Exhausted). An application using database handles must limit the number of database connections by some sort of pooling and semaphores. No automatic mechanism on the runtime/language level can replace programmer reasoning here(except maybe some sort of database pool which blocks until a connection becomes free).
Memory Safety is not the paradise of programming, it "just" eliminates an ugly kind of cancer.
Software Engineering is a highly complex craft+science with lots of aspects. If it were simple, we would not earn good money on it.
In all the above languages, you will get a deterministic crash if heap allocation fails. You either get a NULL pointer from malloc() or new or some sort of OutOfMemoryException. Accessing a NULL pointer typically creates (some sort of) SIGSEV and stops the program. OutOfMemoryException typcially stops the thread.
This is exactly what you want. A deterministic, debuggable crash from a programming error/cybernetic attack. Much better than Silent Subversion from e.g. a buffer overflow.
How else could an out of memory condition be handled ?
(this applies to Windows, Linux, BSD, HPUX, Solaris, AIX, but maybe not to embedded systems)
The idea of a smartphone was also envisioned by HP Labs in the 90s. But then the large-than-life Bill Hewlett and David Packard died. There was no replacement for their engineering, production, research and operations experience. The MBAs could only slash the brain and optimize the economic side. "Optimization" meaning they would kill HP technologies and sell MS, SAP, Oracle, Intel products instead.
So the MBAs could not imagine the great value of a smartphone. They had fired all the engineers who could build it.
Steve Jobs could imagine and had the engineers on board. He made it happen.
Having said that, there are plenty of opportunities and strong companies out there. Apple is doing great both technology-wise and economically. Unlike IBM, they understand that mechanical design, GUI appearance and usability matters. Then there are hundreds of small companies who need seasoned IT experts.
Pump out dozens of applications and while you wait for the response, learn something new. A new language, HTML, a new framework, a new type of database. Learn about the V Model, there will be enormous work in auto, aerospace, rail and medical - as soon as they actually do the work according to the book. Learn effective presentation, if you haven't yet. Write a blog about something technologically relevant...
1.) Forget the BS they told you about "lifelong job security". No commercial company can do that. Not even Google. It's a br4inf4ck to stop you from looking for an external career move.
2.) "creative destruction" is a very real thing, like it or not. Microsoft, Google, and Amazon ate the business of HP, IBM, DEC, Unisys, Fujitsu. These "old" IT companies have a calcified brain and cannot quickly adapt to new challenges. Even worse, they cannot use their own great ideas, because they would threaten existing business. For example, HP's BIRNBAUM was thinking about Cloud Computing back in the 90s. Amazon made it happen in the 2000s and HP is now on crutches.
2.2) Because the business of the old companies evaporates, their employees can no longer be paid at a proper rate. Macro-economically speaking, they must transfer from the old companies to the new ones.
3.) Never stay too long at one company, so that you know how to sell yourself and get interviews. Not applying for 20 years will deteriorate your self-selling skills.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2025
