Posts by fg_swe

Aviation delays ease as airlines complete Airbus software rollback

If you read carefully, the bit flip could occur in a high valued bit of an important variable. This would trigger maximum elevation of horizontal control surfaces. A/c would perform extreme pitch, resulting in high aerodynamic forces, resulting in structure breakup.

Electronic control means the control unit must work almost perfectly. Bitflips must be propely dealt with. Sensor faults must be dealt with.

The same argument can be made about hydraulic and mechanical controls, though.

Engineers need to be on top of any failure mode.

Clearing Up Physical Mysteries

1.) Measure real-world radiation

2.) Talk to a particle physicist how to simulate 1000x the radiation in a lab

3.) Strap control unit with the real software in a HIL setup in front of artificial radiation source (linear accelerator or the like)

4.) See what the HIL reports.

5.) Change software and or hardware.

6.) GOTO 3.

It is almost as if we spend lots of money on CERN and almost if Airbus could cooperate with CERN on this matter. As both entities are funded mostly by EU states.

V-Model, HIL, Redundant Sensors

In theory, this kind of things can be avoided by running the control unit in realistic HIL tests. As mandated by the V-Model.

Sensor failure is part of proper HIL Testing.

Redundant Sensors detect Sensor failure.

HIL Testing can be done in front of a particle beam, which simulates the sun and other radiation sources.

One should think that well-educated and enlightened engineering managers could think of this and make the necessary time, money and machinery available.

Well

This issue seems to expose a deficiency of current aircraft/spacecraft control unit development(HW+SW) processes.

Why did they not find it in a HIL test strapped in front of an appropriate particle beam(simulating a solar storm's radiation over several years) ?

I've added this subject to my document on these matters:

https://di-fg.de/RobusteSoftware.html

Re: Details, Aerospace Software

Note 2: software of this type is developed with the V-Model approach, which is vastly different from the quick-and-dirty approach used for most beancounting and general IT software.

See

https://di-fg.de/RobusteSoftware.html

Airbus does have a good history of faithfully executing the V-Model and this appears to be an unfortunate exception. Nevertheless, they should now subject ALL of their safety-critical control units to artificial particle beam while executing inside a HIL test rig.

Details, Aerospace Software

It transpires:

1.) The problem was a bitflip, caused by solar storm radiation. For some hard to explain reason, the affected variable in main memory was not protected by CRC, ECC or the like.

1.2) Protection is ideally done by hardware, but can also be done in software: Store multiple copies of the variables and compare them upon each use. Handle deviation in a proper way.

2.) The affected software controls the horizontal control surface. This means the aircraft can potentially pitch up or down wildly, up to a breakup of the a/c structure.

3.) The software rollback again protects against solar storm particles.

Questions:

A) Shouldn't Airbus have found this problem in a HIL Test rig under simulated solar radiation ? Particle beam accelerators do exist and are not expensive for a fleet of thousands of aircraft. Needs to be done once for each release and all a/c

Note: Control systems of this kind are typically programmed in Ada, C, C++ and execute on a RTOS like Integrity-178, VxWorks, QNX or the like. Unixes or Windows do not fit the bill, as they are not hard realtime capable. CPU could be an embedded version of PowerPC, ARM or 680x0.

OpenSSF warns that open source infrastructure doesn't run on thoughts and prayers

Nah

Just throttle the free download server to 100kbit/s and at the same time offer the paid download server at 100Mbit/s.

Apple Macintosh FOSS BOM

According to ChatGPT:

Darwin Core OS kernel (macOS is built on it; includes XNU kernel, BSD utilities, etc.)

XNU Kernel Hybrid kernel used in Darwin (part Mach, part BSD)

bash / zsh Default shells in Terminal (zsh is default as of Catalina)

OpenSSH Secure remote login (SSH access, used in Terminal)

rsync File synchronization and backups

curl / wget Command-line tools for data transfer via HTTP, FTP etc.

git Version control system, often pre-installed

Apache HTTP Server Web server, included but not enabled by default

Python / Perl / Ruby Pre-installed scripting languages (varies by version)

LLVM / Clang Compilers and toolchains used for macOS and iOS development

libc (BSD / FreeBSD libc) Core C library used by system programs

cups Printing system for Unix (used in macOS printing stack)

tmux / screen (user-installed) Terminal multiplexer, useful for developers

The Redhat Model

Simply do not give away the latest patches of your FOSS software for free. Also, have two tiers of download servers: a slow one, "free" and a fast one for paying customers. Stand up this service by means of Paypal or Bitcoin.

There are corporations who massively depend on FOSS, e.g. Apple. Negotiate a special support contract with them. They make more than $100 BILLION in PROFIT PER YEAR. Surely they can spend a few millions on FOSS here and there. A Macintosh essentially is 90% FOSS SW and 10% Apple-developed SW.

https://macrotrends.net/stocks/charts/aapl/apple/financial-statements#google_vignette

How I learned to stop worrying and love the datacenter

Nah

It's a nice toy for laymen, which means it is nice for quite a few clueless politicians and moneymen.

Nuclear Industrial Policy

Many countries, including Britiain, did not build new power stations for decades. When they restartet, enormous Learning Cost popped up. Which is reasonable if you think about re-training thousands of highly skilled workers.

The proper approach would have been a steady pace of nuclear-rebuild, one reactor unit at a time(start a new one each 3 years or so). But I guess all the attention and money went to the Canary Wharf set and their crazy contraptions.

Surrender Monkey Rhetoric

Has Margret Thatcher written this, out of her father's veggie shop ?

AIRBUS is world-leading in almost all ways from design to manufacturing and market share. It has been jump-started from public funds, but it operates like a corporation. It can hire at market rate and fire lazies and incompetents.

The same can work for IT, if actually competent decisionmakers would be financing and founding it.

http://afd-zg.de/IT_Airbus.html

Meanwhile, Hetzner cloud works very nicely, no need for foreign behemoth, thanks a lot. So do OVH, Strato and quite a few more. Then there are Linux, ARM, RPI, MaxDB, LibreOffice, GNUpg. All of it rock-solid technology in comparison to the cloudy security of the behemoths:

https://www.heise.de/news/Klatsche-fuer-Microsoft-US-Behoerde-wirft-MS-Sicherheitsversagen-vor-9674431.html

https://nhimg.org/microsoft-azure-key-breach

https://www.theregister.com/2025/09/19/microsoft_entra_id_bug/

Linux has the lineage to out-evolve the deadliest of cyber threats, given the right push

In Detail ?

Did they run a zero-trust policy in their network ?

Was it properly compartmentalized ?

Or was it sufficient to penetrate a single PC/server of the intranet ?

Did they have proper firewalling of DB servers and similar ?

Cyber Defense, Industrial Policy

Commercial IT systems are indeed still very weak. They are often developed using sub-standard methods such as informal+weak scanners, parsers and validators. This opens the castle to SQL and command shell injection attacks. Serialization has proven to be cheap+dangerous. Too many self-trained developers don’t know these basic computer science concepts, neither can they devise a proper syntax+grammar to the IT problem at hand.

https://di-fg.de/RobusteSoftware.html

Too often memory safety is not used due to inertia; it causes 70% of CVE exploits.

Microsoft had a memory-safe kernel in their R+D labs, but never made it a commercial product. It would have undermined the “secure Windows kernel” messaging…

The government needs to step up regulation to defend major industrial players. Red-teaming will also help to identify and plug dangerous weaknesses in industrial networks.

Government also needs to help out JLR, the same way they help out the bankers. What’s good for the financier is also good for the auto worker !

Alternatively, bow down to the Factory Of The World and lose the next conflict.

A Linux alternative? Debian/Hurd shows microkernel Unix dream is alive

Re: QNX on RPI

Well, you can always find the elefant's a$$, if you search hard enough.

But - QNX is a proven realtime OS, with lots of use cases:

https://www.firmenpresse.de/pressinfo210759-jeder-nutzt-qnx-30-beispiele-zum-jubil-um-der-firma.html

Also, it is Unix-like, which makes it interesting for all software engineers with POSIX experience.

Maybe you kindly look at the elefant's trunk and its phantastic capabilities ?

QNX on RPI

https://www.hackster.io/news/blackberry-s-qnx-seeks-hobbyists-and-makers-with-free-non-commercial-license-raspberry-pi-image-7c53320cac11

Looks very much like any other Unix, except that the compiler runs on the development machine, as the target usually is too small for a compiler.

"Niches"

Just because you are a datacenter engineer and have limited vision to the borders of PC and server technology, means little.

For an aerospace control system engineer, datacenters might appear dull. Full of non-realtime machines with enormous RAM, but questionable security. For him, the world revolves around Integrity 178, seL4, QNX etc.

In the automotive world, AUTOSAR Classic is the go-to mid size(In the order of 1MByte RAM and 1MByte Flash) OS standard. We use Windows as a development platform, but could easily do the same work on Linux or BSD, if the Vector tools were available there.

Parallel Program Execution Problems

You certainly need a proper theory/concept of parallel program execution. But there is no shortcut by either means of "shared memory" or "message passing". Both approaches have their pros and cons.

As always, aim for KISS, as overly complex approaches are at least initially hard to get properly running. Control complexity.

Then accumulate experience and you will become a seasoned engineer of parallel programming. Not really novel, the same you did(on a meta level) when you "learned walking" with serial programs.

Plus

Your security and freedom sometimes depends on secure microkernels, as your police, armed and security forces use them to communicate sensitive commands and intelligence. Which in turn protects YOU.

Re: Yawn...

There exist highly successful microkernels, such as seL4, QNX and Integrity 178. Some of them might run in your car.

Oh and Minix runs inside your Intel CPU, without telling you.

Re: Security Point Of View

Afaik, the seL4 correctness proof only claims that no memory errors can occur.

Wrong Assumption

With "sinking the ship" I mean "extract all secrets from the target system OR commandeer the system". Both of which might be catastrophic in banking, policing, defence and other security applications.

For example: an exploit in Bluetooth will only affect "black"(enciphered) data, while the "red"(plaintext) data exists only on the application+ciphering process. Red data is protected and all the attacker can do is to disable the Bluetooth stack at worst. After moving out of danger area, a bluetooth process/stack reboot will achieve availability again.

Example regarding "Commandeering", a Cash Machine based on Windows or Linux will eject all of its cash upon successful kernel exploit. A microkernel based cash machine will only stop working upon an exploit in the TCP stack, but it will not spill its cash to the attacker.

Re: Microkernels have a *much* smaller attack surface.

Its a very big reduction, by up to three orders of magnitude loc.

An exploit in Bluetooth, in IP stack, in a filesystem, in USB stack, in a device driver etc. no longer sinks the ship. The exploit is contained in a subsystem.

All of which is great, security-wise.

Re: Security Point Of View

So it is "just" 10 mio loc versus 10000 loc for the seL4 kernel ?

Does not change the character of my argument.

They Don't

Just because the "commercial IT world" has let Chorus down, does not mean microkernels no longer exist. There are huge spheres with much less publicity, which use microkernels. Starting with A380 and Integrity 178.

See my other posts.

No

Not a microkernel, unlike seL4, Integity 178, Mach, Chorus.

No

Mikrokernels have *much* smaller "effective" attack surfaces, easily three orders of magnitude less.

An exploit in the TCP stack sinks the Linux ship, but not a Mikrokernel. All you need to do on a Microkernel is to restart the TCP process and your secrets are NOT exposed. The worst effect of the TCP exploit is a TCP DOS event, which you might mitigate by firewalling.

See this chart: https://sappeur.di-fg.de/L4gegenueberLinux.html

One Example

"just" a Bluetooth stack is in the order of 300 000 lines of code these days. Waiting to be exploited by drive-by adversaries. It runs inside the kernel with full authority in Linux and Windows.

Very bad from an engineering and security point of view.

Re: ...and nothing of value was lost.

Beer helps ;-)

Integrity 178

An important mikrokernel for aerospace and special applications: https://www.ghs.com/products/rtos/integrity.html

No

There are operational reasons for microkernels, especially if a computer is connected to sensors and telecommunications networks. Soldier's lifes and your freedom, your security sometimes depend on it.

Re: something I have never quite understood

Macrokernels have an enormous attack surface and a single exploit sinks the ship of your computer. See my other post.

Security Point Of View

Microkernels have a *much* smaller attack surface. Thousands lines of code as compared the 40 000 000 of Linux(Windows similar or worse).

https://sappeur.di-fg.de/L4gegenueberLinux.html

A single exploit in these 40 Mio loc will sink the ship, as opposed to the Mikrokernel(seL4 here) frigate with plenty of compartments.

The L4 folks even tried to prove correctness mathematically, to a certain degree.

It's used for government and defence applications mainly.

(X11 included in my diagram, as it usually runs with UID=0)

SAP splashes €20B on Euro sovereign cloud push

Not Necessarily

Example: It is very hard to beat the price of an RPI, which can serve as a Terminal Computer or as a Web Browser Executor. The heavy work can be done on a Fujitsu SPARC server in the datacenter.

StackIT

No need for a history link, it is alive and kicking:

https://www.stackit.de/de/

Well

I guess it depends on the number of Cojones in a certain Berlin office.

When it came to the US fleecing Volkswagen and Bayer, there were none left.

It's also a matter of defence policy - if you want American soldiers to defend yourself in each critical situation - then you are beholden to U.S. demands.

Many Europeans operate exactly under this motto - America should do the fighting and all the dirty, bloody business. Like mowing the Iranian-Yemenite lawn or keeping a certain Euro tyrant in check.

Then the Europeans will be lecturing the Americans how bad they were, when they did the lawnmowing.

IT Airbus From Bottom Up

There already exist quite a few companies, products and systems to be used. They are not AWS-scale, but that might in many cases be an advantage. I am a Hetzner mini customer and they always respond to my questions+requests in time and with competence.

So if you are an IT decisionmaker, just give Hetzner a try. First for small projects, then midsize and eventually large-scale. Hetzner does have an API to spin up and down servers by shell script. Find out whether you like their technology and their customer support.

Then there are OVH, Ionos, Stackit/Schwarz IT, Deutsche Telekom and the British have their own companies. Give them a try, too.

Regarding IT Systems, there are plenty of non-monopolist alternatives to be used:

https://di-fg.de/IT_Airbus.html

Regarding SAP - they are a behemoth, but always anxious to never compete with Google, Amazon and MSFT base services. It can be seen in their rhetoric here. They were anxious not to create an alternative to Google Search and this smells of very much being beholden to foreign interests.

Says Who ?

American GigaCorp ?

One long sentence is all it takes to make LLMs misbehave

123

Test

FBI: Russian spies exploiting a 7-year-old Cisco bug to slurp configs from critical infrastructure

Because CISCO, Palo Alto, Microsoft and the like employ peabrains and rookies to implement their management(or other public-facing services like RDP) interfaces. All of this stuff should be locked behind a small, mathematically proven correct cipher system.

If you cannot trust your key routers and firewalls, how can you be sure there is no covert subversion on-going ? How are you going to detect the enemy operating inside your network, if he can commandeer your firewall ?

https://di-fg.de/MinimalesChiffrierSystem.html

Securing Telecom Routers And Switches

In future routers and switches, the entire management interface must be secured by minimalist cipher end to end. From network management system to network element.

Do NOT use SSL for this. 400kloc and impossible to prove correct.

OpenSSH is much better, but still 80 times too big.

Never expose complex and faulty SNMP, PHP webapps and the like. Lock it behind the minimalist, secure Cipher.

As long as we do not have this, prepare for at least three days of telecom network outage. It has already happened to a certain country, can happen to yours.

E2E Encryption

Routers and Telephone switches have a long history of being rotten. Never depend on them.

Rather, use a strong cipher:

https://di-fg.de/MinimalesChiffrierSystem.html

Like burglars closing a door, Apache ActiveMQ attackers patch critical vuln after breaking in

Indeed

Who runs a Message Queue with root (UID=0) permissions ?

Certainly not the sane ones.

Grow a new Arm: UK advisory body wants investment in local AI chips

Re: What’s the point?

+ Ubuntu

+ ARM

+ RPI

Three world leading IT projects. Glass half full ?

RPI based Netbook

Imagine how this could change the IT world and actually grow a nice business.

But that needs much more than electronics engineers. It needs manufacturing, designers, logistics, customer support, marketing, sales, finance.

Leading back to IT Airbus.

https://di-fg.de/IT_Airbus.html

It looks like the Davos set can no longer pull off what Franz Josef Strauß could do.

Should UK.gov save money by looking for open source alternatives to Microsoft? You decide

Wine

Many, if not most Windows apps can be ported to Linux using Wine.

Go Into Politics

So you think it can be done better ?

Then join a New Party to provide better solutions.

It is almost like an incompetent ancien regime can be voted out of office !

Large Linux Operations

Deutsche Börse - Large Scale Stock/Options Exchange

Google - Search Engine

Facebook - World's Largest Cat Pic Distribution System

There are probably 100 more examples of this kind.

MaxDB ?

I guess for many instances, it is a viable replacement for Oracle:

https://de.wikipedia.org/wiki/SAP_MaxDB