After deadly 737 Max crashes, damning whistleblower report reveals sidelined engineers, scarcity of expertise, more An Aviation Whistleblower report issued Tuesday by a US Senate committee cites numerous oversight gaps within the government and the aviation industry. The report [PDF] was produced at the behest of the Senate Committee on Commerce, Science, and Transportation in response to two Boeing 737 MAX crashes in 2018 and 2019 that …
For example, Michael Collins, a former FAA engineer, testified that FAA managers delegated 95 per cent of the certification of the Boeing 787 Dreamliner to Boeing personnel.
Also in pharma and pesticides "scientific testing" of safety is done by the manufacturing companies, by law. That is very unscientific.
Add the Federal Highway Administration to that list. These transformations go back to the mid-1990s where most of the Federal oversight capability was converted to a corporate facilitation function. In the case of FHWA, that went as far as forcing the States to adopt the same model. All compliance testing of highway/bridge/airport materials was transferred to the suppliers and contractors.
This is chickens coming home to roost.
There is nothing wrong with companies performing scientific testing of their products and then forwarding the results to the regulating agencies. Proper scientific tests will give reproducible and reliable results (that's the 'scientific' part in the phrase).
The real problem is fraud, cheating, and systematic cutting of corners - - with the government agencies firmly looking the other way because they are underfunded, understaffed and under pressure to facilitate the business of the industry they regulate.
There are two problems in pharma and other "scientific testing":
1) Studies with poor outcomes or "null results" rarely get published, so nobody knows that X didn't work or failed some or most of the time.
2) Some studies go fishing for (often spurious) links, massaging the data until they find something (anything) that vaguely fits and claims a result.
The solution is actually very simple, and cheap:
Publish all such trials before they start.
Full methodology of both the trial and the analysis that will be done - the entire paper except for the actual data.
The trial is then peer reviewed before it starts, and you are both required and guaranteed publication of the result of the trial.
Any positive trials that were not pre-published cannot be used to support submissions to the FDA (or equivalent) for certification.
If you don't publish the results within a reasonable time period, that's used as evidence that the thing doesn't work and/or isn't safe, depending on the trial that wasn't reported.
Re: 1) Have you ever tried to get null results published in a scientific journal? It is difficult (though I have managed at times, though that usually requires a proposed / tested / better alternative to be included.
And point 2) is just the natural extension of this, unfortunate as it is.
And in scientific papers the normal process does include a peer review. People try to actively poke holes into your argumentation. If they can think of any test to do on your data to test your hypothesis they will recommend (ok: demand) you do it (I'm talking about reputable journals, not the rubbish ones where you just have to pay up and only a very cursory review - if at all - is done).
However, this is not science. It is engineering. The tests themselves need to be designed properly, which is a science (hmm... a discipline at least) all by it self. Proper test design and proper testing following these rules - that's what you want.
There is, maybe it's only in the UK, a push for a register of research from some academics. This is so that null results can be located, either for their scientific value ( sometimes it's important to rule out possibilities) and to prevent distortions by researchers quietly putting negative outcomes to one side.*
The need for this has grown from the funding lead research pressures. Academics get funding based on their publications. As noted,Journals don't want to publish negative outcomes and no academic gets promotion based on not finding something they were looking for.
When it's commercial companies marking their own homework- testing their own medicines or engineering techniques this surely even more important.
*If I remember correctly highlighted by Ben Goldacre
Clinical trials, with at least an outline of the methodologies, are published before they start.
In general, organized scientific studies (at universities, etc.) go through proposal stage to receive funding (grants) and those proposals are evaluated. They are not necessarily published, but that's a decision for the organizations/people that put up the money, and failure will affect the researcher's ability to get funding in the future. At commercial companies this is even more pronounced (yes, I've done research in both academia and industry during my career).
You seem to propose something stricter to make it harder to "swindle" the regulators (FDA, FAA, you name it). Whether or not the specific proposal is foolproof, I think that this, as rightly highlighted by this article, has to be directed at the regulators: being a scientist or an engineer does not guarantee the highest moral ground beneath one's feet, so don't make swindling easy by allowing self-assessment.
And here the biggest problem (IMHO) has to be mentioned: regulatory positions simply do not pay as well, so the best brains tend to gravitate to the bodies that are being regulated. This is true for Boeing vs. FAA, this is true for Pfizer vs. FDA, this is true for Cisco vs. FCC, this is true for Goldman Sachs vs. SEC or Moody's - you name it. And, once again, brightness does not guarantee integrity.
.. and a fine line between "corrupt" & careless / administratively incompetent (good scientists are not always great at managing teams with a high pressure to publish atmosphere in place at institutes)
As a student on my first (non computing) degree, one of my "set" textbooks was a classic on molecular cell biology, one of the several authors was Nobel prize winner David Baltimore.
His reputation was massively tarnished and years later "improved" as he was initially regarded as being heavily involved in science fraud, and later became regarded more as not so much deliberate as not being on top of supervising work of his "staff" adequately, being careless on how stuff was written up / too supportive / lack of oversight of his staff .
Nobel was way before the scandal BTW.
https://archive.nytimes.com/www.nytimes.com/books/98/09/20/specials/baltimore-scandal.html
https://healthland.time.com/2012/01/13/great-science-frauds/slide/the-baltimore-case/
You might have a look at this
https://rumble.com/vqwdp6-how-many-more-adverse-effects-have-been-covered-up-during-the-trials-maddie.html
There you can see that we are missing basic ethical foundations. It is no longer about the fine points of testing, it is about deep-seated corruption due to monetary concerns.
Engineers marking their own homework is fine, so long as their short and long term prospects don't depend on the results.
That's the likely situation in circumstances where the whole company is acting rationally, ie the fundamental design is fresh, new and competitive, and low risk so far as passing regulatory hurdles is concerned. In that case its likely that marking one's homework is confirmatory, rather than damning.
Whereas if the company has selected to stick with the tired old design that has really hit the end stops, the engineer knows their job is doomed regardless of the support they get from the company. It cannot be safe and competitive. It can be safe and a market dud, or unsafe and not fly. The only way out is unsafe and fly to be competitive.
So I think that every single engineer in Boeing knew from the moment the management chose to stick with the 737 basic design that they were going to run the risk of hitting job terminating decisions.
The irony is that had management opted for the slower, whole new design, it'd have been just as successful in the market despite being late, and flying today probably problem-free.
I disagree. All of us are oblivious to our own blind spots, even with the best will in the world.
It's much worse given that Boeing is no longer run by engineers as with their glory days, but by the same manglement that flew McDonnell Douglas into the ground. (Pun intended.)
Whilst there's some truth in that statement, the reality is that MBAs and financiers have been in control of Boeing since 1971 when it nearly went bankrupt building the first 747s
As for "Why Boeing kept building 737s?" - there's a more prosaic and pragmatic answer - they TRIED to introduce replacements (7J7 and the newer program that's been dragging on forever) but airlines wouldn't buy them and insisted on 737s instead
This happened to the extent that the customers dictated newer engines on the same airframe - the 737Max was rather infamously announced by airlines before Boeing had even committed to making it
Had there been engineers in charge instead of MBAs, they might have been able to resist the pressures on rather obvious safety grounds. Instead Management dictated the introduction of the 737NG
It was also MBA management who allowed substandard airframes and defective parts to be installed in several hundred 737NGs, with the 2003 whistleblowers (stuff was being covered up on the assembly lines and signed off by management at least as as far back as 1997) being shopped back to Boeing by the FAA within a week and they were subsequently driven out of the company
737Max didn't "just happen out of the blue". It's the product of 40+ years of regulatory and management misfeasance predating McD manglement arriving on the scene
The decision between old and new wasn't informed by design and development costs but by airlines having or not having to do new type training and certification for crews. If you have to dish out for retrainig, you might as well start thinking about giving Airbus salespeople a call...
So I'm not convinced by that. Had Boeing said, "here's a new aircraft", it's not like the airlines can or could go off somewhere else and buy an alternative that is 737-compatible. Their choice would have been, adapt or go out of business. And Boeing could have sweetened that by offering something notably better than the A320neo family.
Going to Airbus was never, ever an option for all these Boeing operators, because there's no way Airbus could have fulfilled 8,000 orders (the approx total market size for single aisle aircraft). OK, that may not have been easy to predict, but that's how it has panned out.
One can design something like MCAS to work correctly. Just dont make the junior-mistake of depending on a single sensor.
All (sub-)systems which carry the life of humans must be analyzed regarding all their possible failure modes and likelihoods. This is nothing new, but established practice in auto, medical, aerospace and rail. There exist hard numbers and schemes such as ASIL Levels to do this systematically. Essentially, you assume that every component will fail with a certain likelihood (based on experimental data). From that, the functional safety engineer (yeah, a title) will calculate the likelihood of total system failure.
Certain failture rates are deemed acceptable (e.g. "one (or less) airplance out of 10000 airplanes can have a total flight control computers failure in 1000 years").
Based on these calculations, depending on single sensors will very quicky be ruled out. Experienced engineers will never depend on a single sensor.
Something went wrong in a very, very bad way at Boeing+FAA. They made a rookie mistake on the system level. I suspect Beancounterism.
"One can design something like MCAS to work correctly. Just dont make the junior-mistake of depending on a single sensor."
I'd also add to this: include some sanity checks.
Maybe the obvious ones like "don't let MCAS continue to trim the aircraft into an obviously undesirable extreme out-of-trim condition' and "if the pilot is trimming against MCAS, don't keep re-trimming against the pilot".
These are just two ideas off the top of my head that I would have thought would top the list of requirements (along with multiple sensor inputs). Not sure how Boeing didn't come up with these.
Because if they admitted MCAS was a change to the flight systems then they'd have to design it properly as part of the flight controls using the correct formal safety processes and train people on it. That becomes a new design and so there would be no reason why the sales people at the airlines shouldn't call Airbus.
What is happening is expected, but disgraceful. Employees are left in no doubt that they will be fired if they do not do a managers will, they do what is demanded while raising their concerns to the managers, and when the problems come home to roost the management are then pointing the finger at the very people who warned them.
It's the sort of corruption that you'd expect to find in a third world country and the fact that it not being stamped on probably surprises nobody, but it is disappointing as it shows a total lack of will to detect and prevent similar issues, which will continue occurring throughout our society despite the devastating impact it has on our ability to produce competing products with competitors (eg; China).
Having been a design authority for safety critical systems, I can say that there are certain specific procedures that must be used.
When designing avionics, the airframe company (Boeing) would normally send a set of requirements (and those can be pretty onerous), but in this case Boeing did not ask a couple of questions that would have prevented this whole thing (at least if engineering had been in charge, which has not been true since the bean counters of McDonnell Douglas took over).
Question 1. Can this piece of kit move a flying control surface. The answer here is either yes or no. If the answer is yes (and here it most definitely was) then it is designated by default as a flight safety critical piece of equipment (level A in the industry terminology). That requires a minimum of 2 independent channels operating at the same time*. Now it is possible that the engineers said ''Yes it is flight safety critical" but were ignored / overruled by management. For that to happen it had to go pretty high in the chain.
When Boeing was still run by engineers (as it was when the 777 was being designed) they specified a triplex flight control computer architecture (there are many things to consider even here**).
A level A design can be moved lower but only if it can be thoroughly shown that other systems can stop it from being dangerous. Boeing said (in the documents they submitted) that the amount of control over the horizontal stabiliser was minimal but the final product (which was not reported to the FAA) had full travel control (albeit a few degrees at a time).
The stabiliser has far more pitch authority than the elevators and so when the worst happened, it would be impossible to recover the aircraft if the stabiliser could not be moved (which was impossible to do because of other design decisions). Because of that, a move from level A was completely unjustified. As I said on a thread at the time, relying on a single sensor and a single computer *** was utter madness.
Question 2. If the answer to question 1 was no (moot here but I will go ahead with it), can this interfere in any way with any critical system?
There are more, but you get the idea. If there was any true justice in the world the managers and executives who pushed this through should be having a very long holiday at the expense of the USA in a nice building such as Leavenworth.
* 2 channels is acceptable if the system can alert the pilot to a malfunction and 'get out of the way' so the pilot can take control. This is fine for some equipment but implies that the aircraft is still manually controllable.
** In multi-channel systems even the processors in each 'lane' must have different architectures. Even though they are synchronised so they can vote there is a possibility that a particular architecture has a bug on the microcode; having 3 different architectures makes the possibility of a bug in the microcode at the same point in the program infeasible.
*** The MCAS had two computers and two AOA sensors****, but only one of each was used on a given flight. The equipment that was not used on a given flight would be used on the next flight.
**** AOA sensors are notorious for failing (sometimes intermittently) and even though they are used in other systems, the likelihood of all of them giving the wrong reading at the same time is very low. Apart from that there are many other parameters being checked which would show up a dodgy reading.
Note: If Boeing subcontracted the actual electronics (highly likely as they do not make their own avionics apart from things such as 'future flight deck') then I suspect there would be some very interesting correspondence.
They already had two sensors and two computers. Only SOFTWARE was missing to check both sensors against each other. Disable MCAS in case of implausible reading. And another little piece of SW to display the malfunction to the pilot.
What this means is that entire coporations can come into a state of deep dysfunction, due to corporate psychological failure aka. GroupThink.
There was no money to save from hardware and there prolly was not enough balls in the engineering side of the corporation.
"that they were going to run the risk of hitting job terminating decisions"
Given that two aircraft fell right out of the sky (and we can't even blame the Capissen 38 engine), I would think less "job terminating" and more "freedom terminating".
""fell"... that's a rather... passive description of what happened."
O.K. ...
Power assisted 'fall' with vital control surfaces 'locked out' by a very 'persistent' computerised flying 'aid'.
AKA Mass corporate manslaughter :(
"Engineers marking their own homework is fine, so long as their short and long term prospects don't depend on the results"
Actually it's not ideal, and is actually prohibited in some fields (e.g. electrical engineering, where implementation and testing have to be done by independent persons - at least here in Blighty).
Vested interest is not the sole, or even the main, problem. The big catch is psychological - if you make a conceptual error in design or implementation, you're unlikely to spot it when you come to test..
This post has been deleted by its author
"The big catch is psychological - if you make a conceptual error in design or implementation, you're unlikely to spot it when you come to test.."
Yup. This is why you have proofreaders too
The problem of regulatory capture in the USA is pernicious and widespread. The FAA is merely the most visible part of it in this instance
In a lot of cases it's moved from simple "marking one's own homework" to flat out corruption and using the regulatory agency as an anticompetition tool
For Grenfell Tower, an architect signed off on the fire safety without even checking. They stuck in some words copied from a computer program, and signed it, since "that's what everyone did".
They were not prosecuted.
If we actually wanted safe buildings, the best and easiest way to do that would have been to throw him in prison for 20 years for fraud and manslaughter. And then go back through building applications and throw lots of other people in prison for a month each for fraudulent statements. That would have made architects actually check the buildings are safe before signing off.
The important part of scientific method is peer review. For proper peer review, you need an outside organisation to test the product. Just saying "our results are reproducible and reliable" is not enough. Your own results are meaningless until someone else has reproduced them. That's the actual scientific part of the phrase.
But those approving the products are not the employees of the manufacturer seeking approval as it seems in the case of Boeing/FAA.
Safety studies etc are paid for by the manufacturer (the alternative could be the manufacturer paying the regulatory body to carry out the testing but as it's an open-ended process I don't see anyone signing up to that) and are generally contracted out to others, so it's a bit more separation there too.
Going by the article, the FAA if not equaly guilty, certainly holds some of the guilt for allowing Boeing free rein with their testing.
I always thought the point of a governing body was to govern, not fob off all the work to the outfit who benefits most from finding the testing all hood.
As it is the whole issue has just had a large plaster stuck over the wound to hide the scab.
"Going by the article, the FAA if not equaly guilty, certainly holds some of the guilt for allowing Boeing free rein with their testing."
Oh, absolutely! After the FAA budget was cut, they had to cut back enforcement. In this case, in order to stay in the game they transferred a lot of enforcement costs (and control) back to the manufacturer. I think it's a national shame.
And the FAA budget was repeatedly cut by many congresses over the decades. It's a long standing shameful national situation.
There are serious repercussions, potentially. One has to ask, is the USA a good place to do this kind of manufacturing? Arguably, it will take political reform of the USA before one can regain unthinking confidence in a body like the FAA, and thence in any products built under its regulatory regime.
If the Congress doesn't take these things seriously, doesn't find a way to give the FAA an appropriate budget safe from political meddling, we'll be right back here in a couple of decades, saying "This has happened before".
And if one starts thinking like that, why would you ever buy a Boeing product ever again? Probably the best thing Boeing can do is relocate to Europe.
The FAA is no longer regarded as a trustworthy agency by other agencies and their homework is being checked by other countries. It will stay that way for a long time
CAA (china) approval in particular is critical for Boeing. China is Boeing's single biggest market and without it they're sunk
What's of more impact for them than the 737Max debacle is how the change in attitude from external regulators has impacted the 777X testing/approval process - notice how this has been dragging out? - and a fine tooth comb is being pulled over the entire 787 approval process
America The Beautiful!
Free to screw with people's safety the world over in search of the almighty dollar.
Unfortunately the example has resulted in a lot of greedy people at the top following your example.
Run Boing, get fired for 2 crashes of planes your company badly built, and get 64 Million dollars. Fuck me!
Any system becomes a bad system once it has been compromised. And, unfortunately, no system humans will ever create will be immune from compromise. The key is for all participants to perform to clear and well understood ethical standards, and to talk about them frequently.
I've helped create aircraft instruments, after which I had to flip my hat around and help get them certified. I love writing software and doing systems engineering. I hate testing: It's boring as hell. I only want to do the testing until it passes, and to know that the pass is a good pass, not an accident or a statistical fluke. So I'm a testing asshole.
I worked with FAA DERs (Designated Engineering Representatives) to completely overhaul our testing environment, from the ground up. The equipment we used, the procedures we followed, the test artifacts we generated, the environments within which testing was performed. We used formal proofs when needed, but we much preferred to test our instruments to extremes in the lab, and with lots of flight hours on company test aircraft.
This was always part of the company culture: One of the first things said to me during my first interview was: "We are a test and certification company that happens to make aircraft instruments." Every new engineering hire, after getting up to speed on the product and its development, was then expected to make substantial and meaningful contributions to the test and certification system. I had experience in other safety-critical sectors, including nuclear power, and I was expected to bring all of that experience to bear on all of our processes, including test and certification.
We intentionally chose to not have in-house DERs. We always hired independent contractors, typically a couple junior ones and one senior-as-hell greybeard to get the best out of them. The advantage being that we could rotate DERs on each project, and learn from each other. Well, we never rotated that senior bastard: He was just too good at what he did.
QA is a much underrated function. I worked with the same colleagues for many years through several companies and these two despite being good friends (eventually) were the bane of my life because they used to take pride in destroying our creations. All for a good cause, of course, but note how these people, while critical, were also regarded as somewhat less than us 'real' engineers by the bosses. (Management thought that all they did was press buttons.)
Back in the 2000s our daughter was studying aeronautical engineering and did a summer interning at the 737 plant in Seattle. She told me a familiar tale, of the engineering staff being mostly entry level or older people who were either close to retirement or had retired and were working on contract. She had entertained thoughts of working for the company but the HR department, located in another state, wasn't interested (she graduated with what would be called in UK-speak a "double first" but at the time the business was going through one of its periodic cycles so she got sucked into the energy sector instead.) The hiring tale was all too familiar to me in my own branch of engineering, back then everyone was complaining about a chronic skills shortage but the voices were ignored while the profits seemed to be robust.
(The best thing Boeing could do is spin its aircraft making division off into a separate company and leave the financial engineers in Chicago to continue to do their magic with virtual profits. Engineering is hard and the result of failure can be spectacular. Its best left to the professionals.)
"That's why 3rd party led testing is a must."
Depends how good the testing is. Some of the Kingspan insulation panels used on Grenfell Tower were tested by BRE in the UK. But during the inquiry a BRE employee testified that the company had added fire retardant boards to the test rig - but that detail was not included in the test report, etc. So the test was unrepresentative of the behaviour of the insulation in a real fire.
Source: https://www.architectsjournal.co.uk/news/bre-technician-did-not-know-about-grenfell-insulation-fire-test-hustle
(Note: Kingspan actually only provided a small percentage of the insulation but matter does shed an interesting light on their attitude, as well as the quality of the work carried out by BRE.)
Getting independent testing groups up to speed is expensive and slow, and the sunk costs make it very difficult to change test vendors. What I recommend is in-house test performance with independent OBSERVATION and CRITIQUE.
Good, experienced external test monitoring and auditing isn't cheap, but it's both nimble and very effective. The auditors/advisors (DERs in our case) have their independence and industry reputations to consider. We've quietly "suggested" to DERs we let go (or decided not to re-contract) that they either up their game or consider another vocation. That's why we kept a very senior DER under contract: No junior DER is ever perfect, or even adequate, and we aren't the ones to make them better at their jobs. Hence the greybeard DER.
THIS! Oh, lawdy, did I love testing our shake table fixtures until our instruments started flying (without being installed on an aircraft). Proper fixturing for vibe tests can be modeled to death, but only "shake to failure" gives the data needed to validate the fixture.
Depending on a single sensor (or any other component) to make the potentially life-threating decision to steer the aircraft into ground direction, is a rookie mistake. There exist well-established methods of statistical analysis down to the level of single components such as a resistor, to calculate the total failure rate of a system. There also exist threshholds about total failure rates. E.g. see the "SIL" approach.
Single components, and especially exposed sensors will fail. Think of bird strikes, debris blown over the runway, cables corroding etc.
This type of problem must be caught by functional safety engineers.
It would also have helped to perform simulations with a faulty sensor, but that requires the intution that the sensor WILL fail, too.
The fix is also quite obvious: have a second sensor to check the first one and disable the system if sensor readings dont match. Signal problem to pilot or another strategy to work around the failure mode.
They couldn't do that easily. The rules say the aircraft must have certain control characteristics. The rules do that to try to make it easier to fly. With the new engines on the same frame, the Max failed that rule. They added MCAS so they could persuade the FAA that they comply with that rule.
Without MCAS, the plane would need significant changes to the airframe to fix the aerodynamics to fix the control characteristics. It would be effectively a new plane. It would require retraining the flight crew, too. That's a lot more expensive to design and test than what they did. It would also have been better and safer, but "cheap" won.
This is exactly what the customers DIDN'T want
The reason for flying 737NG/Max is that you can take any 737 pilot and slot him in the pointy end of any 737 with (at most) 1-2 hours reading the familiarisation manuals
This is WHY Southwest and others don't fly mixed fleets and WHY they didn't buy A320s and WHY they pressured Boeing so heavily to keep making 737s
As soon as a pilot has to be "certified" for another plane, he;'s either more expensive, requires hours in seat to keep the certification or less flexible in terms of scheduling (usually all of the above)
Everything that was done to the MAX (including leaving stuff out of the manual) was to AVOID the need to certify it separately to other 737s as far as the meatsack at the controls was concerned
The fix is also quite obvious: have a second sensor to check the first one and disable the system if sensor readings dont match. Signal problem to pilot or another strategy to work around the failure mode.
This, my good man, is why anyone in avionics and safety tends to rely on a quorate type system (and always with uneven numbers)... multiple sensors, returning the same type of data, so that in the case of a dud sensor, you don't get dud data.
If you have 5 sensors, and one has a moment (a bug got lodged in the pitot tube and obstructed it), you still have 4 sensors giving same data, and overruling the dud. Ditto for three sensors. But when you get to two sensors, or worse, one, you don't have that failsafe anymore, and that's why EASA has insisted that Boeing come up with a similar thing to its synthetic airspeed that the 787 uses in its systems as a backup to the backup to the backup.
:-)
There is the very real problem of managing multiple sensors. Sure, something close to perfection could be obtained by pricing a product out of the market. How do we ensure truly adequate safety with limited hardware?
There are two perspectives that may be combined to provide some hints.
The first, of course, is the entire field of Design of Experiments, a fundamental tenet of which is simply asking the question: "Are you ACTUALLY measuring what you THINK you are measuring?" This primarily affects repeatability of a test or experiment, but also applies big time to instrumentation.
The second comes from the extended discipline of Digital Signal Processing, specifically the notion of Sensor Fusion.
When combined, you can create what some call "Virtual Sensors". These are values derived from other values in a manner that is PROVEN to be completely independent of a sensor directly measuring the value. Mathematically, they are always inferior as their error is often the product or sum of the constituent value errors. However, even a shitty additional synthetic sensor is VASTLY better than no sensor at all!
In one case, I synthesized a truly lousy synthetic sensor that was accurate to only three states: "The Value Is Increasing", "The Value Is Decreasing" and "The Value is Steady Within 10%". That's it. Yet that single very crude (yet totally independent) value allowed us to massively improve the robustness of a vital subsystem, leading to awesome certification test results.
Another perspective is to understand how a requirement to "Always Show The Correct Value" is fundamentally and philosophically different from a requirement to "Never Show An Incorrect Value". It was to satisfy the latter requirement that my extremely crude 3-state value proved its worth: If the behavior of the displayed value and my synthesized value disagreed for more than 3 seconds, we would DISABLE THE ENTIRE DISPLAY to force the pilot to use a backup or reversionary instrument (including steam gauges).
We did not need to add a duplicate sensor within our instrument, and frankly we had neither the room, schedule, nor budget to do so. We only had to make certain we had the inputs needed to independently compute that synthetic sensor.
We did this with a bunch of values, some of which were industry firsts that we intentionally did not patent, instead keeping them as Trade Secrets. Though I've been gone from the company for a while, those are still covered under my NDA. So I don't talk about any of them in specifics, to avoid saying things I shouldn't.
Fundamentally, what we did was perform multi-variate statistical analysis on the entire streaming data set. The FAA has standardized multiple synthetic sensors, but those are simply physics formulas with the terms rearranged to isolate the value of interest. Our approach let us tease out some bizarre and surprising relationships and correlations, after which we would determine the "number of valid bits" present (a form of Shannon Entropy).
In the case I cited, we had just 2 valid bits, which we tweaked to give us 3 operationally useful states plus an active "failure" state (crafting "good" failure state bits deserves a totally separate discussion).
Like the FCC, the FAA also has the ability to protect intellectual property while still getting certifications done. First, the DERs were shocked, to the point that they made us prove our testing actually tested what we said it was testing (more Design of Experiments stuff). Once we had them onboard, we had to do it again with the FAA, who had red-flagged our submission package. The FAA visit to our facility fundamentally changed our relationship with them, in that they used our process as a template for how to do multi-pronged certification testing (theoretical modeling, simulation, lab tests, flight tests, and deep statistical analysis) without breaking the bank or taking years.
Our process directly helped other small aircraft instrumentation companies move their instruments from the uncertified Experimental Aircraft market (including the home-built market) to full FAA certification. We loved the competition, and even worked with one of them to add their FADEC technology to our autopilot product line. We certified their FADEC, which we both treated as a straight-across trade, with no money changing hands.
I come from the auto side of things and we cannot have large numbers of sensors, if anyhow possible.
So, two sensors are mostly OK:
[numbers guessed]
Rate of failure of one AA sensor: 1/100 000 [1/flight-hours]
Rate of failure of two AA sensors at same time : 1/100 000 * 1/100 000 = 1E-10 [1/flight-hours]
That 1E-10 is already a very "good" number.
Now, the really interesting number is
"Rate of double AA sensor failure with identical reading" = (rationally thinking, guessing) = 1E-13 [1/flight-hours]
That is prolly already small enough to be an acceptable risk according to methods such as (A)SIL or DO178.
This is how the auto folks think and it seems it is a quite reliable method.
"This is how the auto folks think and it seems it is a quite reliable method."
When you can pull a plane with 100pax over to the shoulder, let me know
In addition, let me know when such safety systems are regarded as adequate on trains or busses
Now lookup the Birthday Paradox for an example of why your assumptions are wrong
I've got to say one important thing: Having a rock-solid test and certification environment LET US TAKE RISKS and INNOVATE!
We did some really wild-assed hair-on-fire engineering for new products and feature development, then immediately (and confidently) slammed the prototype into an aircraft after it had passed our baseline ("Safe For Flight" or SFF) lab testing.
As an example, we needed longer full-operation backup power within one instrument. Our prior solution used a lead-acid gel cell (extremely safe and long-lived), but that approach was too heavy, too large, and had insufficient capacity for the new instrument. We didn't want to use Lithium-Ion batteries for MANY reasons (temperature profile, shipping, recharge cycles, etc.). We instead used ultracaps, and developed (and patented) entirely new ways to make them work in instrumentation and aircraft environments. Early lab tests literally burst into flames, so before initial flight tests (to pass SFF testing) we developed an all-new fire-proof insulated instrument enclosure (which was also patented).
This kind of rapid investigation, prototyping, development and deployment would be IMPOSSIBLE without ABSOLUTE TRUST in our testing environment, and our SFF testing in particular.
To be clear, a block of aluminum can be certified as SFF because it has few failure modes, every one of which can be tested to exhaustion. Our SFF tests merely ensured the instrument wouldn't kill anything outside of itself. SFF testing did NOT ensure the instrument would do anything useful!
Ironically, someone did suggest that maybe large capacitors would be a better (and less... flammable) alternative to using Li-Ion batteries for the 787 (or the next large all-electric plane).
I believe the A350 went back to NiCd batteries during test, but they are running with Li-Ion now (appropriately designed to not combust completely).
It sounds almost to me like there could be two procedural changes that'd help.
1) It really sounds like it'd be better if there was no liaison at all between the FAA and the engineers; engineering concerns are filed like bug reports, and the FAA could then be immediately aware if the engineers had concerns and how they were addressed.
2) It should be made clear to Boeing et. al's non-engineers that any given design is going to see millions of cumulative operating hours. I mean, the saying for saying something is unlikely is "Wow, that's a million to one chance", well in this case a million to once chance per hour means it's reasonably likely to actually occur sooner or later. You're still dealing with statistics (even a triple-redundant sensor or computer, with 3 seperate designs, has that very low chance of triple fault, but it gets to be statistically low.) It would probably be good to make sure the non-managers responsible for anything safety-related have a good handle of statistics.
Very few humans understand statistics, and risk is particularly badly understood.
That's partially an education problem, as maths stats tends not to be taught at all to the under 16s, by which time all the accountants and financial fiddlers have stopped doing any maths because it's "hard", and only engineers and mathematicians are left.
Thats a title and all safety critical R&D efforts must employ at least one of them to analyze failure modes and approve preventative measures (such as triple computers, double sensors, heavy software testing efforts etc).
There is nothing new here, except that Boeing apparently has managed to avoid the well established state of the art, with FAA and EASA helping them in the effort.
One of the problems I recall reading about is, the FAA does not have the budget to have so many actual engineers on staff. Furthermore, airplanes are getting to be more and more complex beasts, with newer and newer technology included. So the FAA will have difficulty finding experts in all these new tech as well.
And so many of those engineers who certify things for Boeing are on Boeing's payroll.
How long before an engineer who refuses to certify whatever Boeing wants is fired/retrenched/demoted/shifted to some other job by Boeing?
"IOW the FAA is not fit for purpose and FAA certification means nothing."
I would say it's inadequate. They do have some good people. More money would let them hire more people and more expertise. Making their budget less volatile (at the whim of whichever party is in power that year) would make the FAA a more attractive career choice for the sorts of people they really want to recruit.
The other difficulty is that the FAA is responsible both for regulating the aviation industry in the US and for promoting it. Those 2 missions sometimes conflict. The NTSB has a long list of safety recommendations that the FAA ignores. If some regulatory authority were transferred to the NTSB, things might change.
"It would probably be good to make sure the non-managers responsible for anything safety-related have a good handle of statistics."
It would also be good to make them personally legally responsible. The American way seems to be that the company can buy its way out with the addition of paying someone rather handsomly to be one scapegoat and throwing the other under a bus.
>well in this case a million to once chance per hour means it's reasonably likely to actually occur sooner or later.
I ran into this problem with the network testgear we were building. All networks have a bit error rate. This is usually very low, maybe one bit in a many billions or a trillion or more. No problem. Until you start working with gigabit or higher speeds. Then you get complaints from the management that simple loopback tests occasionally show a frame with an error in it. Then you try to explain "what's wrong with the firmware".
The manufacturer did not admit guilt, however, which would have prevented it from receiving future government contracts. Instead, it entered into a deferred prosecution agreement.
No company executive faces imprisonment for the misconduct that the biz has acknowledged. Boeing fired CEO Dennis Muilenburg in late 2019 over the 737 Max accidents and he departed with $62m in compensation.
Shame, shame and more shame.
No Boeing executive on the bench. No FAA bit fat asses either. Their behaviours result in the death of hundreds of people but the so-called Justice says it's fine.
This corruption culture is disgusting.
I will never fly in a MAX. The design of this aircraft is bad, and the people in charge who made it possible are criminals.
Last time I flew them, they operated a very nice A320, brand new.
Very sharp and nice looking uniforms and people. Hammer and Sickle still on the uniform - go figure.
Russian airplanes and crews do have their specific challenges - see the Superjet crashes.
Worse than Boeing ? Prolly not.
Aeroflot have one of the youngest fleets out there, with an average age of only 6.1 years (https://www.planespotters.net/airline/Aeroflot-Russian-Airlines). For comparison, British Airways have an average fleet age of 13.2 years and American Airlines 11.8 years.
Aeroflot have always been good when I've flown with them, they definitely don't deserve the reputation they have (maybe a couple of decades ago it was deserved). Also found Siberian (S7) to be good too. Russia still does have some major issues with its smaller airlines, but the main players are as good as any other major airline now.
The Superjet crashes look very "Russian" to me. In the first crash in Indonesia, the chief test pilot of Sukhoi decided that he could ignore the terrain warning system.
The recent crash in Moscow also does not display good piloting.
They also had several turboprop and a jet seaplane crash in Turkey recently.
So, they have their share of troubles.
737-800 is not a Max, neither is 737-800A
737-8 is a Max as is 737-7, 737-9 and 737-10, each with increasing length. These are internal codes that you find in the OEM documents and valuer data. These have never changed since day 1.
You have evidence they were used instead of Max in post crash documents vs pre crash documents?
737-8200 is a Max with 200 seats, the Ryanair 'you're cattle and loving it' version.
The majority of users don't care what they fly, unless there is a 3rd crash, then its game over for the Max.
Brand new trains, with broken speedometers and door buttons hanging off. They also had a bit of an Oops! in that spikes generated by the transition of one supply area to another tended to reboot several of the computers. Including the one that monitors the power supply for all of the computers... including itself.
What the whistleblower report, which is the subject of the article, actually says is instructive. Much of what is moaned about above here by our commentards has in fact been addressed. To take just a couple of quotes from the executive summary:
"The 737 MAX crashes ... called into question U.S. aviation safety oversight, presenting a historic challenge for U.S. policymakers. ... In response, Congress passed the Aircraft Certification, Safety, and Accountability Act, which was enacted into law on December 27, 2020."
and
"Whistleblowers perform a critical public service by exposing wrongdoing in the government and private sector. ... The Committee sought to honor these whistleblowers by addressing many of their concerns when drafting the Aircraft Certification, Safety, and Accountability Act. The law took the important step of extending Federal whistleblower protections, similar to those that were available to Federal aviation safety workers and airline employees, to employees, contractors, and suppliers of aircraft manufacturers."
One employee has been indicted and at least one Congressman has said that he expects others to follow. So the real question may turn out to be, can $62M hire a good enough lawyer to get you off the hook, when you are playing in the big pond?
Does that Act move the task of actually doing the work from the manufacturers' employees to the FAA? Does it fund the FAA sufficiently to enable it to have the appropriately qualified staff to do that work? Protecting whistle-blowers is solving the wrong problem. The right problem would be ensuring that they're not needed.
This post has been deleted by its author
See this summary for example:
https://transportation.house.gov/imo/media/doc/2020-12-20%20Aircraft%20Certification,%20Safety,%20and%20Accountability%20Act%20-%20Summary1.pdf
"This bipartisan, bicameral legislation strengthens the Federal Aviation Administration’s (FAA) aircraft certification process; ensures transparency, accountability, and integrity in FAA regulation of U.S. aircraft manufacturers; addresses issues identified related to human factors, automation in the cockpit, and international pilot training; and authorizes nearly $275 million over the next five years in robust FAA oversight and aviation safety-improving programs and initiatives."
I trust that answers your concerns.
As someone who's worked in the aviation data industry, with access to accident data, I've read stuff that would make your hair curl in terms of reasons for incidents.
Obviously, the primary reason is stupidity of workers/pilots etc - people - in many many case. Even when i entered the industry with no knowledge, I could tell something wasn't right.
I therefore would like the top job, and $62m, cos I'm clearly more qualified.
You will not find me getting within any boarding range of a 737-Max. That does mean that I'm less likely to fly on Ryanair in future. Thats not necessarily a bad thing either.
Airbus - 3 sensors - more cost - but a far more reliable quorum.
Boeing - 1 sensor - cheap - Donald Ducked.
When my car window had a short-out the polarity (hot-cold) of my heater controls was reversed. Weird things can happen when electrical parts go bad.
When one goes bad out of a pair then it's hard to know which is right. That's why you need 3 of them.
The same is true of piezo speed sensors (Air France from Brazil icing).
Loving all the comments.
I have a vague - so probably incorrect - memory that the space shuttle had five flight control computers, one made by a different company 'just in case'. At one of the early launches, possibly the first, there was a hold because the odd one out didn't agree... turned out it was right and the others were wrong.
p.s. flew into Brazil on that Air France plane; it crashed on the way back :o
That is correct - fail safe as in "we lose one computer and still have triple redundancy. The fifth computer by a totally different manufacture (Lockheed, if I remember correctly) was to guard against software errors that would cause the 4 IBM computers to fail identically. During one count down for launch the Lockheed computer refused to synchronize with the data bus. It was correct and the 4 computers were wrong. Rebooting and things were good (analyzed later - 30% chance of failure but once booted correctly there was no problem).
The Lockheed computer was limited to return the astronauts to Earth.
Turns out there was an assumption that there would be no tasks running at boot time during initialization. Later there was a code modification that invalidated this assumption.
This is a great point for debate .... well ... not 1 vs 3 ... that is a no brainer ...
But 2 Vs 3.
its been going on for 20yrs in the rebreather diving community.
We have galvanic O2 cells in our closed cuircut rebreathing loop. They regulate the partical pressure of O2 in the loop. Too little, we die ... nicely. Too much, we die ... horribly. So it does focus the mind.
Problem with O2 cells is they are desiged for surface pressure, medical equipment. Not 5-10atm being halled around a boat in the north sea full of salt water. So they fail, regularly.
So, early manufacturers started putted 3 cells in (at great expense) and used simple voting logic to decide what our O2 reading were. Trouble is ... cells could and would fail at the same time due to being in same batch and subject to same conditions ... therefore you get 2 duff votes, but as they agree, you still die. DOH!
Poseidon have come up with a new solution .... and only uses 2 cells. Not for voting, but for redundancy. In their design they validate both cells against konwn conditions thoughout the dive (every 3mins) by blowing pure O2 over them an watching the electrical responses. If readings are good, you continue your dive, if either cell misbehaves, it is ignored and the good cell is used to keep you alive while you start your dive abort process. (If both fail, its kicks you off the breathing loop to a completely redundant Open Cuircuit backup ... a 2nd plane if you will!)
Sorry for going on ... but I do like Poseidons approach and its the unit I dive. It would be interesting to work out a way if Piezo tubes could somehow be re-designed to allow realtime validation against a known state mid flight? (James Bonds Q style gagdet comes out of plane and covers the tube completely and uses compressed air at set pressue to mimic a given air speed) .... as you can tell, I more at easy under water than above it ;-)
Good debate thou!
Cheers!
"It would be interesting to work out a way if Piezo tubes could somehow be re-designed to allow realtime validation against a known state mid flight?"
I wonder if you could use a GNSS-derived speed for validation of the pitot tube output. Now, I know the aviation industry doesn't like to rely on GNSS for various reasons, but it struck me this might be a valid application. Must ask my pilot friends, after the hols.
I wondered that too. The GPS and Inertial nav systems can both provide a ground speed readout and with the airspeed data from before the pitot tube* fails can also provide a wind speed and direction.
To my mind what should happen if the pitot fails is the autopilot uses the GPS/inertial data to continue flying while alerting the pilot that he ought to get ready to take over. If the ground speed changes too much from what's expected with the last known wind and engine settings hand over to the meat sack.
This has the advantage of minimising the startle effect of the aircraft suddenly throwing you control and telling you what's wrong at the same time while your brain struggles to get up to speed.
*There are multiple on airliners but I'm not sure what the fallback logic is.
"use a GNSS-derived speed for validation of the pitot tube output. Now, I know the aviation industry doesn't like to rely on GNSS for various reasons, but it struck me this might be a valid application."
The problem is that GNSS measures a different thing than pitot tubes do. Pitots measure speed of local airflow: the medium in which the aircraft flies. GNSS measures geographic location. They are used for different applications.
A poor analogy would be road conditions versus location. I'll navigate with GNSS/GPS, but I'll drive by looking out the window and feeling the road. The difference, of course, is that when a car hits a road hazard you steer to the verge and stop. When a plane hits a hazard it's more exciting.
> I wonder if you could use a GNSS-derived speed for validation of the pitot tube output. Now, I know the aviation industry doesn't like to rely on GNSS for various reasons, but it struck me this might be a valid application. Must ask my pilot friends, after the hols.
Sort of: GNSS gives you ground speed whereas pitot tubes give you airspeed. However, my pilot friend says in the event of loss of airspeed indication the first thing to do is change to flying straight and level. So apply 3/4 throttle (or whatever the setting is for your plane - and you should know) and you can be reasonably confident that you're not going to stall. You then have time to assess the situation and go through the checklist.
The regulators could of course, employ - on a part time consultancy basis - retired Airbus staff to QA Boeing ... and retired Boeing staff to QA Airbus ... then sitback and enjoy the sparks! Rotate them around regularly to avoid over familarity ... and retired European engineers woudl get to spend some time states side, and visa versa! Paris is nice in the Spring? ;-)
[Tongue firmly in cheek here!]
The main concern I have with the MCAS design is that it took command and control of the aircraft away from the pilot. Initially, the pilots weren't even told this was possible, then one crash happened while pilots unsuccessfully fought the machine for control of their own damn airplane.
Never mind one vs three, or "it was a software error", or "Airbus did it sooner and differently".
The FAA (who lost all international credibility in certifying the 737-MAX) and Boeing (who lost a lot of money) let a few thousand lines of software run an airplane. Into the ground. Twice.
THAT was the failure.
Reading this, something occurs to me...
So this system was freaking out because the single sensor had malfunctioned.
Why the heck wasn't it also noticing readings from the altimeter? Because pitching the plane nose down to level out the flight doesn't correspond with nine hundred metres, seven hundred, five hundred, oh shit.
Additionally, if the system is going to be controlling the ailerons, it must have some way of sensing the actual current position of the ailerons to know how to move them. You can't be flying in a straight line if the flaps are in the "descend" position. It just isn't logical.
So it seems that there should have been sufficient telemetry from elsewhere to inform the system that things were going really seriously wrong. Why was none of this information considered in tandem with the sensor readings?
Additional information wasn't considered because that would have required a more advanced computer than whatever legacy system they were using to avoid having to re-certify the aircraft and increase the training required for aircrew who'd flown the previous model.
I disagree--my understanding is the Boeing strategy was a sales strategy. The customers did not have to recertify the pilots thus reducing cost of ownership and eliminating the training cost of introducing a "new" aircraft model.
Remember, the -MAX did have to be recertified, which is one of the causes of the disasters, because the recertification process failed.
I seem to recall there was also a little red warning light for MCAS input fault that they wanted to see as an optional extra... I mean... come on! It's not like TPWS... well, if TPMS contributed to the driver assistance systems like automatic lane keeping or ABS or something then I'd expect a warning light if the measurement system detected an anomaly.
Your right it was the pilot re-training that had to be kept to a minimum, I think it was basically a self taught power point presentation for the difference.
However there are degrees of re-certification, i.e. just the bits you've changed to the entire aircraft. I'm fairly sure MCAS was a minimum change in order to avoid more re-certification, which would have meant they would have had to have a comprehensive training package. Hence not having multiple inputs that would require a new computer.
....but everything to do with McDonald-Douglas management.
When Boeing and McDonald-Douglas merged in the late 1990's Boeing was the bigger of the two but McDonald-Douglas management had a well earned reputation in the industry of being the most vicious back stabbing bastards in the business. Which says a lot. Whereas Boeing was still very much an engineering / pilot driven company with equally easy going management in comparison.
Once the McDonald-Douglas sharks entered the management pool the Boeing people did not stand a chance. Almost all the upper and senior people gone with in a few years. And within a decade the McDonald-Douglas people had driven out almost all the senior Boeing aviation expertise people as well. Almost 80 years of corporate engineering and safety memory gone in less than a decade. All of it.
So by the time the Max project rolled around it just became the next in a long line of engineering debacles under McDonald-Douglas management. The bean counters said bolt big engines on a forty year old air-frame and there was no one left in Boeing who would tell the management - that will kill people.
Which it has. Many hundreds of them. And killed the civil aviation part of the company. Just like the McDonald-Douglas management did with McDonald-Douglas. For exactly the same reasons.
I have heard this story before, and I do believe it has a measure of validity.
However, the merger was nearly 25 years ago. How many of the McDonald-Douglas & Boeing heads from that era are left at the company?
WIkipedia shows there have been five different CEO's since the merger. Only one of them (Stonecipher 2003-2005) ever worked for McDonald-Douglas. Muilenburg, an engineer by trade, was the CEO during the development of MAX and was at Boeing his entire career.
No doubt there were certainly some cultural influences resulting from the merger - that happens with every merger. But attitudes come from the top, and history is now showing that to be a big influence across Boeing's mindset. I am just not sold on it being the result of a merger a quarter century ago.
Besides, who is to say Boeing would even exist today if it had stayed under control of an engineering culture? The company might have ended up defunct by now falling victim to 9/11, 2008/9 recession, 2020 'rona, etc. Ultimately this what-if game of finger pointing could go on all day long and change nothing about the facts of the present.
It's a question of company culture.
Before the merger, the culture at Boeing was security first.
Then McDonnel-Douglas took over and it became money first.
And in today's climate, there is little chance that that will be reversed. At best, we might enter an era of PR first, in which decisions will be made on how much PR risk it might cost the company.
In time, we might get back to security first, but the death count along the way will be dreadful.
I worked with a whole bunch of ex Boeing people in Seattle around the time of the merger. Fantastic software engineers and Triple E's. So I got to hear a lot of the company gossip. Seattle was still very much a Boeing company town back then.
Before the merger there was a truly stagger depth and breath of experience going back many decades. There was still WW2 vintage people kicking around as consultants. The technical people tended not to fully retire back then. Aviation was in their blood.
The first thing the McDonald-Douglas top guy did was move the HQ to Chicago. The main reason was to clear out whole layers of Boeing upper middle management. The Boeing lifers who had build all the great planes in the previous decades. Almost none of them took the relocation. I heard 20 out of 400 plus.
Whereas the McDonald-Douglas people from St Louis all jumped at the chance to move to a far better place to live like Chicago. So top level management was soon almost all McDonald-Douglas people who treated the senior technical management like garbage. Who either retired or left. And all the old timers who just loved to hang about and add their immense knowledge, they soon drifted away too due to the toxic company environment. Everyone now hated their job.
So all the people and company knowledge that had made Boeing what it had been, a company that had built great aircraft , was gone in less than a decade. And as you can see in companies like IBM, HP, AT&T etc once a great company culture is lost its not coming back. The IBM that existed from Tom Watson Senior to Lou Gerstner was gone by 2010. The HP of Bill Hewlett was destroyed by Carly Fiorina in the 1990's never to return. And AT&T is now just a brand name for a bottom feeder conglomerate based on SWB. Bell Labs long gone and whole bunch of incredible engineering expertise all gone.
So yeah, the slime-balls from McDonald-Douglas killed Boeing. You should read up on the various McDonald-Douglas scandals during their civil aviation era. Those killed many people too.
Having worked for IBM for 30 years as an engineer and as a manager when it was the "IBM-family" and living in Silicon Valley with many friends working for H-P in the days of the "H-P way", I can only second the comments about the new corporate culture.
"bolt big engines on a forty year old air-frame and there was no one left in Boeing who would tell the management - that will kill people."
The aircraft, including engines, is basically fine. The problem was that Boeing's target market was airlines who already fly previous models of the 737 and who would buy the new model as long as they didn't have to re-train* their flight crews. That was the reason Boeing added MCAS: to change how flight control surfaces behave so that the same pilot inputs produce the same aircraft behavior as with previous 737s. And Boeing really screwed up MCAS hard.
I completely agree about McDonnell-Douglas management. What a goat-rope.
*"re-train" is an understatement. Not only must pilots get a differences course on the new subtype, but they might not be permitted to switch back and forth between different subtypes. "It's just another 737" is a powerful sales argument for Boeing. That's the altar at which they sacrificed safety.
There was an exceptionally educational technical discussion on pprune at the time of the aerodynamics of the Max and why MCAS was introduced. And it was all to do with that the engine was way too powerful for the 737 wing config / cog. MCAS was not so much to make it have the same flying characteristics as previous model of the 737 but to actually keep it stable in flight. Having the same flying characteristics as previous 737's was a lesser technical problem. The planes crashed precisely because of theses unstable characteristics which MCAS on those flights failed to correct. Due to the pilots being completely unaware that it was there and what it did despite their flight control inputs.
One participant in the pprune discussion mentioned spending some time in the sim setting up an equivalent scenario and describing what happened. Pretty close to what happened in Ethiopia. Those poor guys did not stand a chance. The scary thing was that even though the guy in the sim did know about MCAS he still found it impossible to recover most times. The previous occasions when pilots were able to recover was blind luck and nothing else.
The really criminal part of the whole story is that MCAS and the way it worked was a major change and in all previous cases with this kind of change to a range model the pilots would have done quite a few hours in the sim to familiarize themselves with how it worked. As you mentions it was really a new sub-type. But the Boeing sales people had promised the big buck airline customers that it was a no-change no-retraining model (and there were big discount penalties for some customers if it was) upper management rammed though the Max knowing full well just what the risks were. They were told. Some of them should be hauled up on at least a man-slaughter charge for what they did.
And the irony here is that there was a 737 replacement project proposal kicking around for years at Boeing. As a modern competitor for the Airbus 320. Even back in the 1990's. But the bean-counters decided they were just going to churn out more and more variation of the 737 so they would not have to go through the whole certification's process for a whole new aircraft family and with the Max that short slightness reached its inevitable and very expensive conclusion.
The problem is that no matter how many band-aids they stick on it the Max should never have been built in the first place. I know for me the only reason I would consider one safe to fly in is only due to the skill of the pilots, not the stability and reliability of the aircraft. Which might be a problem with airlines whose company policy is for pilots to do the bare minimum of manual flying.
"There was an exceptionally educational technical discussion on pprune at the time of the aerodynamics of the Max and why MCAS was introduced. And it was all to do with that the engine was way too powerful for the 737 wing config / cog. MCAS was not so much to make it have the same flying characteristics as previous model of the 737 but to actually keep it stable in flight."
Thanks for that. PPRuNe has some serious techs; I'll look. Always happy to learn!
"But the bean-counters decided they were just going to churn out more and more variation of the 737 so they would not have to go through the whole certification's process for a whole new aircraft"
Nope. It was made clear to the beancounters that airlines would not buy 7J7s and what they wanted was a newer 737, so the beancounters overruled safety concerns and "made it so"
This predates McD arrival. The rot had already set in. McD just cemented the deal
The counterpoint to this is that the 737NG predated McD being on the scene and THAT airplane should never have been produced either (It was produced by accountants/management, demanded by customers and then the engineers told to make it work)
The bodging and covering up of dangerously substandard fuselage ribs on 737NG production lines also predated the arrival of McD on the scene
There's confirmation bias in the assertion. The truth is that Boeing was already substantially along the road that the import of McD manglement took and if it hadn't been, they wouldn't have been able to drive the company to the extremes that happened
Boeing's woes trace to the introduction of the 747. Massive restructuring of debts in the late 60s resulted in banks and financiers being in charge by 1971. After that the safety/engineering culture was steadily eroded and with or without McD along for the ride something similar would have eventually happened sooner or later (perhaps later, perhaps sooner. it's an unknown) - facilitated by the widespread regulatory capture environment in the USA (not just the FAA - the FTC and PUCs are shining examples of it still in action today)
In the RAF, I am informed that only a qualified engineer can sign a certificate of airworthiness for any aircraft. His, or nowadays her, commanding officer is expressly forbidden from ordering any engineer to sign such a certificate (probably a Court-Martial offence). Which is why anyone in an aviation company should be forbidden by law from ignoring an engineer's professional opinion on airworthiness or danger. Let's remember that had the engineers not been overruled, the Space Shuttle 'Challenger' would not have blown up.*
Richard Feynman's description of his involvement in the disaster investigation shows engineers being overridden by politicians and some curious ideas of reliability:
"It appears there are enormous differences of opinion to the probability of a failure with loss of vehicle and of human life. The estimates range from roughly 1 in 100 to 1 in 100,000. The higher figures come from working engineers, and the very low figures come from management. What are the causes and consequences of this lack of agreement? Since 1 part in 100,000 would imply that one could launch a shuttle each day for 300 years expecting to lose only one, we could properly ask, "What is the cause of management's fantastic faith in the machinery?" "**
The idea that an aircraft that has engines fitted that significantly change the location of the centre of gravity does not need to undergo full airworthiness tests and assessment seems to me more than strange, but I guess that there will not be a Nobel laureate on any investigation panel, so they will be spared the embarrassment of a clear and eloquent explanation of the actual problems.
* https://en.wikipedia.org/wiki/Space_Shuttle_Challenger_disaster
** Opening paragraph quoted from Appendix F "Personal observations on the reliability of the space shuttle", as reprinted in 'What do you care what other people think?' by Richard P Feynman, ISBN 0-04-440341-0
Almost all of the executives that drove McDonald-Douglas and subsequently Boeing into the ground were alumni of General Electric and former CEO Jack Welsh. His alumni have wrecked several other US companies as well, most notably GE itself. When "financial engineering" replaces real engineering, bad things (eventually) happen, like at IBM.
The Ethiopian pilots left the throttle at takeoff thrust. This is why the aerodynamic forces were so high. Further, electric trim was available but they didn't use it to override the computer. To be fair to the pilots, panic can happen to anyone. We weren't there. Would we have been able to avoid the panic trap and save those passengers? We will never know.
As for the Lion Air crash, the flight immediately before the accident had the MCAS problem. Those pilots used their training to override the computer and land safely. The problem was reported but never fixed. As we know the next pilot didn't override the MCAS. We will never know why one pilot responded correctly but the other didn't.
Take your shots at this post but please read beyond the headlines first. The details suggest that pilot training was an issue in both accidents.
Look up the Swiss Cheese model for accident prevention.
Does Boeing own a bunch of big holes in the cheese? Yup! But there are a bunch of other holes which all lined up to allow the accidents.
The involved pilots are one of them. Pressure from airlines (ie: Boeing's customers) wanting to minimize pilot training & maximize pilot cross-certification is another.
There is plenty of blame to go around. And Boeing protecting its customers means they have to take this black eye with no rebuttal.
Boeing manglement has repeatedly tried to shift the blame to the pilots, including making slurs about their level of competence, to avoid taking responsibility for their profit before all else corporate ethos.
Remember there was deliberately no mention of MCAS is any of the Technical Publications, would any sane person reduce engine power if the flight control system was constantly trying to fly them into the ground for unknown reasons?
"The Ethiopian pilots left the throttle at takeoff thrust. This is why the aerodynamic forces were so high. Further, electric trim was available but they didn't use it to override the computer....the Lion Air crash, the flight immediately before the accident had the MCAS problem. Those pilots used their training to override the computer and land safely....one pilot responded correctly but the other didn't."
No. From NTSB report pp 2-3 https://www.ntsb.gov/investigations/AccidentReports/Reports/ASR1901.pdf
During the preceding Lion Air flight on the accident airplane with a different flight crew, ...a 10-second automatic AND [Aircraft Nose Down] stabilizer trim input occurred, and the crew countered the input with an ANU [Aircraft Nose Up] electric trim input..captain moved the stabilizer trim cutout (STAB TRIM CUTOUT) switches to CUTOUT. He then moved them back to NORMAL, and the problem almost immediately reappeared. He moved the switches back to CUTOUT. He stated that the crew performed three non-normal checklists: Airspeed Unreliable, ALT DISAGREE (altitude disagree), and Runaway Stabilizer. The pilots continued the flight using manual trim until the end of the flight.
...Similar to the Lion Air accident flight, a 9-second automatic AND stabilizer trim input occurred after flaps were retracted and while in manual flight (no autopilot)...the pilot flying, partially countered the AND stabilizer input by applying ANU electric trim. About 5 seconds after the completion of pilot trim input, another automatic AND stabilizer trim input occurred. The captain applied ANU electric trim and fully countered the second automatic AND stabilizer input; however, the airplane was not returned to a fully trimmed condition. Cockpit voice recorder data indicated that the flight crew then discussed the STAB TRIM CUTOUT switches, and shortly thereafter DFDR data were consistent with the STAB TRIM CUTOUT switches being moved to CUTOUT.
However, because the airplane remained in a nose-down out-of-trim condition, the crew was required to continue applying nose-up force to the control column to maintain level flight. About 32 seconds before impact, two momentary pilot-commanded electric ANU trim inputs and corresponding stabilizer movement were recorded , consistent with the STAB TRIM CUTOUT switches no longer being in CUTOUT. Five seconds after these short electric trim inputs, another automatic AND stabilizer trim input occurred, and the airplane began pitching nose down.
RF Burns,
It is a cheap shot to blame the pilots, particularly as it feeds the narrative that 'Foreign' pilots are inferior in skills and training.
This excuse comes up time and time again.
The fact that is missed, somewhat conveniently, is that the aircraft acted in a way that was not expected based on known possible fault scenarios.
What wasted time was trying to define/decide the cause of the fault when an unknown 'hidden' system was repeatedly fighting against the pilot, an unknown factor that made solving the problem permanently impossible, based on current knowledge known to the pilots.
Boeing built a 'ghost in the machine' that killed all those people.
There is no excuse possible.
Runaway trim is runaway trim regardless of the cause. Pilots are trained to deal with it. The controls for doing just that were functioning but not used.
Saying "the computer did it" or "Boeing did it" ignores the facts and exposes some bias that isn't helpful. Are pilots relying too much on automation and giving up too soon when things get complicated? This question may be more important that blaming either pilots or Boeing.
BTW are we angry at Airbus for Air France 447 or did the pilots get the blame? Where's the outrage over that one??
"Runaway trim is runaway trim regardless of the cause. Pilots are trained to deal with it. The controls for doing just that were functioning but not used."
Pilots are trained to deal with runaway trim by running checklists to try various methods to regain control of the aircraft. Which method is effective does depend on the cause (and a novel cause might not be solved by the checklist.) The initial NTSB report showed the ET pilots took steps that correspond to the runaway trim checklist.
RF Burns, you keep claiming the incident pilots acted improperly. Please provide your sources. I gave mine.
Neither sets of pilots responded to the MCAS failure in accordance with training since there was no training about MCAS - the pilots were not even aware of the system; a deliberate aspect of the Boeing "design" to avoid pilot training costs.
One set of pilots got lucky freelancing a solution to the problem, the others (and their passengers) did not.
To suggest that this is anything other than a failure by Boeing and those responsible for certifying the system as safe is a travesty.
Behind the headlines: <https://www.afacwa.org/the_inside_story_of_mcas_seattle_times>
Please read more about these accidents. Pilots are trained to deal with runaway trim. The fact that it was caused by MCAS doesn't change the equation. If they had used the trim switch on the yoke it would have overridden the computer.
After the trim is set they could have stopped runaway trim by using the switches on the center console. Of course, it is easy to say this while sitting at my computer. In the heat of the moment it might be easy to forget the training. Sadly, that is probably what happened.
> If they had used the trim switch on the yoke it would have overridden the computer.
I read in one of the analyses that MCAS automatically re-enabled 5 seconds after releasing the yoke trim switches. So the pilot thinks they've corrected and five seconds later the plane is trying to kill them again.
The only way to disable MCAS on the 737MAX is to disable electric trim via the yoke switches -- there are cutoff switches for this on the central console. Doing that leaves only a manual trim option, winding small wheels down by the pilot and copilot's knees that drive metal cables that run all the way to the aft stabiliser. It takes a lot of turns of these wheels to correct a runaway trim, a lot of effort and both crewmembers are needed to wind the small wheels if the stabiliser is under heavy aerodynamic load.
There's a manoeuvre pilots can carry out to unload the stabiliser and make it easier to trim mechanically but it requires a lot of height since they have to put the nose down deliberately while trimming. The Ethiopian Air flight had just taken off when the MCAS cut in and trimmed the plane nose down, the pilots didn't have enough altitude and time to carry out this manouevre even if they knew it was necessary.
Total garbage. And a terrible slur on the dead pilots
I base that opinion on the genuine shock in the pprune discussion thread after the accident by pilots with 737 rating with many 10K's hours when the full details of what exactly MCAS did to the flight controls reposes. It was a collective, it does what?. You cannot be serious. Yet Boeing claimed it did not need new sub-type training for those with other 737 rating..
All previous MCAS failure recoveries got lucky. They had altitude and therefor time to recover. Or happened to press the right override switches in the right order immediately before going thought the checklist. In the Lion Air case the pilots had neither. They were in a catastrophic situation for second 0 of the incident. Because of decsions Boeing made.
Someone else mention the Swiss Chess Accident model. In this case Boeing provided a whole bunch of holes precut in the Max and it only needed two more for an accident to happen. Which are terrible odds for a pilot.
I'm very familiar with The Children of the Magenta Line problem. And how it degrades the skill set on the flight deck. But its always a safety trade off. And given the history of pilot error before automation its a difficult call.
With regards to the implicit attack on the lack of training of non western pilots I have only two replies. Air France AF447. And Air Canada 759 at SFO. In the case of the Air Canada near miss, less than 4 seconds from from a disaster worse than Tenerife. There were over 1000 passengers total on those planes. On the runway and on the aircraft landing on the wrong runway. The coverup by the Air Canada pilots, deliberately letting the cockpit voice recording time out, was a criminal act. Pure and simple.
That's superior western pilots for you.
Although Sully rightly gets kudos for a fantastic piece of flying during an emergency the greatest civil aviation pilot bar none was Carlos Dardano. A Salvadorean who started as a bush pilot.
https://en.wikipedia.org/wiki/TACA_Flight_110
Who dead stick landed a 737 on a levee surrounded by water in a severe thunderstorm. And did such a great job landing the plane that the aircraft was flown out later after two new engines had been put on. There was one minor injury.
You will find great pilots everywhere. And terrible ones too. And what I have learned from reading discussions on pprune over the decades is a great respect for what all pilots do. In seats 1A and 1B.
"Those pilots used their training to override the computer and land safely."
The flight before the Lion Air crash had a third pilot riding in the cockpit and HE was the one who saved the aircraft by noting the oddities. The guys wrestling with the controls were too busy trying to cope with situational overload to figure out what had happened
If you think this is the extent of corporate capture in the transportation world, allow me to introduce you to the Federal Railroad Administration, which has been steadily hiring inspectors from the ranks of failed railroad managers since at least the Bush2 years.
Now we have Positive Train Control and increasing decision power is being taken away from operating crews while trains are still hauling thousands of tons of stuff like chlorine and propane and assorted chemicals that aren't allowed to be transported on highways, often traveling through and being stored in highly populated areas.
Not only do those products present a hazard, but the question of whether trains could run in the future if the GPS system in knocked out is arising. Purely coincidentally, not long ago a certain adversarial nation decided to demonstrate quite casually that they could destroy a satellite with their current tech.
Has there ever been a merger where the good guys ended up in charge? They all seem to end up with the very worst at the top. Almost like someone planned it that way.
I was listening to the BBC's Reith lectures on Artificial Intelligence. I was wondering whether any AI system which controls equipment or machinery assesses the reliability of the inputs it receives. A human operator can often receive an instruction or reading and make the intuitive decision to question it because it doesn't feel right. Now this *can* be a mistake, and people have died because a valid input was rejected by a person due to confusion, but often a surprising input or instruction does mean that something is wrong with the input. The fault in the 737-Max was relying on a single aircraft attitude sensor. The fault in pilot training was not informing the pilots of this potential problem or training them how to deal with it. The fault in Boeing was in trying to use an unsuitable airframe for new style engines without proper engineering changes, in order to 'save money'.
On the other hand, I'm not a Chief Executive Officer in a major aerospace engineering company, I've only got a PhD in mathematical logic and worked one summer in Systems Reliability for BAe, so what do I know?
One question ( I have not read through all comments so may already have been answered.)
If this was a non US company and american people died in both crashes, how much shit would have hit the fan ?
How much more shouting and threats from the US government threatening sanctions, huge fines, extradition of execs to face court action in the US etc.
Total fucking hypocricy once again from the US, same as all the shouting when the BP oil platform collapsed in the gulf od Mexico, Obama demanding UK heads on sticks and billions in reparations, not quite so when US companies were resposible for the same type of ecological disaster worldwide ( Bhopal is just one, use google to find your own.)
Same shit different day. Now and forever more.
"The manufacturer did not admit guilt, however, which would have prevented it from receiving future government contracts. Instead, it entered into a deferred prosecution agreement.
No company executive faces imprisonment for the misconduct that the biz has acknowledged. Boeing fired CEO Dennis Muilenburg in late 2019 over the 737 Max accidents and he departed with $62m in compensation."
Not admitting guilt because it wouldn't be able to get future government contracts? It let planes fly knowing there were serious issues and lied / deceived regulators. It shouldn't be getting ANY future contracts from anyone! That's the idea of punishment - a deterrent. Instead they pay a paltry fine and the CEO lost his job just to be consoled by a huge compensation package. I'll never forget seeing that a lot of companies could implement a lot of changes to make safety even higher but the cost to do so is far greater than paying compensation to crash victims' families. Everyone has a value to a company that big.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
