back to article Boeing big cheese repeats pledge of 737 Max software updates following fatal crashes

Boeing chief exec Dennis Muilenberg has repeated earlier promises that a software update for the troubled Boeing 737 Max airliners is coming "soon". In an open letter published last night Muilenberg acknowledged the "shared grief for all those in mourning" after the separate crashes of two 737 Max 8s within a few months - …


    1. Fred Flintstone Gold badge

      Is that a new brand of plane that only taxies back and forth?

  1. JaitcH

    Trumps Regulation Deletion Doesn't Work Too Well In Many Industries

    The FAA has apparently used Boeing employees to perform testing in lieu of Third Parties.

    Hard to work for two masters.

  2. devTrail

    Still glossing over

    Some pilots have alleged that original training courses on the 737 Max glossed over what MCAS was and how it worked.

    Are the journalists unaware of the details or is Boeing still glossing over?

    What kind of patch are they releasing? Making the system easier to disable, more redundant and tolerant to faulty sensors, both, or something else?

    How will the additional pilot training work? I guess they can't release new manuals and say that from the next day the planes can fly, pilots will need some time for the extra training. Moreover, if the system is shut down the pilots will have to learn to fly a plane that suddenly has a different balance, will they be able to try it in the simulator?

  3. BobC

    MCAS from a Systems Perspective

    I was an engineer at an aircraft instrument maker while it was developing and certifying its TAWS (Terrain Awareness and Warning System) product. I worked in essentially all aspects of the product except for the design of the hardware itself. In particular, I was involved in requirements analysis, software design and development, hardware production (processes and tools), and FAA certification.

    Though TAWS provided only audible and visual warnings, we were greatly concerned that we not urge the pilot to take excessive or inappropriate action, such as by announcing a warning too late, or by announcing an incorrect warning. The official TAWS specification described very well what the system must do, but did much less to define what the system must **not** do.

    One of my prime responsibilities was to conceive of ways to "break" TAWS, then update our requirements to ensure those scenarios were properly handled, and then update our certification procedures to ensure those scenarios were thoroughly tested. Many of my findings revealed "holes" in the official FAA TAWS specification, some of which were significant. (Being a competitive company, we fixed them in our product, then reported them to the FAA as real or potential flaws in competing products. Essentially, we kicked the hornet's nest every chance we could. Fun for us, harmful to our competitors, and safer for everyone.)

    The "single-sensor problem" is well known and understood within the avionics community. However, as our TAWS was initially an add-on product for existing aircraft, we often couldn't mandate many aircraft changes, which could greatly increase the cost and down-time needed to deploy our product. Fortunately, all aircraft are required to have "primary" and "secondary" instruments, such as a GPS heading indicator backed by a magnetic compass. Furthermore, sometimes the display for a low-priority function can be made to serve as a display for a secondary sensor when the primary fails, in which case it is called a "reversionary" display.

    The sweet spot for us was that the vast majority of our initial TAWS customers would already have some of our instruments in their cockpit, instruments that already had all the inputs needed to serve as reversionary displays. Inputs that can be shared with our TAWS product.

    When we looked at the whole TAWS specification from that perspective, we realized there were circumstances when "primary" and "secondary" instruments may not suffice, particularly if both relied on the same underlying technology (such as digital and analog magnetic compasses, which sense the same external magnetic field - meaning a non-magnetic way to determine magnetic heading was needed, which GPS could help provide).

    I had prior experience in "sensor fusion", where you take a bunch of diverse/redundant/fragile sensors and combine them to make better "synthetic" sensors. Back in the day this was done with various kinds of Kalman filters, but today a neural net would likely be more practical (primarily because it separates the training and deployment functions, making the deployed system simpler, faster and more accurate).

    So, for example, let's say all your physical AoA (Angle of Attack) sensors died. Could you synthesize a suitable AoA substitute using other instruments? Given a list of other functional sensors, the answer is almost always "yes". But only if there is a physical system component where all those other sensors meet (via a network or direct connection). We had that meeting spot, and the compute power needed for a ton of math.

    But even synthetic instruments need primary and secondary instances, which meant not only developing two independent algorithms to do the same thing in different ways, but also, to the greatest extend possible, running both of them on redundant hardware. Which, again, our initial customers already owned!

    This extended to the display itself: What if the display was showing an incorrect sensor value? The secondary or synthetic sensor was compared with what was actually being shown on the display. If we detected a significant mismatch, this system would simply disable the display, completely preventing the pilot ever seeing (and responding to) any bad information.

    I'm concerned Boeing didn't do enough analysis of the requirements, design and implementation for MCAS: My guess would be that MCAS was developed by a completely separate team working in a "silo", largely isolated from the other avionics teams. For example, this is an all-too-common result of "Agile" software development processes being applied too early in the process, which can be death for safety-critical projects. And, perhaps, for those relying on them, including passengers, not just pilots. Yes, a company's organization and processes can have direct safety implications.

    Another example: When an automated system affects aircraft actuators (throttles, flaps, rudder, etc.) the pilot must be continuously informed which system is doing it, and with a press of a button (or a vocal command) be able to disable that automated subsystem. It seems the Boeing MCAS lacked both a subsystem activity indicator and a disable button. Though it won't happen, I wish this could be prosecuted as a criminal offense at the C-suite, BoD and senior management levels.

    I believe the largest problem here was all levels of aircraft certification testing: It would appear the tests were also developed in "silos", independent of their relationships to other parts of the system, including the pilot. The TAWS product I worked on was also largely self-certified, but we did so using, again, two separate certification paths: Formal analysis and abundant flight testing (both real and simulated).

    The key element for FAA self-certification to be worth believing in relies on the FAA requirement for aviation companies to work with FAA-credentialed DERs (Designated Engineering Representatives). DERs are paid by the company, so there is great need to ensure no conflicts of interest arise. On our TAWS project, the first DER we worked with was a total idiot, so we not only dismissed him, but requested the FAA strip his credential and investigate all prior certifications he influenced. After that incident, we worked with a pair of DERs: One hands-on DER who was on-site nearly every day, and another God-level (expensive) DER who did independent monthly audits. We also made sure the two DERs had never previously worked together (though the DER community is small, and they all know each other from meetings and conferences).

    Did Boeing work with truly independent DERs? I suspect not: There are relatively few DERs with the experience and qualifications needed to support flight automation certification. Which means "group think" can easily set in, perhaps the single greatest threat to comprehensive system testing. I predict several FAA DERs will "retire" very soon.

    Even from reading only the news reports, I see several "red flag" issues the NTSB and FAA should pursue as part of the MCAS investigations.

    Bottom line, the Boeing Systems Engineers and the FAA DERs have well and truly dropped the ball, not to mention multiple management-level failures. I'm talking "Challenger-level" here. Expect overhauls of both Boeing and the FAA to result. Expect all certs for all Boeing products still in the air to be thoroughly investigated and reviewed by the NTSB, NASA, the EU and China/Russia. Do not expect any new certifications for Boeing for perhaps years.

    1. Intractable Potsherd

      Re: MCAS from a Systems Perspective

      Thanks, BobC - it is good to hear from someone who knows the industry. That you have verified what seems to be the consensus opinion at the moment* is depressing, though.

      *Pending outcomes of the accident investigations, of course.

    2. Fred Flintstone Gold badge

      Re: MCAS from a Systems Perspective

      Thanks for that. I was just sent a link to a document written by a pilot who has also an IT background, and it makes, frankly, for horrific reading.

      As a matter of fact, I preserved it, just in case Boeing tries to get it offline because it is a sane but wholly damning review of what happened, and why. I quote:

      If I have not been clear, so far, let me say it succinctly.

      Boeing produced a dynamically unstable airframe, the 737 MAX. That is big strike #1.

      Boeing then tried to mask the 737’s dynamic instability with a software system, similar to the systems used in dynamically unstable fighter jets (though those jets are fitted with ejection seats). Big strike #2.

      Finally, the software system relied on systems known for their propensity to fail (angle of attack indicators) and did not appear to include even rudimentary provisions to cross check the outputs of the angle of attack sensor against other sensors, including the other angle of attack sensor. Big strike #3.

      None of the above should have passed any muster. None of the above should have passed the “ok” pencil of the most junior engineering staff, much less a DER.

      Go read it. After that, I suspect you won't go near a 737 MAX ever again, even after the patch.

  4. DrM

    Just a tad pregnant

    So, Boeing finally went the Airbus (AirCoffin) route and put in SW the pilot couldn't take command away from.

    Decided they'd go the XL Airways Germany Flight 888T route when the three angle of attack sensors disagreed and killed everyone on board?

    Just a little bit though, just a little bit pregnant, just piles of dead bodies.

    1. Fred Flintstone Gold badge

      Re: Just a tad pregnant

      Airbus starts at least with a dynamically stable airframe (which is where all the 737 MAX's problems originate), and as its software has the last say instead of the pilot, redundancy is not seen as an afterthought but as a critical safety component (and, let's be honest, as the only way to get a FAA certification, at least one that's been done properly).

      Last but not least, Airbus has decades of experience with software running the show, so by now they have a pretty good handle on where issues can arise and what to do to address them now before it ends up killing people. For Boeing to think they can quickly slap something together to fix a fundamental physical design problem and put that pretty much in charge over the pilot is unforgivable, especially since this was so critical to keep the plane in the air. It also raises MAJOR issues about the certification path for the 737 MAX.

  5. Wobbly World

    300 knots at 600 feet 0oop’s!! RIP...

    What do we know about the behavior of the 737 Max 8 plane in the Ethiopian Air incident?

    When looking at the current data on the flight path of Ethiopian Airlines Flight ET 302 on March 12, the airplane got to 1,000 feet. The airplane at that point lost about 400 feet of altitude, which is extraordinary. The next thing that’s interesting is the airplane flew level for about 30 seconds, about 500 to 600 feet above the ground. This is not what a civilian airplane’s supposed to do—you’re supposed to fly away. And the airspeed continued to increase, they got to over 300 knots at under 1,000 feet above the ground. No pilot would consider doing that.!!!

    Why they weren’t able to immediately climb in that 30-second period is a mystery!!!

    But at the end of that the plane did climb, and it climbed very nicely. We don’t know what happened after that—the last thing we saw on radar was the airplane flying very fast and climbing. One would have thought that whatever happened, they figured it out and off they went. But, tragically, they didn’t.

    Can anyone here suggest what was going on to cause the anomalies listed above?

    In the case of the 737 [Max 8]engines are installed below the center of gravity. And so, as the wing loses lift, the engines generate a pitching movement that causes the nose to want to go up. If the nose is starting to rise all by itself and the pilot doesn’t want that to happen, they will have to push the stick to stop it from going up, and that force reversal is a big no-no. Basically you should pull the stick to go up, and you should push the stick to go down—and you should never have to push the stick forward to stop it from going up!!!

    The debris field starts some distance from the crash site was the plane breaking up before impact?

    Could this indicate an explosion or other catastrophe event, smoke was seen coming from the rear of the plane, though that needs taking with a pinch of salt as the witnesses view, could of hidden the fact, smoke was coming from the fuselage or an engine out of view of the witness but from their point of view looked to be coming from the tail, the same goes for the debris, paper and other items seen falling from the plane prior to it crashing, what are the forums opinions of this and the above?

  6. Anonymous Coward
    Anonymous Coward

    Technically it is not an anti-stall device. It is a mechanism to counteract the tendency of the position and size of the new engines to generate positive lift at higher angles of attack. This positive lift reduces the force required on the control column to increase angle of attack and certification means that column forces on the approach to stall should not decrease. MCAS dials in nose down trim to ensure the column force required to increase AoA remains consistent or increases on the approach to stall.

  7. kbutler.toledo
    Thumb Down

    Smell that? It is not burning jet fuel it is pure GREED (optional, of course) but the GREED is pure.

    Based on reports about the MAX airliners that if Boeing made automobiles they would make brakes optional, at a 'small' extra cost, however. You can always just drag your feet to stop the car.

    Profit was placed above safety and concern. I'll almost bet Boeing exec's travel on Airbus.


POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like