back to article Ethiopian Airlines boss confirms suspect flight software was in use as Boeing 737 Max crashed

The Ethiopian Airlines 737 Max 8 that crashed this month, killing all 157 passengers and crew, was actively using Boeing's Maneuvering Characteristics Augmentation System (MCAS) that is thought to have brought down a similar 737 five months earlier. In an interview with the Wall Street Journal over the weekend, Ethiopian CEO …

  1. sanmigueelbeer

    The plane's black box recorder has been recovered, and is in France for analysis.

    Not it is not. The FDR was sent to France in order to download the data. BEA immediately handed over all the data (and the FDR) back to Ethiopian authorities.

    The reason behind this is because the FAA are currently in Ethiopia "assisting" in the investigation.

    Boeing is now working on a fix for the issue, including getting data from multiple sensors before activating MCAS

    The updated software will now have a "feature" that if AoA should disagree, the computer will automatically disabled MCAS.

    1. Trollslayer

      If the sensors are in a critical system why only two of them?

      1. Anonymous Coward
        Anonymous Coward

        > If the sensors are in a critical system why only two of them?

        Ultimately because when the 737 was first designed, the AoA sensors didn't control anything critical and so was fine. Later (many years later) the Max came along and with it a change that made the AoA sensors critical.

        So either the criticality of the change wasn't recognised or someone decided that adding a third would require the other (tried and tested) subsystems that were designed for only two sensors to be re-designed, re-tested and re-certified and so decided not to.

        But even the latter decision isn't necessarily a cynical disregard for safety. As long as the MCAS system could be easily and quickly disabled it wouldn't present a problem. But, apparently, that was missed as well.

        With hindsight its easy to say, what about this, or that or the other but no one in Boeing set out to kill 300-odd people and if I were one of the software engineers responsible I'd be feeling pretty desperate with myself.

        1. paulll

          "As long as the MCAS system could be easily and quickly disabled it wouldn't present a problem. But, apparently, that was missed as well."

          The biggest of the myriad problems here is, that likely wasn't, "missed." It was probably deliberately glossed-over to avoid people asking awkward questions like,"soooo... it doesn't entirely look like a 737. It's not equipped like a 737. Without this computer system it doesn't really fly like a 737. Is it reaaallly a 737?" And then there'd be the risk of having to recertify from scratch, pilot retraining, maybe Airbus scooping up some contracts in the meantime.

          1. Snake Silver badge

            Error! Error!

            The worst line of the entire article is:

            "Boeing is now working on a fix for the issue, including: getting data from multiple sensors to check that the angle-of-attack reading is correct before activating MCAS

            What do you MEAN you are only NOW changing the system to take multiple sensor readings into account before activating a response-changing subsystem on an aircraft??!! DID YOU NOT REMEMBER THE IDEA OF "FAILSAFE"?!!

            You screwed up Royally. With a huge capital "R" for emphasis.

            Between the idea that you failed to design an aeronautics system that was actually failsafe, AND the fact that you failed with the self-certification of the aircraft, it means that you COMPLETELY forgot the hard-learned lessons of the DC-10 THY debacle. Decades of safety protocols and lessons completely and utterly thrown out the door in the rush to get your latest bird in the air and making money.

            Someone's head should roll for this. Massively.

      2. JeffyPoooh
        Pint

        Triple Redundancy is insufficent to overcome bad design and coding

        Trollslayer asked a perfectly reasonable question, "If the sensors are in a critical system why only two of them?"

        Circa 2008, an Airbus A330-303 aircraft operated as Qantas flight 72 had a very similar incident, except it ended more successfully (barely). It was equipped with Triple-Redundent AoA sensors, but the systems and software design still managed to be very dangerous. So triple redundancy by itself is insufficient given design and coding errors.

        To preempt complaints about needing "explicit citations", the purported "risk of relying on Wikipedia", or "speculation in advance of the final report", here is a link to the actual final report.

        See footnote 28 (for example) for the triple AoA sensor factoid.

        Ref: https://www.atsb.gov.au/media/3532398/ao2008070.pdf

    2. diodesign (Written by Reg staff) Silver badge

      re: sanmigueelbeer

      Hi - thanks, that's fixed. Don't forget to email corrections@theregister.com if we get anything wrong.

      C.

  2. Schultz
    Stop

    I would expect a longer process for re-certification

    It sounds like Boeing cut some corners in the certification of the 737 max models. Will it be sufficient to fix the obvious problem, or should the whole certification process be done properly? From the point of view of the aviation safety regulators, option two might look compelling. They failed their jobs on the first try, they might feel compelled to be more diligent now.

    1. Anonymous Coward
      Anonymous Coward

      Re: I would expect a longer process for re-certification

      It certainly does look like corners were cut, to fatal excess. This is why the FBI, Department of Transportation and the Department of Justice are all looking into the matter. The FBI's involvement indicates that there's a high potential for this becoming a criminal matter.

      Boeing and the FAA are carrying on regardless of these investigations, not surprisingly I suppose because to react to them would be some sort of admission. Yet it's highly likely Boeing and the FAA are ready to go, and the DoT may then override them.

      It also increases the chance that the EASA and everyone else will want to re-examine the certification for themselves. I sense that there's a big loss of faith between Boeing / FAA and everyone else, and a big part of restoring that would be some kind of mea culpa reaction from them. Yet we've seen nothing at all of any substance. All throughout this unhappy train of events the messages coming from Boeing and the FAA have been everything but sympathetic to the idea that they've got this one wrong.

      They're not even having the decency to wait for the crash reports from Indonesia and Ethiopia (which are both perfectly capable of producing a good report, and will no doubt seek further assistance from, say, France should they need it to produce these reports). This is especially important because the human factors at play in the cockpit leading up to these tragedies is going to be as important as the technical data gleaned from the FDRs. Unlike the FDR traces, the CVR needs especially careful analysis. This takes time.

      So the PR surrounding the return to flight is stacking up to being one great big insult to the Indonesians, Ethiopians, EASA, CAAC, and all the other regulatory bodies. It's purely about getting American Airlines back in the sky.

      And even then they might be seriously misjudging the public mood in the US. The return to flight is being rushed, and everyone knows it. Someone produced an app telling you whether your flight was going to be operated by a 737MAX. And everything about the return to flight smacks of a big corporation with problems to sweep under the carpet forcing an aircraft with a questionable (USG is currently asking the questions) certification down the throats of the American public, and doing everything possible in the land of PR to disguise the facts that there was ever a problem, that there's a criminal investigation going on in the background, that the fix isn't based on careful study of the problem (there's no official reports of the crashes yet). This PR effort could easily backfire, and would make criminal arrests even more catastrophic for the company.

      Welcome to 21st century aviation safety in America.

      Actually, I think the US government recognises the existence of a big problem, and sees a need for a proper fix. Hence the unprecedented investigations by the DoT, DoJ, FBI. Uncle Sam needs Boeing to be solvent and operating properly. If Boeing destroy themselves there's no one to act as design authority for all Boeings flying today, which would ground all of them globally, immediately, permanently. You can see why Uncle Sam is keen to avoid that, and why it wouldn't want to have to nationalise Boeing. Sending in the feds, cutting out the rot and showing the world that things are getting back to normal minus the personnel who have caused these fatal shortcomings is a way for them to gain some trust back from the EASA, CAAC, etc.This still might not save the 737MAX quickly, but it would improve the odds that a slower fix blessed by a reformed FAA would be more acceptable globally.

      It's also a way of deflecting attention from the fact that the FAA has been starved of resources by Congress and Administrations for decades, and this is likely the root cause of these tragedies. An emaciated FAA has been unable to restrain the actions of a company run by MBAs, not Engineers, who themselves are fighting for their lives in the cut throat global business of supplying airliners. Who'd have thought the French would be better at it than Boeing?

      1. Charlie Clark Silver badge

        Re: I would expect a longer process for re-certification

        Actually, I think the US government recognises the existence of a big problem, and sees a need for a proper fix.

        How long has this FAA been without a proper boss under the current government? But, yeah, the wider point is that successive US governments have caved into the lobbyists for self-certification.

        The bigger risk for the US airline industry is the potential end of mutual recognition of certification. Boeing and the FAA pulled a fast one with the 737 Max 8. If they're smart, they'll do everything necessary to demonstrate that the certification is valid, ie. essentially redo it.

      2. Anonymous Coward
        Anonymous Coward

        Re: I would expect a longer process for re-certification

        >> the fix isn't based on careful study of the problem

        This. WTF is going on.

        A computer system was required to do something to make it the "same" plane. Now said computer system isn't needed, will auto-disable.

        But it is still the "same" plane?

        >> Uncle Sam needs Boeing to be solvent and operating properly

        Too big to fail.. sounds familiar...

        >>Who'd have thought the French would be better at it than Boeing?

        Au contraire, is this not evidence the Americans don't do it right? A pretty high cost for the yipee-ki-yay management and engineering.

        Boeing execs and their families should exclusively fly 737 Maxs with the least options before anyone else is put on that thing....

        1. A.P. Veening Silver badge

          Re: I would expect a longer process for re-certification

          "Boeing execs and their families should exclusively fly 737 Maxs with the least options before anyone else is put on that thing...."

          Please add Boeing bean counters and all FAA staff to that list. And while we are at it, including congress (both houses) and the executive branch wouldn't be a bad idea either.

        2. Anomalous Cowshed

          Re: I would expect a longer process for re-certification

          The big problem is this: regardless of what all these companies say, safety is, at best, the last of the airlines' concerns, far behind fuel efficiency, cutting costs, being able to cram more mugs aboard each aircraft, being able to monetise more services and features previously provided free of charge and taken for granted. At worst, it is merely optional, as demonstrated by the revelations that several safety features are offered as optional extras.

          And yet safety should be the main concern. By all means sell cheaper versions of aircraft with less fuel-efficient engines, or less efficient stacking of the seats in them. But let ALL the aircraft have every proven safety feature as standard. And let people, who have come to regard air travel as a kind of glorified bus service, see to their safety and clamour for it, or stop using this means of transportation. It may be touted as the safest way of travelling, but when something goes wrong, or someone decides to cut the slightest corner, which they seem to be doing rather liberally, these days, your life as a passenger is forfeit.

        3. Anonymous Coward
          Anonymous Coward

          Re: I would expect a longer process for re-certification

          But I bet any Pilot flying them would have "Full Training" not simulator, including what to do is MCAS computer program causes the elevators to go TITSUP (Make your own acronym)

    2. Chris G

      Re: I would expect a longer process for re-certification

      This is beginning to look like the 'Volkswagen effect'

      To save time and money while getting around legislation and maintaining a competitive edge they have taken short cuts without thinking the whole thing through or possibly without caring enough.

      Being the biggest kid on the American block and too big to fail, helps to engender that kind of attitude.

      I wonder if this goes to criminal charges, if the company will have an engineering patsy ready to fo some bird while the executives still get their bonuses?

      1. Anonymous Coward
        Anonymous Coward

        This is beginning to look like the 'Volkswagen effect'

        Make that "US business practice effect"

        It isn't restricted to cars or even just vehicles(intel), when money people make the final decision (without ethical oversight) then expect the product to be atleast an embaressment to the engineers and scientist that worked on it and at worse a ticking time bomb that will cost everyone else.

        Here in the UK ignored people are dying in the streets and hospitals, after brexit they will be "dealt with" by armed ex-military goons enforcing private law, all so the current affluent can have the American Dream i.e. rob everyone else and tell them it is their own fault.

        1. imanidiot Silver badge
          Stop

          Re: This is beginning to look like the 'Volkswagen effect'

          "Here in the UK ignored people are dying in the streets and hospitals, after brexit they will be "dealt with" by armed ex-military goons enforcing private law, all so the current affluent can have the American Dream i.e. rob everyone else and tell them it is their own fault."

          Stop spreading FUD.

        2. Anonymous Coward
          Anonymous Coward

          Re: This is beginning to look like the 'Volkswagen effect'

          What do you get our of talking bollocks?

        3. HieronymusBloggs

          Re: This is beginning to look like the 'Volkswagen effect'

          "Here in the UK...."

          I was with you for the first part of your post, but you blew it when you made a UBC (Unsolicited Brexit Comment).

    3. I ain't Spartacus Gold badge

      Re: I would expect a longer process for re-certification

      If the Seattle Times piece I read is true, then I think the aircraft will need to be re-certified. Or at least the MCAS fix will not be quick - and will need lots of work.

      This boils down to two quesions:

      1. Can you make MCAS not kill everyone, by flying the plane into the ground, without a major re-design?

      2. Can the plane safely be flown with MCAS disabled.

      The answer to 1. is easy. You can turn it off at the slightest sign of sensor error. Even though it only operates on one sensor, it has access to two - and the plane has other sensors that can be compared with it.

      But the answer to 2 is the killer. Can the plane be flown safely with MCAS turned off? The FAA were told that MCAS only needed authority to change the stabiliiser pitch by 0.6°. And that was considered safe, as it could be easily compensated for by the main flight controls. But the Seattle Times said this was then changed in test flying to 2.5°. So does that mean the plane becomes dangerous without MCAS? It's because the engine cowlings generate more lift as the plane's angle of attack rises - and if this needed 4 times as much correction as first thought - then does it mean the plane is unsafe due to being longitudinally unstable? The certification files were then not updated to show this changed MCAS design.

      But according to Seattle Times there's worse. MCAS keeps on working. It looks at the AOA data from it's single sensor and says do I need to push down on the nose? But doesn't apparently take account of earlier actions it has already taken. Thus if the AOA sensor is knackered it will just keep pushing the nose down until it's moved the stabiliser all the way to the stops! Which looks like a software design error to me. And also means that the certification documents that say it only has authority for a 0.6° correction are utter bollocks!

      If all the above is true and you take an alarmist/worst-case attitude, then you could argue that flight testing showed the airframe to be unsafe and MCAS has a fundamental design flaw over-and-above the only operating on one sensor fuck-up.

      1. Bonzo_red

        Re: I would expect a longer process for re-certification

        According to the NY Times, the crew would have had 40 seconds to disable MCAS then would have needed to disengage the motors and manually set the tail.

        https://www.nytimes.com/2019/03/25/business/boeing-simulation-error.html

        1. I ain't Spartacus Gold badge

          Re: I would expect a longer process for re-certification

          The article doesn't quite say 40 seconds - even if the headline does. You've got 40 seconds to respond after MCAS has triggered in that fault condition - in order to kill MCAS and recover.

          But the normal response of the Lion Air pilots was to adjust the trim from their joysticks - which used the same motor as MCAS does to adjust the trim of the aircraft. So they are warring over the same system as MCAS uses. MCAS operates every ten seconds, and can move that trim all the way to the end if the pilots don't stop it. But they can keep countering it successfully, as the Lion Air pilots did for 8 minutes.

          The correct procedure is to kill the trim motor power, which stops MCAS from killing you. The rush is then that if MCAS has trimmed the aircraft nose-down you have to get to the manual trim wheel, and turn that for a while until you've got back to level trim. Though as long as MCAS hasn't managed to move the stabiliser trim all the way to the stop yet, your work-out correcting it with the trim wheel will be less. And normal flight controls (pulling back on the stick) will overcome that nose-down trim, again so long as MCAS hasn't moved it all the way to the stops.

          1. werdsmith Silver badge

            Re: I would expect a longer process for re-certification

            you have to get to the manual trim wheel, and turn that for a while

            The most common place for manual trim wheels is in the centre of the cockpit between the seats or by the pilot's knee. Won't take much getting to. Unless they have moved on computer augmented control layouts.

            1. I ain't Spartacus Gold badge

              Re: I would expect a longer process for re-certification

              But it takes a lot of turning. It's some large number of turns between full trim both ways - which matters if you're in full nose-down trim and in a hurry to avoid hitting the ground.

              It should have been "go to" rather than "get to" though.

            2. Paul Hovnanian Silver badge

              Re: I would expect a longer process for re-certification

              "Unless they have moved"

              Still in the same old place. The problem with these wheels is that they require a number of turns once MCAS (or some other fault) has driven the trim fully against one stop. Meanwhile, the flight crew is busy pulling back on the yoke, hoping that the elevator has sufficient control authority to overcome the horizontal stabilizer.

              The benefit of a software mod that senses an AoA input disagree is that MCAS can be cut before it runs the stabilizer to full stop, making the pilot's job easier. If the patch is applied correctly, it may even be able to remove the (faulty) MCAS trim signal and obviate the need to kill the entire electric trim system (with toggle switches). Leaving the manual thumb switches operational and relieving the crew of a bunch of wheel-cranking.

              The problem with any fix is that Boeing and the FAA will now have to examine every branch of the resulting fault tree. Very carefully and deliberately. And one of the fears that Boeing has: The longer you look at any system, the more bugs you might find. And fixes or modified crew procedures for each will have to be developed.

            3. Alan Brown Silver badge

              Re: I would expect a longer process for re-certification

              "The most common place for manual trim wheels is in the centre of the cockpit between the seats or by the pilot's knee"

              The B737 trim wheel is something like a couple of hundred revolutions from one end to the other.

              Yes you can turn it by hand. Yes it's geared down to allow you to do that without power assist. If it's been whacked against the stops it's going to take one pilot frantically spinning it whilst the other is doing full stick elevator to try and keep the thing from going dirtside - and at either stop there's NOT enough elevator to bring the aircraft back to level flight (Unlike smaller aircraft, on airliners you fly and trim with the _entire_ tailplane. Elevators are effectively there for small corrections and manouvres)

              The wheel has a popout handle to make it easier to do gross movements (or did on the 737-200 simulator I did my turbine ratings in) but it's still going to take a some time to retrim and in the meantime you can easily run out of sky if the pilots have taken a while to figure out WTF is happening and pull the motor breaker.

              Essentially, what Boeing did with the Max was to Rat Rod a 50yo airframe and hope they could get away with it by not explicitly spelling out that software is now essential to keeping the aircraft flying straight&level/preventing pilots from badly screwing up by applying power too early in a stall and if that software goes wrong Bad Things Happen - remember the constant refrain has always been that Boeings are aerodynamically stable without needing computer intervention and Airbus are not (If you stall in a Max or NG and apply power _BEFORE_ nosing down, you'll never be able to get your nose down, so this was already a design on the ragged edge of oblivion. MCAS just adds to it)

              There comes a time when engine evolution gets the things too large for whatever they're bolted onto and Max is it - speculation had been around for a while as to when it would happen on 737s - you can't give the 737 airframe longer legs (to move the engines back under the wings) for a bunch of reasons, so the choice for Boeing was Max or a new airframe. It's looking pretty clear that they made the wrong choice.

              The FAA being subject to regulatory capture has been known about for a while but I'm not sure even this will solve that problem in the USA. You can expect EASA and other regulators to start looking much more closely at what gets approved by the FAA as a result of this and the 787 battery fires (If they'd used Lithium Iron Phosphate chemistry, those wouldn't have burned, but would have weighed a little more).

              1. Updraft102

                Re: I would expect a longer process for re-certification

                I think they cancelled the wrong plane. The 757 entered production well after the 737, and Boeing ended production on it in favor of ever-longer stretched versions of the much older 737 airframe. They should have kept the 757 and not kept stretching the 737 to where it was almost as large as the 757 (which shares the same fuselage diameter)... the 757 was designed from the start with much longer landing gear and engine pylons designed for low-hanging, large-diameter high bypass fans from the start. Flattening the bottoms of the engine nacelles worked well enough for the NG series, but as everyone's already said, the short landing gear of the 737 does not accommodate the ever larger engines they want to keep adding.

                It's my impression that the 737 Max would possibly work fine without MCAS, but it wouldn't be eligible for inclusion on the same 737 type certificate with the change in handling with the new engine geometry. That was a big selling point-- all the captains already certified for the NG series would be certified for the Max as well, and the only training would be a short course on the differences.

                Of course, had MCAS been built in a sensible way, we probably would not be talking about any of this, as it would have worked as intended and not flown planes into terrain. MCAS should not have been omitted from the manual, it should not have ever been programmed to take action based on only one sensor (!), and it should not have done anything without a warning to the crew that it was taking action.

                The trim wheels are quite noisy by design, so the pilots would have heard the clacking, but even then it would only have been evident that there was an auto-trim runaway, but not why it was happening. Full auto-trim authority should be available to the pilots without taking any "heroic" actions... no more difficult than disengaging autopilot. This is supposed to be a Boeing, where the pilot is the ultimate authority and the avionics are there to assist in flying the plane, but not override the flight crew. If Boeing had been focused on making the best plane they can, and not worrying so much about maintaining a common type rating with older 737s, I imagine it would have been like I suggested. Even with the low ground clearance and reconfigured engine geometry, I think the Max would have been fine if not for Boeing's attempt to claim this was just like all the other 737s. Clearly, it is not.

          2. JeffyPoooh
            Pint

            Re: I would expect a longer process for re-certification

            'I ain't Spartacus' wisely noted, "You've got 40 seconds to respond after MCAS has triggered in that fault condition - in order to kill MCAS and recover."

            In the 2016 move 'Sully', the investigators famously added a 35-second reaction time to the simulations. (Yes, yes, yes... I know it's just a movie...)

            Point being: It's interesting how 35 seconds is so very close to 40 seconds.

            If we allow 35 seconds for reaction, does the remaining 5 seconds provide enough time to switch things off and wind the trim wheel back?

            Interesting point, no?

      2. Chris G

        Re: I would expect a longer process for re-certification

        @Spartacus. What your comment implies is that there has been a failure to implement full testing and action on pilot feedback that in my (not professional) opinion, amounts to criminal negligence.

        1. I ain't Spartacus Gold badge

          Re: I would expect a longer process for re-certification

          ChrisG,

          I don't disagree with you. However, I'm cautious. I've only read newspaper reports so far - and neither crash investigation is complete. The Ethiopia crash is barely even started - and even the info from the Lion Air crash is only preliminary.

          If true, I find the info from the Seattle Times really disturbing. Because it implies failure everywhere:

          1. Boeing didn't fully understand the requirements the airframe imposed on the MCAS design until test flying - which is fair enough that's why we test. But then on finding such radical differences, didn't re-asses the system. Or even report the problem back to regulators. The documentation wasn't updated.

          2. Boeing also seem to have misunderstood what their own software could do. MCAS should only have authority to move the stabiliser by 0.6°. Then on test flying that was upped to 2.5° without telling anyone. But in reality the authority is unlimited, since the system has no memory of what it's previously done - so it will keep adjusting the stabiliser until it reaches the stop - setting which will crash the aircraft.

          3. Why is there a stabiliser setting which can cause unavoidable aircraft loss anyway? I'm no expert, but I can imagine there might be reasons for this, but it does surprise me a little.

          4. Finally failure of regulation. The FAA push less important stuff off to self-certification. But only the FAA get to decide what bit they'll certify and what bits Boeing get to. The Seattle Times allege that FAA management were pushing the certification team to push more-and-more areas off to Boeing in order to get the plane certified quicker. Either because the FAA's resources have been cut, or because of pressure to get the plane into service to meet competition from Airbus' A320Neo. If true this suggests regulatory capture, pressure from government due to lobbying or underfunding - or some of all three.

          1. nkuk

            Re: I would expect a longer process for re-certification

            It makes you wonder what other shenanigans and shortcuts have been going on in other areas if there is a huge reluctance to recertify and shift certification from the Authority onto the manufacturer.

            1. I ain't Spartacus Gold badge

              Re: I would expect a longer process for re-certification

              It makes you wonder what other shenanigans and shortcuts have been going on in other areas if there is a huge reluctance to recertify and shift certification from the Authority onto the manufacturer.

              Before you get too worried, remember that we've only just had the first year ever with zero fatalities in large commercial aviation. And also many people made allegations that Boeing had "got at" the FAA for certifying the 777 for trans-oceanic flight with only 2 engines. And yet, that's been one of the safest planes ever.*

              The reason to rush the certification is known. Airlines were ordering the A320 NEO because of the fuel savings. I suspect that the more important questions are going to be into what the FAA checked, what they didn't and why those decisions were taken. The MCAS issues should be relatively simple to solve, I think it's going to be the trust issues that take longer to fix.

              *Fun fact. The only large commercial model that's finished service with a "perfect" safety record is the Tupolev 114 - well some people got killed in one but it was a road accident, as it was on the ground at the time. Although I'm not sure you can say it didn't injure its passengers. It was a 50s turbo-prop and it was apparently up to 112dB inside the cabin!

          2. SImon Hobson Bronze badge

            Re: I would expect a longer process for re-certification

            3. Why is there a stabiliser setting which can cause unavoidable aircraft loss anyway?

            This was answered further up the page.

            Trim adjusts the angle of the whole horizontal stabiliser (the "small wings" at the back of the plane). That's the most efficient way to do it - have the whole surface doing the same thing. Some other designs have a fixed section and a large movable section - which means that you are likely to have two surfaces not doing exactly the same thing.

            The elevator that is worked by moving the control column is much smaller - provided the plane is trimmed correctly, only small amounts of force are needed to manoeuvre the plane. So the elevator can only over-ride the trim up to a certain point - after that, the effect from trimming is more than the elevator can counter. Hence the description above of having to realise what's going on, disable the trimming system, and then frantically wind the manual trim control far enough for the elevator to be able to get the nose up again. And while you are frantically doing this, with your co-pilot doing a gym weightlifting workout* holding the control column fully back, the plane is entering an ever steeping dive due to the excessive trim.

            Don't forget, this isn't a case of "it trims the aircraft to a certain attitude" (ie constant nose down) - it trims the aircraft to a "rate of change of attitude" (ie increasing nose down).

            * OK, perhaps slight exaggeration since the flight controls will still be power assisted or even fly by wire. But I bet that guy is pulling that column mighty hard against the stops !

      3. Ian Johnston Silver badge

        Re: I would expect a longer process for re-certification

        Thirty years ago I worked with a tensile testing machine which did something similar: oh "Hold" it checked the crosshead every second to see if it was moving and stopped it if it was, but it didn;t check to see if it had moved. We discovered this when a final year engineering student mounted a delicate test rig which he had designed and build over the previous six months, went for lunch and came back to find his pride and joy was now half an inch thick.

      4. Arctic fox
        Headmaster

        @I ain't Spartacus Re: "Can the plane be flown safely with MCAS turned off? "

        With that one sentence you have cut to the heart of the matter. If that is the case then the directors of the company should be facing several years singing "that's the sound of the guys working on the chain gang".

    4. macjules

      Re: I would expect a longer process for re-certification

      Remember the old acronym for Boeing? Bits Of Engine In Neighbour’s Garden. Sadly ne’er so true as now.

  3. Yes Me Silver badge

    Appalling software, it seems

    There's a very specific Reuters story. It seems that

    the MCAS system - which forces the nose downwards to avoid a stall, or loss of lift - will only operate one time for each event rather than impose repeated corrections like those believed to have pushed the Lion Air jet into a dive... MCAS will be disabled whenever two sensors that measure the ‘angle of attack’ - a parameter that determines how close a plane is to an aerodynamic stall - differ too much... a change from the previous set-up which only linked MCAS to one sensor at a time, ignoring the other, and which may have resulted in a single point of failure on Lion Air 610... Previously the “AOA disagree” warning would not have halted the MCAS software because the system was designed to focus on either the left or right sensor, alternating between flights. It was oblivious to whether readings from the sensors were aligned.

    WTF? Alternating between sensors on alternate flights was terrible design and seems like superstition rather than science. And ignoring the pilot's repeated attempts to raise the nose (tens of times over) when the plane is at very low level soon after takeoff was obviously wrong. If I'd been involved in that software design, I'd be in despair. (In the Air France crash over the Atlantic some years ago, forcing the nose down against the pilot's mistaken reaction to a stall might have saved the plane, but that was at very high level in horizontal flight.)

    1. Olivier2553

      Re: Appalling software, it seems

      And ignoring the pilot's repeated attempts to raise the nose (tens of times over)

      I thin I read somewhere that the MCAS was not supposed to force the nose down more than 4 times in a row, but each time the pilot would correct the mistake resulted in am unwanted reset of the counter.

      1. Trollslayer

        Re: Appalling software, it seems

        As a professional test engineer HOW THE F**K wasn't that scenario covered?

        Let me guess, there was a release date decided early on.

        1. anothercynic Silver badge

          Re: Appalling software, it seems

          Let me guess, there was a release date decided early on.

          That's *exactly* what happened. Airbus was finishing off their A320neo certification and Boeing had to respond soon after to avoid losing business from American... Enough said.

    2. Anonymous Coward
      Anonymous Coward

      Re: Appalling software, it seems

      Alternating between sensors on alternate flights was terrible design and seems like superstition rather than science.

      The old mariner's adage "never go to sea with two chronometers" would seem to apply here, because if you have two of anything and they give different readings, which one do you trust? Triple (or even more) redundancy is the rule for many safety-critical systems, but if Boeing charge $80,000 for a warning light then goodness knows how much more they'd charge for a third AoA sensor.

      1. Anonymous Coward
        Anonymous Coward

        Re: Appalling software, it seems

        Although reading the Zero Hedge article linked below, it appears that the $80,000 is for a second AoA sensor as well as the warning light, implying that the standard setup is a single sensor.

      2. Anonymous Coward
        Anonymous Coward

        Re: Appalling software, it seems

        My destroyer had three, wound and compared each day. That was aside from what the satellites and standalone navigation set, which the quartermasters adored, were telling them. Heck, you could add the laser ring gyros to that list of time keepers.

        1. Anonymous Coward
          Anonymous Coward

          Re: Appalling software, it seems

          Early chronometer using ships would set to sea with maybe 20 chronometers on board. Some of these would be lent by other captains. The resulting logs when they returned from the voyage would enable assessment of which were the most trustworthy, so that eventually the number could be reduced as a reserve of trustworthy chronometers (and designs) was established.

      3. tip pc Silver badge

        Re: Appalling software, it seems

        there should be no need for the $80k warning light in the first place. MCAS should have been designed with 3 AoA sensors and to stop working if only 2 sensors where operational. The AoA sensors needed a warning light and some notification of what systems relied on it and would be down if AoA was offline.

      4. eldakka

        Re: Appalling software, it seems

        because if you have two of anything and they give different readings, which one do you trust?
        In the MCAS case, you'd disable MCAS if you were warned that the AoA sensors are disagreeing and that one may be faulty. That is, you'd ignore the AoA sensors entirely. MCAS isn't required to fly the aircraft, it's a system to make the aircraft behave more like the previous 737 models, rather than a system necessary for general flight safety.

        Therefore if all the following were true then 2 AoA sensors are sufficient:

        1) There is a way for the pilots to know that there is something wrong with the AoA sensor suite;

        2) The pilots know MCAS exists;

        3) The pilots know that MCAS depends on the AoA sensors;

        4) The pilots know the failure mode symptoms of MCAS;

        5) The pilots know how to disable MCAS;

        6) The pilots have been trained to fly the aircraft without MCAS assist (in which case MCAS wouldn't be necessary at all, as its primary purpose was to make the plane fly like the old 737 and thus not require any additional flight training, thus if the pilots were going to be trained how to fly the plane without MCAS, there'd be no reason for it).

        In the Lion Air case, it appears that none of the above were in place.

        Point one is enabled by an optional, at extra cost, "AoA sensor disagreement warning light" I think it was described as. That is, a light that turns on (or a status on the MFDs) that just tell the pilots the AoA sensors disagree. If you want to know the actual AoA readings are, that is the actual angle of attack angle the sensors are reading the plane as being at, as opposed to just a warning light, that is yet another extra cost option on top of the warning light cost.

        And as is apparently the case, the other points were not clearly laid out in the flight manual or the conversion "training", which is apparently all of 1 or 2 hours.

      5. JeffyPoooh
        Pint

        Re: Appalling software, it seems

        QF72 had triple redundant AoA sensors.

        See my post above for further interesting details.

    3. Anonymous Coward
      Anonymous Coward

      "forcing the nose down against the pilot's mistaken reaction to a stall"

      But AFAIK, in that case the computer had switched to an alternate mode because it no longer had reliable inputs from the sensors - and was up to the pilots to handle the situation, but they weren't able.

      Something alike of course could happen on a Max as well, if the MCAS disables and for any reason a pilot puts the plane in a situation MCAS was designed to not allow (which is what Boeing feared) - just it looks the MCAS didn't disable and moreover did reset when a sensor is faulty - probably the Airbus autopilot could have crashed the plane too if it tried to fly the plane with incorrect data as well.

      1. Anonymous Coward
        Anonymous Coward

        Re: "forcing the nose down against the pilot's mistaken reaction to a stall"

        Please, explain what's wrong in my post.... do you really believe the Airbus shouldn't have gone to alternate mode and the autopilot should have kept on flying the plane with incorrect Pitot data?

        Or that pilots not trained in the new Max flying characteristics don't risk to stall it without the MCAS? It's different explicitly disabling it, or having it and then it disables and the pilot has to start piloting the plane in a different way.

        Beware that the Max can keep on *increasing* the AOA with the same stick input - the pilot is required to decrease stick input to keep the same AOA - which may be not "instinctive" - especially when busy performing a take off and following a SID procedure, or landing, maybe in bad weather.

        The fact that the MCAS was ill-designed doesn't mean that without it the airplane is safer.

        There are two big risks - automation that hinders pilots to control the plane in the proper way, and automation that properly disables forcing pilots to take control - and not everybody does always the right thing.

        These two accidents are exactly an example of each.

        1. Stork Silver badge

          Re: "forcing the nose down against the pilot's mistaken reaction to a stall"

          I think one of the problems is that Boing (and FAA, or with their implicit consent) glossed the MCAS over and barely mentioned it, in order to pretend it was basically the same plane as the launched in the late 60es.

          My impression was that the pilots were not in details told about MCAS and what to do if they thought it played up - which is why you found pilots dragging out the manuals at a critical time...

      2. James Anderson

        Re: "forcing the nose down against the pilot's mistaken reaction to a stall"

        Early (prototype) Airbuses had a similar problem.

        If the software detected wheels down and low altitude it went into labour saving mode and did an automatic landing.

        A test pilot was showing off the low speed handling at the 1988 Paris Airshow when the software took over and did a perfect landing in some nearby forestry.

        This feature never made it to production and avionics engineers have been very cautious about automatic takeover of control ever since. (obviously the message did not get through to Boeing).

    4. Chairo
      Mushroom

      Re: Appalling software, it seems

      " Alternating between sensors on alternate flights was terrible design and seems like superstition rather than science."

      Actually it is the worst possible design choice. A system that fails only every second time is really difficult to diagnose and will lead to disaster just as surely than if it fails every time.

      Technician 1: Hey, I think this test failed.

      Technician 2: No, I think you made a mistake, try it again!

      Technician 1: Oh, you are right, it is working now. Thanks!

      -> Boom

      1. Adrian 4

        Re: Appalling software, it seems

        AIUI it's not the sensors that were swapped on alternate flights, but the entire flight control computer (if that's the right term .. not my area of expertise). Each computer had its own dedicated sensor, and as a result the active sensor changed with it : because each computer had a dedicated sensor. It's not even clear if it's able to cross-check them. Presumably there's some sort of link that allows one machine to read the other's sensor, but only indirectly.

        This doesn't sound much better than superstition either, but presumably was/is a long-established protocol with some level of justification. It wasn't new for MCAS.

  4. Anonymous Coward
    Anonymous Coward

    The Best Analysis Of What Really Happened

    https://www.zerohedge.com/news/2019-03-17/best-analysis-what-really-happened-boeing-737-max-pilot-software-engineer

    I'm a software engineer, and we're sometimes called on to fix the deficiencies of mechanical or aero or electrical engineering, because the metal has already been cut or the molds have already been made or the chip has already been fabed, and so that problem can't be solved.

    But the software can always be pushed to the update server or reflashed. When the software band-aid comes off in a 500mph wind, it's tempting to just blame the band-aid.

    — Trevor Sumner (@trevorsumner) March 16, 2019"

    1. bazza Silver badge

      Re: The Best Analysis Of What Really Happened

      Yep, and it seems like Boeing just want to stick on an even stickier band aid. Great.

      1. bombastic bob Silver badge
        Unhappy

        Re: The Best Analysis Of What Really Happened

        it's a fair bet they'll be compelled to retrofit planes with the $80k warning light option/upgrade. If not, they should be. THAT and some updated training materials, better software. Probably won't be done with it for a little while...

        1. anothercynic Silver badge

          Re: The Best Analysis Of What Really Happened

          They are already doing that. That was the only thing they could kill some of the bad PR... especially when it became clear that the 'disagree lights' were 'optional safety items'. The training manuals are the biggest problem. They didn't mention MCAS.

          1. Anonymous Coward
            Anonymous Coward

            Re: The Best Analysis Of What Really Happened

            In a December 2016 FCOM I have, they did mention MCAS... in the glossary, stating only the expansion of the acronym -__-

    2. martinusher Silver badge

      Re: The Best Analysis Of What Really Happened

      Software may just be clouding the issue. I was watching a video about this plane that described the elevator and trim system; it has what's called a full flying stabilizer that provides the trim with the usual hinged part at the back as the elevator proper. This setup should work well except it has one potential flaw -- if you give it full down trim what you're doing is increasing the angle of attack of the stabilizer at the back and if you pull full up elevator you're significantly altering the airfoil by bending the rear part upwards. This could result in a tailplane stall. I've only seen this in light planes and models, you don't encourage it because its bad news -- you lose control of the plane, it suddenly goes into a steep dive and recovery can be difficult because the odd attitude of the plane may prevent air from flowing properly over the tail and so re-establishing control. Its can be really difficult to recover from. (Stalling the tailplane is the sort of maneuver you don't want to do in a model, you really don't want to do in a light plane unless you're flying with a really experienced instructor and have a fair bit of height to work with and you definitely don't want to do in an airliner full of people.)

      Nothing the software could do should be able to make a plane go out of control. I won't go near a MAX until there's been a satisfactory explanation for how these planes crashed. The MCAS may have started the chain of events but if control inputs can cause the plane to suddenly go out of control then this needs to be fixed before the plane is safe to use.

  5. Mayday
    Thumb Down

    Pull Breakers

    A shitty way to disconnect a system.

    I'm a pilot (certainly not the ilk of a 737 jockey) and if the way to turn somehting off or otherwise stop it was to pull a breaker as opposed to a simple red button somewhere that's not good enough. Sure, for something like a trim or flap motor being stuck is one thing, but to stop the computer kicking in and doing something like this isn't good enough.

    1. Jos V

      Re: Pull Breakers

      Just a small correction there. It's two switches next to each other, covered by a guard lever. They're located on the right side of the center console, below the flap lever.

      1. simonlb Silver badge

        Re: Pull Breakers

        If that is the case, then if I were having to pilot one of these for the first time, I'd ask, "What are these two switches down here? I've never seen these before. What do they do?"

        If these are readily accessible to the flight crew right next to the other standard flight controls then they must relate to something critical to normal flight operations and therefore I'd insist on knowing the full reason for their requirement otherwise I'd refuse to fly it.

        1. Jos V

          Re: Pull Breakers

          There is a procedure for "runaway elevator trim" in the flight manual. However the situation calling for this was when the original STS (speed trim system) was installed only, where it would trim nose-up at constant rate.

          MCAS does a nose-down trim, and switches off for 5 seconds after the pilots counter trim from the yoke, and then repeats.

          The pilots were not familiar with this behaviour, so it didn't trigger the action to turn off the two switches.

          On the flight preceding JT610, a Batik air captain (typically the grey-beards coming out of long service from sister company Lionair), who was also 737-max8 rated and sitting in the observer seat, did conclude it had to do with auto-trim (he wasn't in control, so had a bit more room to evaluate the situation), and advised the flight crew to turn the switches to off position, which recovered the situation.

          1. Jos V

            Re: Pull Breakers

            Having said that, one thing seems to have been missed in the news. Both aircraft went very fast at low altitude. Whether that was because of unreliable airspeed indication, or because no attention was given, I'll leave that to investigators, but..

            Airplanes have a nasty characteristic in this situation. When there is a pitch down , at high speed, low altitude, the earodynamic forces on the elevator will actually go into blow-back. The hydraulic actuators won't be able to avoid this and the control surfaces will be pushed to full nose down.

            The only way to get out is to reduce speed, and full pull on yoke, plus full up trim. When already pointing down, this is almost neigh on impossible. Certainly at an altitude without margin. Ethiopian was at 9000ft, but 1000ft above ground...

            Both aircraft ended doing a full nosedive in the last moments.

            Don't take my word for this. I don't do conspiracies. You can look it up.

            1. I ain't Spartacus Gold badge

              Re: Pull Breakers

              Jos V,

              I guess you'd expect the planes to be going fast. If they'd got the throttles set to high power for climb-out, but weren't climbing because the stupid MCAS is trying to kill them, then the plane is going to be gaining speed instead of altitude. Which perhaps they were too busy to notice, or wanted to keep high power as they were trying to get the plane to go up, if only the computer wasn't actively working against them.

              1. Jos V

                Re: Pull Breakers

                That's exactly what implied Spartacus. We don't know the reason yet so don't step ahead of yourself. And as for the Boeing vs. Airbus dispute, I don't have time for those people.

                What is wrote is factual without bias.

                We will find out. And then we can all banter about how we all know better.

    2. wolfetone Silver badge

      Re: Pull Breakers

      "I'm a pilot (certainly not the ilk of a 737 jockey) and if the way to turn somehting off or otherwise stop it was to pull a breaker as opposed to a simple red button somewhere that's not good enough"

      A kill switch was probably an added chargable extra, like the $80,000 warning light.

    3. imanidiot Silver badge
      Coat

      Re: Pull Breakers

      The way you talk about 737 pilots and simple red buttons makes me think you're full of shit to be honest. Certainly it's not hard to read up on what likely happened, how the trim system works, how the cutout switches work and how MCAS functions. No need to be badly informed. Read up before spewing plz.

    4. Anonymous Coward
      Anonymous Coward

      Re: Pull Breakers

      I been told of an Autopilot system on board a certain aircraft where there's a handle that will physically pull an actual pin, thus mechanically disconnecting the Autopilot actuators from the flight control cables. Yank the handle, the pin is pulled out, and the Autopilot system is *physically* and *mechanically* disconnected.

      The Autopilot can then flail and oscillate and generally go insane, to no effect (once the handle is pulled).

      They'd be wating to keep that pin polished smooth and very well greased.

      Nowadays we trust software too much. We shouldn't.

  6. Mark 85

    I have this gut feeling that they may get it right..."may" being the key word. But I do feel bad about all the airlines and their customers being caught up in this mess. Once Boeing gets the fix and any hardware, it will not be an overnight task to install it.

    The thing is for the $80,000 they wanted for the "upgrade" should have been a standard item on all the aircraft. I guess profits and marketing run things and not engineers. I would hope that some governments take an interest in looking at the certifications and that Boeing task some heavy hits to stock value, the board, and the upper manglement. Yeah.. wishful thinking on my part.

    1. Anonymous Coward
      Anonymous Coward

      Doesn't matter how long it takes to install

      If it is determined the software fix isn't sufficient and they need a hardware fix in the form of a second/third AoA sensor and disagree light, they will have plenty of time to install it on the grounded Max 8s while they wait for the various authorities to recertify the plane. If they could install the fix overnight they'd still have to wait for the recertification.

  7. Donn Bly

    Why not just disable the system and put the planes back in the air?

    Every pilot that sits in one of these aircraft is now painfully aware of the MCAS system and why it was installed, and since the system was designed to offset the unawareness of the pilot the MCAS is now at best superfluous and at worst detrimental to the operation of the aircraft.

    We know by now that the planes CAN fly without the sensor and its attempted automatic corrections -- and probably fly safer without it -- so why not disable it and get them back in the air?

    The only thing that comes to my mind is that the powers that be want to toss all certifications for the aircraft and start over, since they no longer trust any of them. that seems something like throwing the baby out with the bathwater. Still, I would feel a lot safer and would rather fly on a 737 MAX than ride cross-country on the highways in an automobile -- especially during spring break season over here in the states.

    1. Richard 12 Silver badge

      It's not flyable without it

      Boeing insisted that pilots trained on the 737NG can fly the MAX without costly simulator time, just a short paper course.

      They've insisted that before, and 47 people died when a brand-new 737-400 crashed near East Midlands Airport because the pilots only had paper conversion training from the previous version of the 737.

      It appears that Boeing forgot the lessons of 8th Jan 1989 and the 737-400 when launching the 737MAX.

      FFS Boeing. How many people have to die before you get it into your heads that ALL new aircraft REQUIRE simulator training? A similar model number is not enough!

      1. sanmigueelbeer

        Re: It's not flyable without it

        Boeing insisted that pilots trained on the 737NG can fly the MAX without costly simulator time

        The MAX family was born out of panic. American Airlines told Boeing that they were buying 250 Airbus A320neo because Airbus promised up to 15% fuel savings. So when the news broke, Boeing executives were terrified that other operators would follow American Airline's lead.

        Boeing's initial plan was to design a brand new model from scratch. Because this was a completely new design, it will have to undergo a lot of testing and verification stages before anyone can fly them commercially. This means time. And time is what Boeing doesn't have at that time.

        So Plan "B" was to follow Airbus and strap the same engines to a 737 (and give it a commercially pleasant name). However, Boeing executives were pressed for time. They can't make too many design changes that will trigger a full blown FAA recertification. So every design had to look like a "touch up". They replaced all the analogue gauges (aka steam guages) to digital, for instance, but improving the electronic displays was pushed back because this will require pilot retraining.

        Another thing that Boeing did was to make changes that will not trigger pilots requiring flight simulator time.

        And because of this, Boeing "downplayed" the importance of the MCAS.

        So the MAX was born in 2 years (which took half the time) and, if I remembered correctly, the first commercial flight happened 2 years later.

        1. Anonymous Coward
          Anonymous Coward

          Re: It's not flyable without it

          Yep.

          It also seems that Boeing weren't terrified enough of the consequences of getting the design and certification shortcuts wrong, and weren't terrified enough by the possibility of crashes ensuing from them.

          That's their real problem. Fly something / anything cannot be allowed to be more profitable than doing it right, but that's what they seem to think will work.

      2. sanmigueelbeer
      3. fobobob

        Re: It's not flyable without it

        Worse still, that (Kegworth crash) was due, in part, to pilot confusion over the source of cockpit smoke from a failed engine; the -400 has a different air conditioner configuration from -300, and pilots were not properly made aware. Assumptions about which engine supplied air to the cockpit lead to them shutting down their remaining good engine. You'd think Boeing would be even more concerned about anything even remotely related to actual flight controls.

    2. Dave K

      Re: Why not just disable the system and put the planes back in the air?

      Without MCAS, the 737 MAX is more prone to stalling due to higher-than-expected nose lift as a result of the different engines being mounted further forwards.

      Sure, the FAA could request airlines to disable the feature and resume flying, but can you imagine the fallout if a pilot (who hasn't had additional training because the 737 MAX supposedly doesn't require it) inadvertently raised the nose too much because he wasn't used to the different flying characteristics of the 737 MAX without MCAS, stalled the plane and crashed it?

      You can't take risks with people's safety. Boeing needs to fix the 737 MAX so that MCAS relies on multiple sensors to determine AoA, has a clear warning device fitted as standard to notify pilots of an AoA sensor failure, can't drop the nose of the plane sufficiently to cause a crash, and to ensure pilots are trained regarding how to control the MCAS system (and to disengage it in the event of an emergency). Only then will the 737 MAX be considered safe to resume flights.

      1. Jimmy2Cows Silver badge

        Re: Why not just disable the system and put the planes back in the air?

        Only then will the 737 MAX be considered safe to resume flights.

        But will anyone want to fly on one? Even if full recertification and proper pilot training happen, trust is surely completely destoyed at this point.

        1. dogcatcher

          Re: Why not just disable the system and put the planes back in the air?

          Just to be on the safe side I think I'll avoid travelling in anything made by Boeing.

        2. Dave K

          Re: Why not just disable the system and put the planes back in the air?

          It depends, trust has been damaged but can recover. The DC10 suffered from a fatal design flaw and had to be grounded whilst modifications were made, but has then had a long service career. I don't think people will be the problem here. If the plane is modified, the modifications scrutinised and declared safe and the plane returned to service, I don't think there'll be too many issues with people using it. Whether airlines themselves get cold feet and stop ordering it (as happened with the DC10) is another story of course...

          1. I ain't Spartacus Gold badge

            Re: Why not just disable the system and put the planes back in the air?

            DaveK,

            I've forgotten who now, but an airline cancelled an order for 50 737MAX yesterday. It's got a lot of orders. So Boeing can survive that. But on the other hand a lot of the orders are for huge numbers of the things, so it only takes a few airlines cancelling to be losing hundreds of orders.

            1. A.P. Veening Silver badge

              Re: Cancelled orders

              Garuda already cancelled four days ago, see: Indonesia's Garuda is canceling its $4.9 billion order for the Boeing 737 Max.

        3. PhilBuk

          Re: Why not just disable the system and put the planes back in the air?

          Put the Board and entire Marketing dept. on the plane and fly it round the world. Then maybe we will think about it. Schedule a stop off at Lukla and try landing there. The result would be a benefit to mankind.

          Phil.

    3. Francis Boyle Silver badge

      Re: Why not just disable the system and put the planes back in the air?

      We know by now that the planes CAN fly without the sensor and its attempted automatic corrections -- and probably fly safer without it -- so why not disable it and get them back in the air?

      How many thousands of lives are you prepared to bet on that evidence-free 'probably'?

      Entrusting people's safety to under-trained pilots is more than 'probably' a very bad idea.

    4. Cuddles

      Re: Why not just disable the system and put the planes back in the air?

      "We know by now that the planes CAN fly without the sensor and its attempted automatic corrections -- and probably fly safer without it -- so why not disable it and get them back in the air?"

      Why do you think the MCAS system is the only problem? The issue with MCAS has exposed the fact that the 737 MAX has not been properly tested. Having now discovered at least one serious deficiency in the basic design, as well as a host of corresponding deficiencies in the management and training side of things, only a fool would blindly assume that this must be the only problem present. At this point the only option is to go right back to the start and do full testing and certification from scratch as if it's an entirely new plane, which it effectively is.

    5. Peter2 Silver badge

      Re: Why not just disable the system and put the planes back in the air?

      We know by now that the planes CAN fly without the sensor and its attempted automatic corrections -- and probably fly safer without it -- so why not disable it and get them back in the air?

      We now know about this fault, which has crashed two aircraft killing three hundred and forty six people. This design flaw should have been picked up by the regulator; my understanding is that the aircraft has two sensors fitted but the software design was to use one of the sensors, and then ignore the input from the second. The result is that if that single component provides bad input (with doesn't require a malfunction; a bunch of insects crawling in could provide bad readings) then the aircraft is deliberately flown into the ground by it's software complete with screaming pilots yanking at controls that don't work.

      While the revised software design is using both sensors, doing a sanity check on the incoming data and disabling the system if the inputs are radically different, the quality of the initial design is well beyond appalling. If i'm writing code then I make better design decisions doing a website, which does not conceivably have lives depending on it.

      The regulator did not pick up on this problem, as Boeing provided them with factually incorrect information pertaining to the design. Yes, this particular fix has almost certainly resolved this particular problem. However, what other problems are lurking within the rest of the aircraft that nobody knows about?

      Having been caught lying to their regulators to gain certification Boeing has in effect nullified their certification since what they have said now cannot be trusted, and dead bodies exist to prove it. I suspect that while the US regulator might get pushed into letting it back into the air quickly on the basis that "if we don't, Boeing loses sales to Airbus!" this would be obvious to everybody concerned and the European regulators are unlikely to let it fly in European airspace. What Africa & China would then do is an open question, but one thing is for sure, this is not a good look for Boeing or the FAA and it may well impact American aircraft sales in the future.

      1. A.P. Veening Silver badge

        Re: Why not just disable the system and put the planes back in the air?

        "What Africa & China would then do is an open question"

        Africa is an open question, but taking the fledgling aviation industry in China into account, CAAC will insist on a very thorough re-certification, strictly and totally independent from FAA. If Boeing is luckier than it deserves, CAAC and EASA will do the re-certification together, but I am not going to bet on it.

    6. Kubla Cant

      Re: Why not just disable the system and put the planes back in the air?

      We know by now that the planes CAN fly without the sensor and its attempted automatic corrections -- and probably fly safer without it -- so why not disable it and get them back in the air?

      I have read that there is a minimum requirement for self-stabilising flight that probably cannot be met without MCAS.

      There's also the consideration that MCAS was supposed to make flying the 737 MAX just like flying its predecessors, so no training required. Disable MCAS and all the airlines incur a training requirement, which they were assured they wouldn't need. Who pays for that?

      1. imanidiot Silver badge

        Re: Why not just disable the system and put the planes back in the air?

        What Kubla Cant mentioned is the main issue. MCAS was supposed to "fix" the nose-up pitch instability at low speeds when approaching a stall. Any pilot should be able to solve this in NORMAL conditions, but when you've been in a holding pattern in Class 3 IFR conditions for 3 hours and starting an approach in busy airspace a pilot is likely to be in less than perfect condition to keep the nose from suddenly rising if the airspeed drops due to windsheer for instance.

        The thing is that this aircraft likely wouldn't even be certified as Airworthy without MCAS. It certainly isn't a system that can be just left out.

        1. Jos V

          Re: Why not just disable the system and put the planes back in the air?

          imanidiot, just one caveat there. MCAS doesn't operate when flaps are extended, or A/P disabled, and only works under high thrust, low speed. You'd find that on approach none of these conditions would be met.

          When you look at the FDR data from JT610, you will see that the problems started as soon as they retracted flaps to 0.

    7. Mark 85

      Re: Why not just disable the system and put the planes back in the air?

      We know by now that the planes CAN fly without the sensor and its attempted automatic corrections -- and probably fly safer without it -- so why not disable it and get them back in the air?

      MCAS is/was a bandaid, a patch. Kill it and what else is waiting for the unsuspecting? If we believe the reports in the media, this whole plane is basically cobbled together, training is minimal. Since the cost of training is an issue with the airlines bottom line, it's being cut. Even this "patch" was an option that more than a few airlines didn't buy.

    8. sanmigueelbeer

      Re: Why not just disable the system and put the planes back in the air?

      We know by now that the planes CAN fly without the sensor and its attempted automatic corrections -- and probably fly safer without it -- so why not disable it and get them back in the air?

      There is one other reason why no MAX operator will mandate turning off MCAS: Fuel savings. If an aircraft is frying the wrong pitch, it will translate to higher fuel consumption for that flight. "How much fuel will it save" is currently the latest battleground Boeing and Airbus are using hook potential customers.

      Airbus A320neo claims up to 15%. If turning off MCAS causes the flight to have a fuel savings of <14% it will be "bad". Every percentage point is a lot particularly when profits are razor thin.

  8. train_wreck

    $80,000?

    $80,0000 was the cost of this upgrade? I seem to remember reading that it was just a firmware update of sorts, changing the control logic to disable MCAS if the AOA sensors disagree. The "warning light" is located on a primary LCD interface in the cockpit, so no additional hardware was needed there.

    Could be wrong, as I'm no aviation expert.

    1. Anonymous Coward
      Anonymous Coward

      Re: $80,000?

      Ah, so basically an $80,000 fee for the "please don't kill me" option.

    2. Trollslayer

      Re: $80,000?

      Correct, it just displays the sensor status on existing displays.

      1. imanidiot Silver badge

        Re: $80,000?

        No, thats the AoA indicator option (Which will remain optional). The AoA disagree option now proposed by Boeing is simply a red flag in the display saying "AoA disagree", that AFAIK doesn't actually provide anything about the actual sensor reading.

        1. Anonymous Coward
          Anonymous Coward

          Re: $80,000?

          About as much use, in fact, as a black button which when you press it causes a black light to come on under a black sign reading "Do not press this button."

          1. hopkinse

            Re: $80,000?

            That's for the Hotblack Airlines version :-)

            1. Anonymous Coward
              Anonymous Coward

              Re: $80,000?

              Currently dead for tax purposes.

  9. Anonymous Coward
    Anonymous Coward

    A kill switch?

    Should anything be called a ‘kill switch’ on a plane or is this Boeing’s new name for the autopilot?

    1. Anonymous Coward
      Anonymous Coward

      Re: A kill switch?

      Tesla already copyrighted that as the new name for their Autopilot feature.

  10. Anonymous Coward
    Anonymous Coward

    "Safety is our highest priority as we design, build and support our airplanes,"

    Well, it clearly isn't is it, or you would have fitted that cockpit light to every MAX sold, you profiteering wankers.

    1. Anonymous Coward
      Anonymous Coward

      Re: "Safety is our highest priority as we design, build and support our airplanes,"

      You should read the ceospeak "Safety of our bonuses and shares value is our higher" etc. etc

      They bet the A320 Neo would have been a failure, when they were shown they had been wrong they started to panic...

    2. Pascal Monett Silver badge

      It clearly isn't because if it were, Boeing would have required input from multiple sensors and correlation with a gyroscope or something.

      Seriously Boeing ? A lone data feed can instruct the software to push the nose down ? On the input of one single sensor you change the attitude of a passenger-carrying plane ?

      There was a time when redundancy was the name of the game. Once upon a time an airplane had three computers doing the calculations and at least two had to get the same results before any action was taken.

      Am I to understand that Boeing is now using only a single Playstation for its calculations, and there wasn't enough USB ports to allow data from another sensor ?

      1. Hans 1

        Airbus introduced this feature on the A320, the very first A320 in 1987, and they use three sensors, if one is giving incorrect info, the other two must give the same or the system will disengage. If is NOT infallible, though ... as for the Rio-Paris crash, they decided to go through a storm, the pitot tubes froze, auto-pilot disengaged, a co-pilot kept pulling the nose up and as the aircraft stalled, he continued to pull the nose up, ignoring NOSE DOWN advice for MINUTES .... until the captain came back from rest, trying to figure out what was wrong ... he noticed the pilot's mistake, but sadly much too late - how that co-pilot got through pilot training is anyone's guess.

        So, Airbus systems disengage if there is a doubt, however, the system expects somebody in control who can actually fly an aircraft ...

        1. Jos V

          Hans 1. No and Yes. AF447 was a 330, not 320. And the 330 uses 3 primary (PRIM) and 2 secondary (SEC) computers between all controls.

          320 has 2 ELAC, 2 FAC, and 3 SEC computers.

          The issue was more or less (it's much more complicated than that) that there is no physical connection between captain and FO sidestick. If you push full nose down on the left, and full nose up on the right, the result is no action at all. This is indicated by a control disagree alarm, but I'm not sure if this was implemented before, or after AF447. Same as a button on each sidestick that when pushed, says "I have control now", and the other side gets cancelled.

    3. batfink

      Re: "Safety is our highest priority as we design, build and support our airplanes,"

      "Safety is our highest priority" (except for the opportunity to charge an extra $80,000...)

      1. Alan Brown Silver badge

        Re: "Safety is our highest priority as we design, build and support our airplanes,"

        "(except for the opportunity to charge an extra $80,000...)"

        To put that in perspective - $80k is about what you'll pay to get 4-5 hours at 20k feet in the left hand set at the pointy end of a real (but empty) 737 (it's a lot cheaper in a simulator)

        Yes it's a blatant rip off but in the overall scheme of aircraft costs it's small change from down the back of the sofa.

  11. Anonymous Coward
    Anonymous Coward

    "Relatively few pilots were aware of MCAS, though: it wasn't mentioned in the basic 737 Max pilot's manuals."

    Really ???? That's criminal, if true. How can you lie by omission to the pilot and make him unaware of any system running in his aircraft ?

    "The aircraft's designers did come up with a safety warning light to alert pilots if there is an error in the angle-of-attack sensor data feeding into MCAS, a condition that would lead to the safety system making lethal decisions. But this warning light doesn't come as standard, "

    Same comment as above. Criminal.

    I really feel for the poor pilot (and the passengers) who struggled like a madman to go up, while an undocumented system was bringing the nose down ...

    1. Phil O'Sophical Silver badge
      FAIL

      it wasn't mentioned in the basic 737 Max pilot's manuals.

      Good documentation costs money, standard practice these days was probably just to tell the designer to write a blog entry about it.

  12. jmch Silver badge
    Thumb Down

    ' "Safety is our highest priority as we design, build and support our airplanes," Boeing said last week in a statement.'

    Clearly it's not or the 2 crashes wouldn't have happened

    1. Anonymous Coward
      Anonymous Coward

      The truth

      ' "Profit is our highest priority as we design, build and support our airplanes," Boeing didn't say last week in a statement.'

  13. Trollslayer
    Flame

    US and self certification

    Aircraft manufacturers have latitude with self certification in the US.

    Anyone else remember the 787 from Japan to the US where the Li-ion batteries nearly caught fire?

    Another twenty minutes in the air and...

  14. Hans 1
    Joke

    $80,000 for an LED light, crap, I did not know Apple were in the avionics business.

    1. vtcodger Silver badge

      Design by Apple.

      Apple would probably mount the light on the bottom of the pilot's seat where it wouldn't interfere with cockpit aesthetics. And they'd power it with a non-user replaceable battery that had to be replaced annually by a $300 an hour Apple technician. 2 hour minimum service call. Plus drive time. Plus $2.00 a mile mileage from Cupertino.

      (But, in fairness, I am not aware that Apple design idiosyncracies have ever killed a customer).

      1. Anonymous Coward
        Anonymous Coward

        Re: Design by Apple.

        That is harsh and unfair, you failed to mention that in accordance with the latest iPhone design, if the lamp glass failed you would need to replace the lower half of the fuselage.

  15. caffeine addict

    The aircraft's designers did come up with a safety warning light to alert pilots if there is an error in the angle-of-attack sensor data feeding into MCAS, a condition that would lead to the safety system making lethal decisions. But this warning light doesn't come as standard, and many airlines, particularly at the cheaper end of the market, didn't order them. Boeing has now said the $80,000 upgrade will be installed as standard on all new 737 Maxes.

    Can you think of another industry where

    * a warning light (for a device that can actively try to kill you) is an optional extra

    * companies baulk at the idea of spending an extra 0.00006% for a safety device (based on $122m cost, according to wiki).

    1. Martin Gregorie

      Can you think of another industry where

      * a warning light (for a device that can actively try to kill you) is an optional extra

      * companies baulk at the idea of spending an extra 0.00006% for a safety device (based on $122m cost, according to wiki).

      Yes, of course - anywhere where that beancounters and MBAs make the final decisions.

  16. Anonymous Coward
    Anonymous Coward

    Executive testing required

    There's a very simple way to ensure this is fixed correctly.

    Send the board of management/CEO on all test flights for the software patch.

    http://aircrewremembered.com/parachute-packers-prayer-martineau.html

  17. TrumpSlurp the Troll
    Trollface

    And now for the missing word round.

    I haven't seen the word "retrofitted" in any of the reports.

    The previously optional warning light (gauge?) will now be fitted as standard on all new builds. Fine. What about all the existing aircraft without the warning light (gauge?)?

    Boeing now have the tricky task of persuading everyone that the new software and the warning light(gauge?) will not require recertification of the aircraft nor recertification of all existing 737 pilots before they are allowed to fly it.

    However, given the surreal state of politics on both sides of the pond anything is possible.

    1. Anonymous Coward
      Anonymous Coward

      Re: And now for the missing word round.

      "The previously optional warning light (gauge?) will now be fitted as standard on all new builds. Fine".

      The good news is that no one will be buying any of the new builds. So they can put in or leave out whatever they please.

    2. Jos V

      Re: And now for the missing word round.

      Well, the warning light is there already. It's on the glareshield and lights up as "WARN". Which will put the attention of the pilots to the ECAM display where it will say what it's about.

      Since MCAS was only reading from 1 sensor, there is no disagree, and hence no warning! Which is damn stupid.

  18. json

    Suffice it to say, this will push back further the idea of a pilot-less commercial aircraft.

  19. Anonymous Coward
    Anonymous Coward

    MCAS

    I think that it's a tragedy that so many people have died due to a potential software fault.

    All systems should have the ability to override from a pilots perspective. Even to reduce air speed/angle of descent.

    1. A.P. Veening Silver badge

      Re: MCAS

      "I think that it's a tragedy that so many people have died due to a potential software fault."

      Hardly a software fault, the software functioned flawlessly. Shame it got garbage as input, but given the garbage input, the garbage output was to be expected.

      This was a major hardware design fault.

  20. adam payne

    The aircraft's designers did come up with a safety warning light to alert pilots if there is an error in the angle-of-attack sensor data feeding into MCAS, a condition that would lead to the safety system making lethal decisions.

    How many instances in the past have there been where a single faulty sensor has brought down a plane?

    Why would you design a system that can make lethal decisions based on one sensor? Have they not learnt from previous crashes?

    But this warning light doesn't come as standard,

    The warning light that could save everybodies life is not standard, how stupid can they be?

    Boeing has now said the $80,000 upgrade will be installed as standard on all new 737 Maxes.

    What about the ones already built? Will they be upgraded free of charge?

  21. Anonymous Coward
    Anonymous Coward

    Did Boeing cut corners?

    Over on the PPRUNE the word on the street is that Boeing cut corners on the 737-MAX ... after using bigger engines and placing them further forward it is said that they should have re-designed the tail-plane and horizontal stabilisers but this would have been time consuming and costly and it appears they opted to implement MCAS instead.

    MCAS is understood to be disabled while flaps are set, suggesting that the problem occurred after flaps were retracted at the top of the take-off clime.

    An important consideration here is that MCAS can be activated by a single Angle of Attack (AoA) sensor which could become faulty and provide erroneous readings.

    Several other pilots have reported problems (near miss events?) with MCAS.

    There is also confusion as to what the FAA actually approved since MCAS was originally specified and tested with +/-0.6 degrees of stabiliser trim authority but some time later this changed to +/- 2.5 degrees and it is unclear whether this was re-approved.

    It appears that all 737-MAX aircraft currently operate with MCAS and have a potential single point of failure.

  22. Anonymous Coward
    Anonymous Coward

    Simple answer?

    Do not buy Boeing aircraft until they stop following default US corporate practice*. Others might consider any US products in this way.

    *US corporate practice is to maximise all profit margins until what the customer receives is not as good as it should be, or even what the customer things. (Yes, that is what Volkswagen did.)

    1. Anonymous Coward
      Anonymous Coward

      Re: Simple answer?

      Someone - Henry Petroski? - remarked long ago that most major engineering disciplines go through four stages:

      1. Can't do.

      2. Can do, but with unacceptable failure rate.

      3. Technique mastered: can do with virtually no failures.

      4. Back off the quality to the point where the financial impact of failures balances the cost of preventing them.

      I guess Boeing has traversed the whole sequence.

    2. Anonymous Coward
      Anonymous Coward

      Re: Simple answer?

      I guess I missed where Volkswagen AG was an American company? What you cite is followed by corporations the world over. Unless you think that non-US companies excluding Volkswagen are somehow immune from a profit focus.

      1. Potemkine! Silver badge

        Re: Simple answer?

        Unless you think that non-US companies excluding Volkswagen are somehow immune from a profit focus.

        Of course they aren't, however in the US the government does not its job to regulate the abuse, on the contrary, it does what it can to make the abuse possible in the name of ideology.

  23. Anonymous Coward
    Anonymous Coward

    Odd

    "The aircraft's designers did come up with a safety warning light to alert pilots if there is an error in the angle-of-attack sensor data feeding into MCAS, a condition that would lead to the safety system making lethal decisions".

    So... if the sensor goes wrong and sends incorrect data to MCAS, this warning light will tell the pilots why MCAS is trying to kill them (and will almost certainly succeed).

    Unless they happen to know how to kill the thing.

    1. I ain't Spartacus Gold badge

      Re: Odd

      There's a standard procedure for trim runaway - an already existing problem where the auto-trim system isn't working. And Boeing decided that MCAS failure should be treated in the same way. Which is to disable the the stabiliser trim motor - and do it by a manual wheel. The problem being that MCAS doesn't fail in the same way as trim runaway, so pilots weren't thinking that was the problem and responding in the way Boeing had hoped for.

      1. Anonymous Coward
        Anonymous Coward

        Re: Odd

        It is also intermittent (evy 10 or so seconds?). Which again, is rather confusion over most mechanical or electrical failures, that are random or constant.

        An autopilot constantly kicking in is confusing. I'll have to go back and read if MCAS is off when autopilot is off? That is also confusing, as if AP is off, then the pilots would not expect to turn off automatic things.

        But then again, if told "this aircraft has MCAS automatic trim/nose down" then yes, the first thing you'd think when excessive nose down and intermittent nose down kicks in, is turn the crap off!

  24. Ugotta B. Kiddingme

    It seems to me there is a relatively simple temporary solution

    I am NEITHER an engineer nor professional pilot and therefore lay NO claim to any authoritative knowledge on the matter. If you are a professional pilot and/or aeronautical engineer, feel free to correct me. This idea, however, seems logical to me:

    Until a more permanent solution is devised, instead of having the MCAS system actually force the airplane to pitch down when it thinks it detects an impending stall, have it blare an unignorable alarm WITH an audible voice giving instructions such as "pitch down immediately to prevent impending stall!" or something appropriately similar. The humans at the controls are ultimately responsible for the aircraft and its contents. Provide them with the urgent data and corrective course of action but let them actually PERFORM that action. Not a perfect solution, but it is a quick one that will prevent this specific type of tragedy from recurring.

    1. Anonymous Coward
      Anonymous Coward

      Re: It seems to me there is a relatively simple temporary solution

      MCAS is necessary because the aircraft must handle a certain way according to regulations. It has to do with how AoA must monotonically increase when the control column is pulled further back and monotonically decrease when it is moved forward. Thanks to the MAX's new engines, this doesn't happen "naturally" so must be implemented via software. A 737 MAX without MCAS cannot be certified to fly.

    2. A.P. Veening Silver badge

      Re: It seems to me there is a relatively simple temporary solution

      There is a correct, extremely simple, but maybe not so temporary solution: Ground those planes.

  25. Potemkine! Silver badge

    Consanguinity

    When a State Agency asks a private company to certify by itself the safety of its devices, you're sure people will die because of this policy.

    The ones who remove the means meant for the FAA to do a good work should be put in trial (even if I know this will never happen). They are part of the root causes.

  26. Cynic_999

    Cross check

    Apart from doing a cross-check to see whether both AoA sensors are giving the same information, it would be entirely possible to perform a sanity check using other sensors. There is a mathematical relationship between airspeed, mass, pitch angle and AoA. Knowing any 3 allows you to calculate the 4th. The flight computer should have the current mass of the aircraft (which changes as fuel is burned), and airspeed and pitch angle can be obtained from sources completely unrelated to the AoA indicators.

    But I question whether any aircraft that can become irrecoverably unstable should be certified for civilian passenger carrying duties.

  27. steviebuk Silver badge

    No its not

    "Safety is our highest priority"

    If it was this would of never fucking happened.

  28. error 13

    > instructed the on-duty pilots on how to disable the safety system so they could regain control of the aircraft

    I would applaud the irony of this statement if the outcome wasn't so tragic

    1. Anonymous Coward
      Anonymous Coward

      How many crashes?

      How many crashes are needed before this "safety feature disabling feature" becomes a dangerous aircraft?

      I mean, we can disable ABS/traction control on a car for when snow tires/chains are added. But if your ABS was known and designed to also turn on at 70mph randomly, applying rapid breaking and shuddering to the car, but "it's designed like that, and easily disabled by pulling these two breaker switches and reading this manual at 70mph"... then yeah, I'd get a *different* car. One that does not need MCAS in the first place.

      It's a design that is designed to fail. And not in a "fail safe" way. :/

  29. circusmole
    FAIL

    It seems that the 737 MAX...

    ..is somewhat like the latest generation of military fighter aircraft. Fundamentally unstable and impossible to fly without the aid of a computer. When the computer (for whatever reason) makes a mistake it's time to bail out - difficult in a 737 Max.

  30. Compuserve User

    Checking your own work

    So do you think there will be any criminal prosecutions due to the deaths from a faulty aircraft(s)?

    Should the FAA remove Boeing from their own internal QA process and employ a trusted 3rd party like NASA or SAIC?

    1. sanmigueelbeer
      Happy

      Re: Checking your own work

      So do you think there will be any criminal prosecutions due to the deaths from a faulty aircraft(s)?

      Sure, if the plane was manufactured by some two-bit company. But for a multi-billion dollar company, with fingers in every part of the government (including Capitol Hill), good luck on that one.

      Ethiopian Airlines is still waiting for the final result before making their move for a US-based lawsuit.

  31. Marty McFly Silver badge
    Childcatcher

    Useless article...

    MCAS was turned on, and they are making an announcement of it. Uh-huh. That is like saying "the wings were attached and the engines were operating". Of course, the media is going to jump on this like it is a smoking gun prior to any results of the investigation being released. Hype, hype.

  32. sanmigueelbeer

    Ethiopia set to release preliminary report into cause of Boeing 737 MAX crash

    Ethiopia set to release preliminary report into cause of Boeing 737 MAX crash

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like