Bad software crashed Boeings. Now it appears the company lacked a singular software supremo Boeing has created a new role for a vice president of software engineering and filled it with a veteran of similar gigs at SpaceX, Tesla, and Google. The Register reports the new job because Boeing’s appointment announcement points out that it previously lacked such a role – and also because the company has admitted that …
But it's not just the software.
Driving the software with non redundant sensors (which Boeing has done twice that I'm aware of so far) is not software but is certainly doesn't make that task easier.
Let's see if they can turn improve Boeing's delivery as well as its image.
I'm not going to downvote you but as a programmer or software application management boss you have to understand the environment that you are working in ... well, that's the way it used to be but the world has changed. These days it's not a pity that they didn't think it that important earlier because the standard is to get the app written and released - and "fix" the bugs with an update later.
"Only one sensor? No problem, I'll create a virtual one ...."
"But it's not just the software."
This, exactly.
Having worked there, including during a period when their software really hit the fan, so to speak, they suffer from placing it on a pedestal. To be handled only by the High Priests of the mainframe, sequestered in the monastery of the data center. The MCAS failure resulted in software that (from what I've read since I wasn't there during its inception) worked properly per specification. But the system design (including sensor hardware, cockpit warnings and flight crew training policies) all failed miserably.
You can't just cobble together an ill-conceived system and then hope to 'fix it in software'.
they suffer from placing it on a pedestal. To be handled only by the High Priests of the mainframe
That's not exclusive to them and it isn't new either. The exact same situation existed, for instance, at PSION in London in the 80s, but there it had admittedly at least less lethal consequences.
There are sometimes good reasons for silos, usually security, confidentiality or specific approval/audit/certification processes, but you need someone with sufficient authority* occasionally sanity checking if silo building isn't actually losing value rather than adding.
I agree with you that in Boeing's case, the error started with basic fundamentals. If you change an airframe so much that its basic characteristics dramatically change and you then decide to kludge your way past that, you better make damn sure that your kludge is at least as good as you can possible make it and watch it like a hawk. Which it wasn't, and wasn't. Cue planes nose diving, followed by reputation, shares and sales.
* Because otherwise exactly zip will happen otherwise
The specification must have been pure junk. The purpose of having two sensors is to increase reliability. If one fails you use the other, if you can tell which is correct, so you reduce failure rate by half.
The way this was implemented did the opposite. Since the software alternated between senors for sequential flights but only used one, they doubled the risk of failure since there were two parts to fail and both had to be working in order for two flights to complete successfully.
Even an idiot coder or specifier should have caught this.
Driving the software with non redundant sensors (which Boeing has done twice that I'm aware of so far) is not software but is certainly doesn't make that task easier
Just want to remind people that the MCAS is just one part of the equation.
The whole thing all started with a faulty reading from one of two the Angle of Attack (AoA) sensors. Take note: One-in-two.
One sensor is giving porkies. Who to believe? The left one or the right one? In both cases, the MCAS randomly accepted the input from the faulty AoA sensor.
Airbus has THREE (3) AoA sensors. Boeing put two in the MAX to bring the cost down.
In the meantime, EASA and FAA has quietly cleared the MAX to fly.
Sorry to have to pooh-pooh you, but it's not correct to say that 'Boeing put two AoA sensors in the MAX'. It implies that older versions didn't have two, but the 737 has always *had* two. But Boeing decided that to get around the problem of not having a 'quorum' of AoA sensors, they'd switch sides for MCAS on each flight.
They found that *adding* a third would add to the cost and also add to the complexity, and it would move away from the 'this is the same aircraft, just new engines and pretty new winglets, nothing else to see here, move along' mantra that allowed them to get away with grandfathering in a *lot* of old crud (which they wouldn't have gotten away with if the FAA had designated the MAX as a new plane). Ditto for the EASA requirement for a synthetic AoA (via the synthetic airspeed calculations that the 787 had). And EASA is still insisting on this change eventually with the MAX.
And also, no, FAA and EASA have *not* 'quietly' cleared the MAX to fly. There was no 'quietly' and there is no clearance yet. The FAA's chief pilot flew the plane and was positive, and the EASA chief said that their primary concerns have been appropriately addressed (and the findings are being analysed by the JOEB in Gatwick). But they have *not* cleared the jet to fly commercially yet.
It implies that older versions didn't have two, but the 737 has always *had* two. But Boeing decided that to get around the problem of not having a 'quorum' of AoA sensors, they'd switch sides for MCAS on each flight.
You are correct but MCAS only appeared in the MAX, yes?
What I am saying is the MCAS focused entirely on a faulty AoA sensor and started a nose down. MCAS did not consider "verifying" at the speed or the altitude when doing the nose down. It just kept doing it until impact to the ground.
Another thing, of all the MAX customers, only Southwest Airlines purchased the option to enable (in software) the AoA Disagree Alert. If I remembered correctly, the AoS Disagree Alert costs about US$40k and it shows up as a "AOA Disagree" in the PFD. That is all it does. For US$40k.
After the crash, Boeing made the AoA Disagree Alert a "standard" ... even though the AoA Disagree Alert did/does not really work.
If I remembered correctly, the new software means if one of the AoA should provide faulty readings, it will only inform the cockpit and not wrestle control.
So you think it's ok to blackmail the customer with a chance that all will die, for a feature that cost the manufacturer nothing to switch on? Is that getting maximum security out of the aircraft?
I think it's utter madness, personally.
"So you want to know if your sensors are acting up? Well, pay $40k then..."
Why not make fuel readings cost options too? "You need higher precision on the last 10% of fuel? OK, pay $100k." "It's just a software toggle, but we like fleecing our customers."
Focusing on *any* single aspect of this picture is asking for trouble. Here's something I posted here on 3 Aug 2019...
The MCAS kit as originally specified was allegedly intended to have a limited-authority (maybe 25% of jackscrew travel, or something like that??) one-shot effect on a flight control surface. Perhaps in those circumstances it *might* just about have been acceptable to not have much resilience designed in (but the system might also have not had the authority to achieve the intended effect either).
As time went by, the fundamental MCAS design got transmuted into "keep retrying till the aircraft/system is back in control. No limits." So 25% authority on a one off basis, to full authority, whatever it takes, and nobody considered it might call for improvements in sysem resilience and recovery mechanisms?
Presumably MCAS variations got a "delta" design review rather than a "start from a blank sheet of paper" review, just like the 737 in general hasn't had a proper design review for decades.
(This from one of the earlier well-informed blogs on the subject - aircurrent maybe?)
....
See also
https://www.seattletimes.com/seattle-news/times-watchdog/the-inside-story-of-mcas-how-boeings-737-max-system-gained-power-and-lost-safeguards/
"Two people involved in the initial design plans for MCAS said the goal was to limit the system’s effect, giving it as little authority as possible. That 0.6-degree limit was embedded in the company’s system safety review for the FAA."
...
"It also calculated what would happen on a normal flight if somehow the system kept running for three seconds at its standard rate of 0.27 degrees per second, producing 0.81 degrees of movement, thus exceeding the supposed maximum authority.
Why three seconds? That’s the period of time that FAA guidance says it should take a pilot to recognize what’s happening and begin to counter it.
Boeing assessed both of these failure modes as “major.” Finally, the analysis looked at the inadvertent operation of MCAS during a wind-up turn, which was assessed as “hazardous,” defined in a cold actuarial analysis as an event causing serious or fatal injuries to a small number of people, but short of losing the plane (that’s called “catastrophic”)."
Worth a look.
Exactly. The question is really, will "Greg Hyslop, Boeing chief engineer and senior vice president of engineering" insist on the necessary hardware and pilot training.
Boeing hasn’t launched a new commercial aircraft since the Dreamliner in 2004. It delivered the first of those planes in 2011. In the short run R&D suppresses quarterly results, and quarterly results are what earn bonuses for the Chicago office.
Part of Boeing's problem is their continuing reliance on "wetware" in an increasingly automated environment. Too many of their systems still rely a human in the loop to look at an instrument and think "Yeah that's not right." when it misbehaves.
All well and good provided the human sees the problem soon enough, and intervenes quickly enough, but absolutely deadly when the computer gets the "bit between its teeth". Thus there were several near misses with Boeing's flare retard landing system and MCAS before disaster inevitably struck.
I am sorry, as a pilot I am not at all comfortable with software wrestling control of an aircraft away fro a human pilot. Software should alert and/or assist but not take control and fight the pilots decisions. It is the pilot that makes the decision that "yeah, that's not right".
That has struck in my mind since I first read it many, many years ago. As a young teen at the time I'd probably say that was and is the core basis of my entire critical thinking process whenever I hear statements or announcements like this, especially if a politician is doing it :-)
This post has been deleted by its author
That is pretty much the point of "formal japanese":
A language where one can deliver a coherent, gramatically correct speech, that says absolutely nothing at all, offending no-one because it is expected that nothing is going to be said and the intended audience appreciates the effort of the performance rather than the content of it.
Boeing needs a complete safety team. That means you need not only software people, but safety engineers; there is such a field and it is common in avionics companies that take safety seriously.
Then systems engineers, ITAs (Independent Technical Authorities) who can be Boeing employees but have a veto over something going out of the door (cannot be overridden - the ITA is the ultimate signatory) and various design authorities. The list is quite long for a proper safety organisation.
Unless and until the Boeing attitude changes (so they become engineering focused, not MBA focused) this is simply putting lipstick on a pig.
Agreed.
Once upon a time, Boeing was all about safety. Boeing was the definition of safety. Boeing didn't need a software supremo because everyone had safety in mind.
That lasted until the CEO was no longer an engineer.
Like NASA, the beancounters took over and now here we are. No moon base, and no more redundancy in Boeing planes.
Get the beancounters out of decision roles, stat !
The most irritating thing I've found about beancounters is that they always assume that we engineers are just nerds who are only interested in our toys and aren't aware of real world things like costs, manufacturability and reliability. You get ignored or patronized and quite often you end up wasting effort working around their edicts. To an engineer they're just ill-informed and inexpert, people who are so convinced of their primacy that they don't take input unless its what they want to hear.
The tendency to outsource stems from their primacy. Their detachment leads to thinking of the core business as overhead (even though they're often taking the lion's share of renumeration) so they're prey to any slick salesman that comes across with a 'we can do it cheaper' line. (Because they can -- few beancounters know what 'loss leaders' are).
... always assume that we engineers are just nerds who are only interested in our toys and aren't aware of real world things like costs, manufacturability and reliability.
To be honest, In My Experience, from vascillating between being an engineer and a project manager, engineers (and scientists) are never going to be finished with something on their own volition.
As long as there are some tweak, adjustment, improvement, et cetera, to be made, most engineers will put that at "80% done" and they just won't stop tinkering.
One needs a Lead Engineer, Product Manager or Project Manager with authority to have to go at the requirements, "Does it meet x, y, z, q ... ... well enough?" and yank it from their hands when it does.
Case in point - the "Leave it to the boffins" Corona Response -> Chaos and Dithering. Politicians are needed to take important decisions on an ill-informed basis*, because the experts will *never* do it as long as the possible solution space has not been exchausted yet!
*) And to take the flack for it too. Sadly in the US, UK and SE we have politicians who can't take the flack so they won't make any decisions, letting the experts tinker and fiddle forever as they are wont to do.
Various Boeing have crashed because of metal fatigue. They didn't create a minion of metal fatigue.
If software is on the plane it's the vp of engineering's problem. This is just a way of saying - our engineering isn't crap, it was just a software problem and we're fixing that with a new job title
Metal fatigue is a problem that should be identified during maintenance. I this not a design/engineering issue. If you built aircraft that never experienced metal fatigue they would be so heavy they would never get off the ground. This is a regular occurrence in military aircraft, which are pushed to their limits, it is checked for after every sortie. I assume it is checked for in commercial aircraft, just not as regularly.
Yes the software was crap, but that wasn't the root of the issue.
The root of the issue was they they fitted the 737 airframe with new engines the cowls of which were basically too large and really mandated a complete redesign of the airframe. However they didn't do that, they moved the engines forward so that they could be moved upwards for the cowls to clear. This lead to the centre of thrust of the engines being in the wrong place. This in turn meant that the plane could (weasel words time) exhibit undesirable flight characteristics. They decided to try to fix these flight characteristics with a cludge in software.
Having a software supremo would have been unlikely to have prevented this situation.
What Boeing were trying to do here was basically fit new more efficient engines to an airframe that wasn't suited to them. They wanted to do this not only to save money in development but also to save their customers money in aircrew training. A whole new plane would not only have cost more to develop and have taken longer to get to market, but it would have meant customers needing to get their aircrews certified on a new plane. However if they could just sell it as a 737 there would be no need for customers to retrain their aircrews which would make the plane more attractive to existing 737 users.
Basically it was Boeing gaming the system to make money.
What the worlds regulators need to do is send a clear message to Boeing and the FAA by refusing to certify the MAX under whatever name Boeing try to relaunch it as a new iteration of the 737 and define it as a new craft requiring re-certification for aircrews.
This post has been deleted by its author
It had people at the FAA to muddy the waters and smooth things over, and it used that advantage to keep the FAA from taking a good look at what was going on.
Yes, Boeing cheated, but IMO the FAA has a large share of responsibility in this matter and nobody is talking about that.
If the FAA had done its job properly, it would never have accepted to just wave the MAX through and would have rightly decided that pilots needed recertification.
The FAA does not exist to save customers money, it exists to save people's lives, and it utterly failed in this instance.
You are correct.
The FAA allowed Boeing to carry out a lot of it's own certification. The FAA's argument for which is that proper certification would take too long. Which shows that the body in charge of certification doesn't really believe that certification is necessary. Kind of like the DVLA taking your word for it that you can drive and handing you a licence.
Which is why I said that the world's regulators need to send a clear message to Boeing and the FAA.
"Kind of like the DVLA taking your word for it that you can drive and handing you a licence."
Which is pretty much what they do. Ok, there is a test when you first start (normally in your late teens), but for the next seventy years you just declare that you can drive.
The FAA is a US publically funded organisation. As such, it is part of the "big government" which US voters dislike. Allowing private companies to succeed or fail in a free market is "the US way".
For US readers: how much are you willing for your taxes to rise to pay for more FAA?
The FAA was set up to fail by underfunding. Nobody is talking about that either.
What about the kind of asshole who says "We are supposed to regulate big business, but every time we try to, they ring up their tame Congressmen and Senators who shit all over us until we back down. We are trying to do our job properly - but Government tells us what to do on one hand, and interferes to keep us from doing it on the other."?
The history of the Thalidomide Scandal in the US may provide a parallel. The FDA came under pressure from the manufacturer to approve it, based on the manufacturer's data from trials. The physician responsible for reviewing the drug, Frances Kelsey, refused to approve it, asking for additional data about side-effects. The manufacturer was evasive (and may also have attempted to pressure the FDA to approve anyway). Kelsey was adamant, and shortly afterwards reports of birth defects stopped the drug from being used.
The difference between this and Boeing is that Boeing is American, and has key lobbyists in Government. Mere government employees don't stand a chance.
And another thing. You just had an election. How many of the Congressmembers and Senators who just got re-elected have lobbied for Boeing against the FAA in the past? How about the ones who lost?
Boeing gamed the system because they could. And your elected representatives will continue to help them because there are no consequences to it. Occasionally, this state of affairs will kill people.
Typical European misconception!
We have no issues with agencies like the FAA, OSHA, etc. These agencies keep people alive in industries that are inherently dangerous! Aviation, construction, etc. What we have a problem with is run-a-muck agencies like the EPA, Labor, etc that are always used as a political weapon by a particular Party in this country.
There is a matrix that defines just how much attention the FAA takes and how much they leave to the company DERs. If you are very experienced, have well defined processes and a good track record, then the FAA monitoring may be as little as reading two documents, the PSAC and the SAS. (For newbies it could be in the 20's of documents, through out the project.)
I think that Boeing will be moving to another part of the matrix after this, as internal processes have been borked by some method to allow this bad solution through, and so they are just not as trustworthy.
But they were strenuously trying to bury the problems by blaming the airlines, training, pilots, anything but themselves. If these crashes had happened in Europe or the US then the authorities would have been screaming much louder and much sooner. One would think that the view taken was that lives of non-Americans are worth less.........
This has uncovered all sorts of endemic cultural issues at Boeing and the FAA but I rather suspect that after some furious activity claiming that all is squeaky-clean, both parties will be back to the old ways again. It needs someone like the European regulator to stick two fingers up and require the Max and any subsequent revisions Boeing makes to any models be completely re-certified as new aircraft.
Both the FAA and Boeing and proved they cannot be trusted therefore can you actually trust any of the certifications?
What the worlds regulators need to do is ...
What the worlds regulators should have done is ...
There you go.
Q: Is it possible to imagine that the FAA and the rest of the world's regulating bodies did not know or at least suspect what was going on by just looking at the damn bird?
A: Nah!
It was Boeing, so they let it pass.
O.
"
The root of the issue was they they fitted the 737 airframe with new engines the cowls of which were basically too large and really mandated a complete redesign of the airframe.
"
Not sure whether a " complete redesign of the airframe" would have been necessary. Could probably have managed with a redesign of the undercarriage (& retraction mechanism) to give it more ground clearance.
ISTM that the type approval would be no more than needed for the engine re-positioning. Not sure whether pilots would have to be re-certified for a slightly longer undercarriage - but it would not require major retraining if all flight characteristics remained the same. In fact it would make a tail-strike less likely and so give greater latitude during takeoff rotation and landing flare.
>still would have needed a new type approval, and full pilot re-certification, which is what they were trying >to avoid.
It was the Airbus A321Neo that stole all their orders. The desperately needed an airframe with more efficient engines, more seats and had no time for a full FAA classification and costly pilot certification.
If it had worked, it would have been hailed as a "stroke of genius".
Apparently redesign of the undercarriage wasn't an option (for engineering reasons).
Another clear reason for not raising the height of the fuselage when on the ground is that airports would have needed new sets of steps for the passengers. Which would have cost them money. This money would have been extracted from the airlines, making the 737MAX less attractive as a purchase option. A key market for the MAX is budget airlines, who use steps rather than jetways as it's cheaper.
Remember that the point of these was that they were a drop-in replacement for existing 737s. In Europe the biggest budget airline, Ryanair had planned to mix them in with their existing fleet across all routes, rather than restricting them to certain airports. Making them more expensive to fly would make them less attractive to buy.
"Not sure whether a " complete redesign of the airframe" would have been necessary. Could probably have managed with a redesign of the undercarriage (& retraction mechanism) to give it more ground clearance."
You'd think that a longer undercart would allow the engines to be mounted in a more sensible location, however said longer undercart would almost certainly have mandated a redesign of the airframe. And we're back where we started.
Aren't most modern planes only flyable by wire?
Many modern aircraft are indeed fly by wire with a reversionary mode (particularly Boeing aircraft that were designed specifically for fly by wire systems).
Fly by wire systems are clearly flight safety critical and therefore get a great deal of scrutiny from a safety perspective at multiple levels (*). Boeing and Airbus (as well as Bombardier and Embrear) do not design or manufacture the fly by wire electronics or actuators; those are specified by the airframer to a suitable subcontractor as these are rather specialised designs.
The B777 (the original) is a fly by wire design and the electronics has a triplex architecture - 3 independent computers (and that means electrically isolated as well so a major defect in any one cannot bring the others down with it) along with 3 independent sets of sensors.
The problem with the 737MAX is that the aircraft has very little automation and MCAS was a bolt-on driven by certain rules (Part 25 stick force, primarily) but it was clearly never given the scrutiny that a fly by wire system should get.
* Boeing as an engineering company does not seem to exist any more, particularly since the McDonnell Douglas 'merger' (more like a reverse takeover) so in a very sad way it is not really surprising to see a total lack of ethics and scrutiny in the search for ever more profit regardless of the safety of the aircraft.
The fly by wire systems I am familiar with (which includes the B777) have very strict failure requirements, and those same failure requirements should have been applied to MCAS. Provided it can still control the flying surface then even a duplex system (single redundancy sensor and computers) cannot be considered safe unless it gives back complete control to the pilot in the event of a disagreement from the sensors and simply stays dormant in that situation.
The pilot who would need to be trained for such an event, clearly.
Not so. Airbus would have similar changes to their software to accommodate the same engine size change on the Neo, but Airbus is not subject to the same "stick-force" rule because Airbus has no force feedback to the pilots per the aerodynamic forces on the plane. MCAS is solely for stick-force regulations. The air frame is perfectly well suited to the engines. In a world that can see the GeeBee R1 fly, the 737 MAX is fine.
It was the airlines refusing to pay for training, gaming the system to save themselves money. The airlines had specified "no training" as basic requirement.
The 737 MAX center of thrust is more in line with the fuselage, producing lower pitch up force due to thrust. The engines are also farther forward, which is often an advantage to stability, but they are larger in diameter and interact with the wing a bit more, so they produce an increment of lift at high AoA, which a small increment of trim from MCAS was supposed to handle.
A software supremo means that software developers have someone to back them when the systems engineering is incomplete and a good one would have avoided this situation by determining what the systems architecture was designed to handle.
It's a pressurized tube with wings, a tail, engines, and landing gear. What way could being completely redesigned not add more potential problems?
Anyone ever see a rocket crash, or a self driving car?
https://www.cnbc.com/2015/07/20/musk-this-is-what-caused-the-spacex-launch-failure.html
“That’s an unfortunate thing because we could have saved Dragon [supply ship] if we had the right software there,”.
https://qz.com/1701070/spacex-missed-a-satellite-collision-warning/
"missed a potential collision warning because buggy software did not relay an alert"
https://text.npr.org/809207519
"allows its Autopilot system to be used on roadways that the software is not designed to handle, creating safety risks"
"...will lead a new, centralized organization of engineers who currently support the development and delivery of software embedded in Boeing's products and services"
That's an interesting move to take. It can end up being a double edged sword. Creating what amounts to an internal outsourcing organisation for your software engineering function, from which your other departments procure engineering services, can add a huge amount of operational friction and organisational politicking. The counter-argument is normally that the best people to tell experts how to best do their jobs are their fellow experts, which is generally fair, but if that new organisation is not sufficiently empowered to actually make meaningful decisions then they end up in an even weaker position - both unable to steer the project (because they have no corporate-level say) and unable to steer their own output (because they're 'just a supplier')
The fact that Jinnah is coming in with a job title of "only" VP indicates to external observers there is little interest in empowering him to Say No To Other Departments, and at best he'll be a peer of the other corporate project/department leaders. If they were really serious about making a centralised group of empowered, effective and world-class engineers we'd be seeing him rocking an SVP or GM title and heading up a genuine, dedicated subsidiary or global service line with a remit to overhaul organisational processes and culture from the root. Without that it's hard to conclude this is anything other than window dressing.
In any case the actual work will be outsourced to India, to people who don't know what the system is intended for (rocket, plane, brick, ...) and therefore won't be able to anticipate any glaring mistake in the requirements (like A requires B, B requires C, C requires D, but you can't have A and D at the same time - real story, took 2 weeks to the client to decide what to do).
......because the Boeing 787 is now ALSO getting more attention. The South Carolina plant has been shipping 787 planes out the factory door with factory ladders lying around in various inaccessible parts of the airframe!
*
And the FAA may also be reconsidering all the SELF-CERTIFICATION it as been allowing....where Boeing personnel have been signing off THEIR OWN WORK ON BEHALF OF..........THE FAA!
*
Even so, FAA spine or FAA no spine, I won't be flying in the new 737 Max (or whatever they call it today).....ever.
Does it really matter if the top-level execs are only focused on profitability and their bonuses and not on engineering the best product possible and making profit as a sideshow?
He'll get down-voted every time he tries to improve quality by hiring expensive American software engineers instead of cheap Indian bods. My guess is he'll quit in a couple of years because of all the obstruction.
I perceive the software problem to be a symptom, not the cause.
It's nice they want somebody to oversee software development (I guess tacitly admitting they didn't have anybody before), but really it would be better to cure the disease rather than whacking on a sticky plaster and thinking all is well.
The cause was the software issue combined with poorly trained pilots. Neither caused planes to crash on its own. Properly trained and experienced crews didn't panic, and solved the problem without crashing.
Boeing can only fix their software approval process. It appears their software engineers had insufficient clout to get their (necessary) way. Boeing have added a veep to go in to bat for the software side of things.
Ultimately, new plane designs crash, although less frequently than they used to, because we're still discovering new failure modes.
Ultimately, the root cause of the problem was terrible management overriding engineering which yielded, quite predictably, a terribly engineered aircraft.
A computer that has control of a moveable flying surface without pilot intervention that relies on a single sensor known for its unreliability and that is using that sensor information only and that was not hardened against single event upsets.
Really - what about true air speed, altitude above ground, rate of climb and dive (independent of the AoA sensor) which could all feed into this but even then with everything relying on a single computer the design is provably unsafe.
At the time of design or entry into service, the angle of attach sensor wasn't a known issue.
What no one has asked or disclosed is who is making the sensors? Name and shame.
When I worked in aircraft parts manufacturing, I/we tracked defects. If I saw more than 2 or 3 parts or operations with the same defect, I was out looking for the root cause.
"What no one has asked or disclosed is who is making the [AOA] sensors? Name and shame."
Angle Of Attack sensors are reliable, up to the point that they suffer a bird strike or other FOD. Which is why all A/C have two or three of them. Only using one of them as a basis for actuating MCAS, or not adding a third sensor, is another discussion altogether.
So why were the very properly trained American Airlines pilots union so very very pissed off with Boeing when they got a chance to sit down with them? Because they knew it was a BS move.
https://www.theguardian.com/business/2019/may/23/boeing-737-max-crashes-american-airlines-pilots-union-mcas
The US pilots are on record, anonymously, as well as through direct reports, of complaining vigorously about the plane. It took just the right sequence of conditions to cause the crash. The pilots were trained to handle the 737, not an elephant that suddenly starts bucking like a horse.
That in itself is not an issue. You have to have independence in the checks and balances, and outsourcing part of the work is a perfectly acceptable way of achieving it.
e.g. the people that write the specs, are not the people who review them, or who approve them.
the tests specifications are written from the system requirements by a different team that wrote the software specifications. etc.
That outsourcing may be to a separate part of the same company or a separate company, so long as the reviewing and approval is independent and by suitably qualified and experienced people it doesn't really mater who is doing the work, internal or external.
Boeing may have poor software quality but the software involved in the Boeing 737 Max crashes did what it was specified to do. The problems were in the overall system design, safety analysis and change control with changes that had a large impact on safety being made without changing the safety analysis.
The failures were not the result of subtle difficult to predict problems but issues that should have been identified and addressed. Given this it would not be suprising if Boeing had issues in their software development process as there seemed to be systemic problems with respect to their safety engineering processes but software failures were not implicated in the crashes.
No sir,
The problem was that the software was relying on flawed information from a defective AoA, the pilots not being aware of the presence of this particular software feature in the first place, and therefore not thinking of the option to just switch it off (button was provided, yet not documented anywhere).
Software engineers do NOT "write code to spec". Software engineers "engage with the stakeholders" until the spec is not s***. I don't need ANY training in aerospace engineering to understand the problems with code that relies on a single input. If I were writing software for airplanes, I would have more than just a little bit of knowledge of aerospace engineering, and in a hurry.
One of the critical failure points was when whomever was writing the software accepted that s*** spec.
If it's Boeing I ain't going this will not resolve nor address. Fundamentally the loading area Centre of Gravity of the wings and engines extended forward shift the pivot point of the plane. Software can not fix the laws of aerodynamics Boeing has violated with the redesign.
Boeing tried to ignore:
1) The rules of aerodynamics / gravity
2) Software cannot fix hardware faults
3) Not having an honest process and going after the money
The first point is not resolvable on the 737DEAD^ahem MAX in any form
The second point is a costly immutable fact, ask Airbus
The last point is whomever is foolish enough to take this role is a likely sacrificial lamb
Fundamentally the loading area Centre of Gravity of the wings and engines extended forward shift the pivot point of the plane.
Loading area of the wings? You mean wing loading, which is how much pressure per surface area of the wing. As in, 10 tonnes spread over 10 square meters = 1 ton per square meter. Give it 20 square meters and you have half the wing loading. Can you relate this to the topic?
Centre of gravity...of the wings? Do you mean Centre of Lift, as in the balance point of the upward forces in flight? Compared with the Centre of Gravity (mass)? I promise you, the Centre of Lift is quite close to the Centre of Gravity of this plane in cruise, give or take some "trim." As it is with all planes that fly.
I guess you wanted to say something about Centre of Thrust, combined with moment arms, and then bring in auto-trim logic, tail-strike protection, thrust levels, and such. There's a lot of great information available if you want to learn. (FLCH isn't just a dirty word.)
Software can not fix the laws of aerodynamics Boeing has violated with the redesign....Boeing tried to ignore: 1) The rules of aerodynamics / gravity
Can you tell us which "law of aerodynamics" Boeing violated?
2) Software cannot fix hardware faults
It cannot fix faults, but control logic can assist with them. Look at how cars manage fuel mixture and spark timing advance. Or how anti-lock brakes work.
3) Not having an honest process and going after the money....The second point is a costly immutable fact, ask Airbus
Yeah, Boeing really had no shame. How does Airbus come in to this?
"veteran of similar gigs at SpaceX, Tesla, and Google."
Oh, and also Aurora. So either this guy has a magic touch, and quality instantly improves, or he's a typical C level ladder climber who hops around every four years?
Establishing a culture of quality or safety takes a long time. Establishing it when there are entrenched bad habits at a massive company like Boeing takes even longer.
The hiring is a good first step, but there's much more work to do.
He hasn’t worked at a company like Boeing. His challenges will be different. I doubt heading Boeing software is a step up. His compensation package can’t be even close to a typical startup going IPO.
I think he genuinely liked the challenge. Turning around a large organization requires a different set of skills.
I do really wish him success.
He has experience only with startups and very dynamic companies. I really wish him luck. He is going to be running into entrenched Boeing managers.
He will be running into $7/hr outsourcing to cyient. I have hired engineers out of India and you can’t even get a new college graduate working for those wages.
Why do I wish him luck?
I had to turn around an engineering team of 220 engineers who wrote code that met the requirements but no one could figure out how it worked. When it worked. The politics was tough. Good training for me.
It took me 2 years to get them to write good code. It was my toughest job ever.
Fast forward to a several billion dollar company. Pitched quality battles from 5AM to 8 PM. Multiple entrenched groups, cultures, sensitivities, regions. Just managers doing everything possible not to meet the spirit of software quality as their KPIs were tied to shipping their products on time.
I inherited the mantle from my VP who got fired for doing the right thing. I left roughly after a year. I can claim some success but I was a burnt out shell for a year.
I know a lot of big companies whose accountants found smart to externalise to the max everything related to IT. The consequence is those companies lost all of the embedded knowledge, and relied on underpaid and overloaded staff of technicians and engineers leased by meat-sellers like Cap Gemini, HCL and al. And now they wonder why IT works badly, with security and privacy breaches, bad software, people angry and/or desperate about IT support...
Is Boeing one of those?
Language: The Germans, French, and Italians have been working with us for decades. The Germans and Italians since 1945, and the French in fits :) We and the English speak a different language but we have figured out how to communicate sometimes.
This is just hiring incompetent people (will not call them engineers) to write software for $7/hr. There are good engineers in India and here, some even run large tech companies. But you get what you pay for. Currently, a decent DBA costs $120K or so. Fully loaded costs (the company's costs) would be roughly $150K. Yeah, the good ones know their value. I still can't figure out how Cyient gets away with it.
Yeah, the guy now has to figure out if he has any power to make changes. Does he have the power to block a release? Does he have to power to force a group to adopt good software engineering practices? Will he even be invited to participate? Will the entrenched managers work around this guy and ship out software around his back?
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
