back to article Boeing's Starliner capsule corroded due to high humidity levels, NASA explains, and the spaceship won't fly this year

Boeing’s CST-100 Starliner capsule, designed to carry astronauts to and from the International Space Station, will not fly until the first half of next year at the earliest, as the manufacturing giant continues to tackle an issue with the spacecraft’s valves. Things have not gone smoothly for Boeing. Its Starliner program has …

  1. storner
    Trollface

    Easy-peasy

    They'll install AI-controlled sledgehammers on each valve, rebrand the capsule as StarlinerMAX - and ... LIFT-OFF!

    1. sanmigueelbeer
      Joke

      Re: Easy-peasy

      rebrand the capsule as StarlinerMAX - and ... LIFT-OFF KA-BOOOM!

      TFTFY

  2. Jan 0 Silver badge

    Is Boeing's valve test facility located in the Atacama Desert perchance?

    1. Anonymous Coward
      Anonymous Coward

      It's a probably a case along the lines "young Boeing engineers don't know where Florida is, and what kind of climate it has".

      1. Anonymous Coward
        Anonymous Coward

        probably a case along the lines ...

        Or perhaps it just hung around in Florida way longer than was expected, due to all the other delays..?

        1. Anonymous Coward
          Anonymous Coward

          Re: probably a case along the lines ...

          There's a reason why Grumman was known to be able to build sturdy planes that could work on carriers at sea for a long time... maybe the fact that Boeing no longer had a plane on a carrier since the 1920s meant they forgot what happens when you choose the wrong materials in such situations?

          1. Our Lord and Savior Rahl

            Re: probably a case along the lines ...

            Er... The Hornet is Boeing. That flies on carriers all the time.

            OK it was originally a McDonnell-Douglas project but still, being anal about facts is important.

            1. Anonymous Coward
              Anonymous Coward

              "OK it was originally a McDonnell-Douglas project"

              Exactly. It wasn't designed nor built by Boeing. The Phantom II wasn't Grumman too and was McDonnell-Douglas, a company which had many carrier planes as well.

        2. NoneSuch Silver badge
          Go

          Re: probably a case along the lines ...

          Space flight is hard.

          Do remember, SpaceX had it's own repeated teething troubles before they became NASA's darling.

          In computer terms:

          SpaceX is Intel

          Boeing is AMD (OK, maybe ARM?)

          1. Lennart Sorensen

            Re: probably a case along the lines ...

            How is spacex intel? spacex at least tries to innovate and compete. intel looks a lot more like the old dominant players in the business happy to keep selling people more of the same until someone comes along and starts to eat their business (like AMD sometimes does). And intel has made plenty of flops in terms of design (pentium 4, iAPX 432, itanium, etc, not to mention failing to get new process nodes to work while their competitors catch up). intel reminds me of boeing these days. They think they know everything better than everyone else and then things go wrong, but they manage to fix it eventually and claw their way back.

            So no spacex is not like intel.

          2. phuzz Silver badge
            Thumb Down

            Re: probably a case along the lines ...

            Boeing being like IBM would be a better metaphor.

            Been around forever, used to making very expensive products for governments and massive enterprises, and these days, well past their best.

            Also, there's the now (very) out-of-date phrases: "None ever get fired for buying IBM", and "If it ain't Boeing, I ain't going!".

        3. Anonymous Coward
          FAIL

          Re: probably a case along the lines ...

          No. It's just that Boeing engineers lived in airconditioned apartments and commuted in airconditioned cars to airconditioned offices. Only mechanics had to work in the heat and humidity (and when would an engineer take advice from a mechanic).

  3. Paul Herber Silver badge

    an issue with the spacecraft’s valves

    I've got a few 12AU7s in a box somewhere. It really is about time they started looking at TTL though. CMOS might be better in space though.

    1. circusmole

      Re: an issue with the spacecraft’s valves

      ...but if they want to stick with valves with proven reliability and performance and some grunt I have 4 KT88's that I never got around to using if they want to give me a call? All that germanium stuff will never catch on.

      1. Dagg Silver badge

        Re: an issue with the spacecraft’s valves

        The big problem with KT88's is not the valves its the bloody transformers you need to use.

        I used to build and maintain AF and RF gear using valves. Used some KT88's to make a uni student radio station in NZ in the 70s.

        Put my back out moving it around.

        1. bombastic bob Silver badge
          Devil

          Re: an issue with the spacecraft’s valves

          fortunately I found a great place online to buy transformers (replacements for guitar amps anyway) for tube(valve) amplifiers, like 6L6 and KT88 (and other) power stages, even Hammond brand (but mostly amp replacements).

          And I bet those transfomers won't corrode in Florida summertime weather, unlike Boeing's rocket fuel valves.

          I spent the summer in Orlando back in the early 80's when I went to U.S. Navy Nuclear Power School there. Hot, humid, and gnats spiraling in clouds. And you could set your watch by the afternoon thunderstorms.

          [Launches take place from Canaveral because it's one of the lowest latitude spots in the continental USA, and so earth spin gives you a slight boost on takeoff, saving fuel. So if you make a rocket it BETTER be Florida weather proof]

          When I was in the Navy I was so used to electrical, electronic, and mechanical/pipe systems being weatherproof and salt-water tolerant that I'm kind of amazed that Starliner's engineers missed this obvious operational condition.

      2. Anonymous Coward
        Anonymous Coward

        Re: an issue with the spacecraft’s valves

        I remember my first visit to an old timer's radio "shack". He didn't need a room heater - the 813's filament was very bright.

        However - nothing compared to a BBC TV transmitter room on a rare "open day". A long queue in the middle of nowhere - with most of the public apparently expecting to see "Meg". The technicians were delighted when people were actually interested in the equipment for itself.

    2. Yet Another Anonymous coward Silver badge

      Re: an issue with the spacecraft’s valves

      >CMOS might be better in space though.

      I think Boeing were going for that authentic Apollo era 60s sound

      1. Paul Herber Silver badge

        Re: an issue with the spacecraft’s valves

        Nobody can hear your authentic 60s valve sound in spaaaaace.

        1. Yet Another Anonymous coward Silver badge

          Re: an issue with the spacecraft’s valves

          > in spaaaaace.

          Well that's where Boeing's secret plan comes in

      2. EarthDog

        Re: an issue with the spacecraft’s valves

        I don't recall rust as a problem in the 60s. There were other problems, but somehow humidity was not one of them.

    3. chrisw67

      Re: an issue with the spacecraft’s valves

      Going straight to transistors might be a step-too-far. Perhaps try Nuvistors (https://en.wikipedia.org/wiki/Nuvistor) first?

      1. bombastic bob Silver badge
        Devil

        Re: an issue with the spacecraft’s valves

        I remember seeing one of those Nuvistor tubes/valves in a color TV tuner (a TV from the early 70's).

        (and it did not rust in hot humid weather, lasting over 15 years until I sold it at a swap meet in the mid 80's)

  4. Andrew Scaife

    Boeing, Boeing...

    Bong. "Boeing Starliner!"*

    *May require lubrication before reuse. Fnar, fnar...

    1. spireite Silver badge

      Re: Boeing, Boeing...

      Presumably, trying to open sticky valves required some vibration to help loosen them up as they were self-lubricating, fnar, fnar...

  5. Binraider Silver badge

    Florida? Humid? Who would have known. An average dew point of about 17deg C.

  6. PerlyKing
    Facepalm

    Remind me...

    Wasn't Boeing selected at least in part due to its long history of building launch vehicles? Many of which launched from Florida?

    Double facepalm - when one facepalm is just not enough.

    1. Greybearded old scrote Silver badge

      Re: Remind me...

      I had a work mate who kept a shop dummy hand on his desk, because sometimes two won't do the job.

  7. ThatOne Silver badge
    Facepalm

    Cheap & cheerful, but mostly cheap

    > the chemicals from the spacecraft’s oxidizer reacted with water condensation inside the valves to form nitric acid. The acidity corroded the valves, causing them to stick.

    Yes, that happens when you don't bother too much about specifications and use the cheapest materials you can find...

    One thing makes me wonder though: Florida's humid air is outside, and the spacecraft's oxidizer is (hopefully) inside the piping. So how do those two manage to interact? I'm no rocket scientist, but it seems to me those two should have pretty few occasions to meet, definitely not enough to block half of the engine's valves (which in itself is quite an achievement).

    1. John Robson Silver badge
      Boffin

      Re: Cheap & cheerful, but mostly cheap

      What's in the pipes *before* the tanks are filled with fuel?

      1. Wellyboot Silver badge
        Facepalm

        Re: Cheap & cheerful, but mostly cheap

        I would hope they'd be purged with a non reactive gas before adding the fuel.

        Boeing - relearning chemistry that Wernher knew about in 1944.

        1. Spoobistle
          Joke

          Re: relearning chemistry

          I wondered why Boeing might be interested in hypergolic propellants, then I realised some are zero carbon. So I guess this is just a learning curve en route to the "747-Green"!

          Dammit, why can't I have the bomb icon and the hazmat icon and the joke icon all at once?

          1. Yet Another Anonymous coward Silver badge

            Re: relearning chemistry

            >hypergolic propellants, then I realised some are zero carbon

            Hypergolic fuels - environmentally friendly is the first thing I think of

            I don't think that's what Organic Chemistry means

        2. John Robson Silver badge

          Re: Cheap & cheerful, but mostly cheap

          You would hope so... dry nitrogen isn't that hard to aquire...

      2. wub

        Re: Cheap & cheerful, but mostly cheap

        I'm no expert, but if it were me, I'd purge the tanks and all the lines with an excess of dry nitrogen. Not terribly reactive, cheap, and it would get all those pesky minor contaminants out of the way.

    2. Vulch

      Re: Cheap & cheerful, but mostly cheap

      Leakage of the hypergolic propellants out of the valves is a known problem on all spacecraft using them. The solution is to put each valve in a box that is vented to the outside so the leakage can evaporate away in vacuum. On the ground before launch the vent is usually covered by a "remove before flight" tag or a tissue paper seal that ruptures. Why this was not done or didn't work in this case is an execise left for the investigators.

    3. Steve K

      Re: Cheap & cheerful, but mostly cheap

      I imagine they do test firings also before shipping the capsules, so could be residues from that?

    4. Chris G

      Re: Cheap & cheerful, but mostly cheap

      Nothing cheap materials wise with Boeing and so far not much to be cheery about.

      Ever since Boeing and SpaceX were award contracts, Boeing has been receiving nearly double the money and wanted to up the cost per seat, something that in 2019 was assessed bu NASA to already be 60% higher than SpaceX.

      When you consider what the Starliner has cost so far without delivering and comparing its performance with the new boy, one wonders why the contract is still going?

      I suspect the army of Boeing lawyers waiting in the wings is the main reason.

      As things are at the current rate of success, Bezos will be delivering before Boeing.

      1. Anonymous Coward
        Anonymous Coward

        Re: Cheap & cheerful, but mostly cheap

        Yeah, you may be right about Bezos, but that's kinda the point. NASA is willing to bleed red ink to keep things from becoming a one horse race. Part of why they are paying Boeing, Blue, and the others to try to develop a non-monopolistic space industry. Boeing also spreads a lot of cheddar around Washington, which also pays for their military and airplane subsidies.

        Boeing going to end up a distant third the way things are going, but NASA may still believe the price is worth it to have a choice other than a space pirate and the whims of a post-silicon valley Willy Wonka.

      2. ThatOne Silver badge
        Coffee/keyboard

        Re: Cheap & cheerful, but mostly cheap

        > Nothing cheap materials wise with Boeing

        Are you sure? Billing lots of $$$ doesn't necessarily mean that you used expensive materials, it can mean you have high lobbying costs.

    5. Gene Cash Silver badge

      Re: Cheap & cheerful, but mostly cheap

      Unfortunately the nitrogen tetroxide was not completely inside the pipes.

      The issue is that a tiny bit leaked past the valves and met the humidity in the outside air, and the resulting highly active acids made things a lot worse.

      1. I ain't Spartacus Gold badge
        Flame

        Re: Cheap & cheerful, but mostly cheap

        The more I read about space propellants, the more I think, "those are nice chemicals. I'd really like to work with them."

        1. timrowledge

          Re: Cheap & cheerful, but mostly cheap

          Obligatory mention of Derek Lowe’s “things I won’t work with” articles. See https://www.science.org/action/doSearch?AllField=Things+I+won’t+work+with+

          Do not read whilst near a keyboard if you have any liquids in your mouth.

          1. Gordon 10

            Re: Cheap & cheerful, but mostly cheap

            Also obligatory mention of Charlie Stross’s A tall tail.(Rocket Fuel themed SF)

            https://www.tor.com/2012/07/20/a-tall-tail/

      2. bombastic bob Silver badge
        Boffin

        Re: Cheap & cheerful, but mostly cheap

        maybe purging them with ammonia gas would neutralize the acidity ?? Metals in a slightly basic pH tend to form protective oxide layers.

        OK that would be too simple and obvious. Just vent some ammonia into the thruster exhaust ports to keep the pH under control and voila! Problem solved! So what if it smells like glass cleaner.

  8. Richard Pennington 1
    Facepalm

    Getting hot and sticky ...

    NASA has been launching out of Florida for years. The Europeans have been launching Arianes out of a tropical rain forest at Kourou for years. The weather issues have surely been known (apparently, to everyone except Boeing) for years.

    1. Yet Another Anonymous coward Silver badge

      Re: Getting hot and sticky ...

      Ariane no longer uses oh shit hypergolic fuel - the 1st and 2nd stage are cryo O2+H2

  9. seven of five

    If it is a Boeing, it ain't going

    Not even sure whether this is good or better.

  10. Mike 137 Silver badge

    Bad choice of materials?

    "chemicals from the spacecraft’s oxidizer reacted with water condensation inside the valves to form nitric acid. The acidity corroded the valves"

    This should have been anticipated at the very first design stage. Commercial chemical works address such issues (and worse) all the time successfully.

    One has to wonder whether this was an outcome of outsourced component design/production. The Dreamliner had component design outsource problems, as did a notable (non-Boeing) aircraft carrier (to the extent in the latter case that some pipework had to be hammered into alignment).

    Central design control is essential where technologies are this complex, as local interpretation of requirements can diverge from central expectations. But of course it's more expensive than fire and forget outsourcing.

    1. Anonymous Coward
      Anonymous Coward

      Re: Bad choice of materials?

      Bears crapping in the wood ?

    2. Flocke Kroes Silver badge

      Re: Antici... pation

      This problem was anticipated. Boeing came up with procedures to ensure the valve is dry before loading oxidizer and remains dry until launch. Clearly there is a hole in those procedures but so far no-one has found it.

      1. Dagg Silver badge
        Facepalm

        Re: Antici... pation

        Probably some pointy haired project manager determined that it added no value and so could removed to same time and money.

  11. ecofeco Silver badge

    If it's Boeing...

    ...it ain't going.

  12. Anonymous Coward
    Anonymous Coward

    *LOL*

    I love to see a MIC contractor fall flat on their face. Good on 'em, especially after their abusive lawsuits against Canadian Bombardier in the past for beating them fair and square on pricing.

    For the past few years, Boeing is batting 0/10. *LMAO*

    1. Yet Another Anonymous coward Silver badge

      So all they have to do is get a 300% tariff put on SpaceX and they are set.

      Fortunately congressmen are much cheaper than rocket scientists and you only need to pay one of them.

    2. Anonymous Coward
      Anonymous Coward

      Is that why bomb canceled its firefighter/seaplane line?

      We have been missing them this fire season. Which is to say we have almost every one that was ever built in the sky right now, and we could use more of them. The same thing will probably happen next year as well but it seems the assembly lines for the Short brothers designs all got shut down.

      Be totally happy to be wrong about that, as we are paying for a DC-10 and a bunch of war surplus helicopters ATM that aren't as effective as the ol Superscoopers, while also being much more expensive.

      1. Yet Another Anonymous coward Silver badge

        Re: Is that why bomb canceled its firefighter/seaplane line?

        Don't know what happened to them re Covid - but a company in Vancouver took over the good bits of Bombardier.

        They were going to build an improved version of the waterbomber, and the Dash8 turboprop commuter

      2. ThatOne Silver badge

        Re: Is that why bomb canceled its firefighter/seaplane line?

        > aren't as effective [...], while also being much more expensive

        That's progress, get used to it.

  13. Adair Silver badge

    Flights cancelled due to...

    the wrong kind of humidity.

    That's actually worse than trains being cancelled due to the 'wrong kind of snow'. Boeing are looking like an outfit that have lost their way, or maybe found a different way, but whatever it is it sure doesn't encourage confidence when it comes to trusting lives to their care.

  14. steelpillow Silver badge
    Facepalm

    Controlled stupidity

    Controlled humidity in a spacecraft final assembly and storage facility? Whoever could have anticipated the need for that? Surely not as highly experienced an aerospace supplier as Boeing.

    They seem less and less like the world's biggest aerospace company and more and more like the world's biggest landslide of yesterday's beancounters. The answer is blindingly obvious - employ more lawyers.

    1. Wellyboot Silver badge
      Devil

      Re: Controlled stupidity

      YES!

      Onboard every test flight verifying it's working to the contracted* requirement.

      *Not necessarily a flyable spec.

      1. Giles C Silver badge

        Re: Controlled stupidity

        Reminds me of ISO 9001

        It doesn’t have to work as long as you have followed the documented procedures.

        Or

        If you want to test a box of fragile electronics by kicking then off the roof of a building, make out an area which is cordoned off for the landing , design a standard test model (size 10 doc martins) and it will pass.

        The fact it results in a pile of scrap and can’t function is irrelevant- you have followed the correct process.

        1. Boris the Cockroach Silver badge

          Re: Controlled stupidity

          HEY! where'd you get a copy of our top secret test procedures and ISO 9001 documentation?

          Although to be fair its more like generating a paper trail

          P.O. issued :tick

          Drawing/technical info issued :tick

          Cell #6 built up by setter #3 :tick

          first part inspected :tick

          In process check :....................

          Yupp blame the min wage minion who never signed off his bit

          1. Yet Another Anonymous coward Silver badge

            Re: Controlled stupidity

            I once spent a whole weekend practicing dropping a piece of equipment from desk height in a way that didn't break it - so it could pass a required CE test on Monday

            1. Anonymous Coward
              Anonymous Coward

              Re: Controlled stupidity

              My favorite was the tests run at my first job, for a different project. They were using the ancient Sperry AN/YUK-502 hardware (which doesn't even have a *stack*, but it has "radiation hard" iron core memory, so... But I digress.)

              The hard drive for the beasts were encased in massive safes with shock absorber springs capable of taking a stream of .50 caliber shells and continuing to function (I presume they never tried shooting any of the connecting cables.)

              These things would be dropped 50 feet to simulate waves on the oceans. The resulting BOOM would make the whole complex shake...

              BTW, they were RUNNING while being dropped, and had to function without failure despite the impact! They may have been incredibly SMALL hard drives, capacity-wise, but they were TOUGH!

              The monitors could also take .50 caliber shells, so they were fair game for as much frustrated slapping as you felt like unleashing on them. They didn't care. *LOL*

              1. Yet Another Anonymous coward Silver badge

                Re: Controlled stupidity

                capable of taking a stream of .50 caliber shells and continuing to function

                That's the tricky bit, building kit to take incoming .50 cal is easy enough

    2. hoola Silver badge

      Re: Controlled stupidity

      Put like that you could easily believe they are putting the thing together in any old warehouse using off-the-shelf plumbing parts.

      Just give it a polish, maybe a coat of rattle can paint and add about 1000% mark-up and job done; space-certified parts.

  15. Teejay

    Where is Feynman?

    This is Challenger all over. Sure, nobody died. But with Challenger, the insulation rings got inflexible in the cold. In this case, more than half (!) of the valves had corroded. Although they are trying to spin it as 'nothing happened', this shows bad testing and bad risk analysis.

    1. Yet Another Anonymous coward Silver badge

      Re: Where is Feynman?

      But the many of the valves didn't fail - so it's worth launching anyway.

      You just have to change the requirements to 50%(+/-5%) operational valves.

  16. very angry man

    ok so you cant get it wet!

    what happens if you fuel it after midnight?

    1. John Brown (no body) Silver badge

      Re: ok so you cant get it wet!

      Well, it would increase production somewhat. Although not necessarily in a good way.

  17. Potemkine! Silver badge

    I guess there are plenty of brilliant engineers in Boeing. We know that the MAX failure was caused by management who wanted to maintain costs to the lower level and refused to listen to the technical ones. But here, what is the reason for such a huge and obvious failure? What is so wrong at Boeing?

    1. ThatOne Silver badge

      > I guess there are plenty of brilliant engineers in Boeing

      Are or were? I know that if I was a brilliant engineer working there, I would leave ASAP, first because I apparently can't really work there at my full potential, second because a longer stint at Boeing in my resume would be a career-destroying stain, stigmatizing me as "most likely incompetent".

  18. Sixtiesplastictrektableware

    Just keep swinging?

    At least they caught this thing before any bodies were on the line.

    As long as they don't have to completely disassemble it to repair and can assure safety then they're just late to the space party. They pull that off and that undoes some damage to their rep. Easier said than done, though.

    If Bezos and his dickship can actually break orbit, maybe... he might... I dunno what that plan is, or Toothbeard for that matter.

    I was trying to chin up and cheer for breaking up the space-opoly, but it unraveled quickly what with only SpaceMusk doing anything useful up there.

    Better optimism next time, I guess. Pretty much like Boeing, at this point.

    Now, if they can build something that flies on optimism...

  19. Binraider Silver badge

    NASA are clearly very unhappy, as they have just opened up another prelim request for proposals. Dream chaser hopefully will be all over it

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like