Falcon 9 fireworks display grounds SpaceX The SpaceX launch schedule has been knocked back by a couple of months as a result of the loss of its Falcon 9 CRS-7 mission rocket on 28 June, it has revealed. The company's president, Gwynne Shotwell, said of the pause: "We're taking more time than we originally envisioned to get back to flight. But I don't think any of our …
How do they manage to analyse Rocket failures?
I always imagine that if they do actually manage to find the pieces, that many of them will be almost unrecognizable, they must then be pieced back together and then hopefully they can start to debug the causes.
Or are there alternative methods which they can use ? Sometimes it appears as though they find the reasons PDQ...
I haven't worked in this field, but I would imagine that they have a huge amount of sensor data streamed back right up until the instant of destruction which should help to indicate where things have started to go wrong. But yes, it does seem impressive how quickly they can come up with a comprehensive statement on what happened and how they hope to prevent it happening again.
They figured this one out from telemetry. There was an over pressurisation event in the second stage tank, which had counter intuitive causes. They used analysis (ie timing which sensor fired when) to find out where the pressure spike started (IIRC), which indicated a strut failure around the helium tank (COPV). The strut however were rated for 3x the expected load so should not have failed, so they then tested a LOT of struts (1000+) and found a couple that broke well below the certified load. Further analysis of the bolts (I think) showed bad grain structure inside the bolts which caused the failure.
In addition to other answers, accident forensics is an extremely interesting discipline of science, art, deduction, and thorough knowledge of physics in context. Fire investigators, collision investigators, and murder investigators have a lot in common - sometimes in the same incident!
I'll just make up an example. In the wreckage a particular piece of metal is found, which is bent and twisted in a certain way. By analyzing how it is bent and twisted a fairly good idea of which way the other components were affecting it can be determined. This can be confirmed using a Finite Element Analysis and simulation, played in slow motion to confirm. Minor scuffs or paint chips may be indicators of impact by another object, which leads to the question of what other object _could_ impact it in the situation, and where that object might be now. If there are burn marks, soot, etc., then from things like which parts of the twisted object's surfaces have soot on them it may be possible to determine whether the burn came before or after the twisting, and which direction the flame came from. Also, from the 'stretch marks' in the bent portion, one can notice whether the cracks have soot in them. If not, then the bending happened afterwards. A classic question in the event of an explosion is whether the metal is bent in (meaning an external impact) or out (meaning an internal explosion), or both (a missile impacted punching a hole, then exploded.)
NTSB examiners have many times basically re-assembled an entire airliner on a frame in a hangar, hanging as many of the pieces as possible on the frame in their original locations, to help determine what happened. I think that was done in the case of TWA flight 800, when they determined that an explosion inside one of the fuel tanks was the cause, leading them to the short in the wiring that went through the tank.
http://www.spacex.com/news/2015/07/20/crs-7-investigation-update
In summary:
3,000 telemetry channels.
Video (internal to the vehicle and external) and debris analysis.
"Detailed close-out photos of stage construction"
Or basically the same press release with added expert opinion http://www.nasaspaceflight.com/2015/07/spacex-falcon-9-failure-investigation-focuses-update/
IIRC the vibration sensors gave them a really good clue to where/what it was.
They show where the first "impact" was, and from the obvious direction (floating in the tank or direction of travel of the rocket, I forget which), could trace it to the separated tank and the position of one of it's struts.
In addition to pressure sensors and emergency sensors that helped set the abort warning.
ISTR reading somewhere that logs of acoustic sensors on the rocket were used to pinpoint what was going on. I believe they have thing like strain gauges here and there on the ships too, and they send continuous telemetry AFAIK. They certainly have a lot more than bits of wreckage to look at (which are often absent if the thing falls into the sea, of course).
Essentially you do a fault tree analysis of everything that could fail, then identify the symptom patter (or syndrome) of that fault and log it.
That way when something happens (and it's an ELV, something will happen) you can work back to see what patterns showed up in the telemetry.
Keep in mind that at 1 sample per second for all channels over the 139 secs is a bit shy of 420 KB, although some of them will be single bit switch reporting. A high frequency sensor reading like a microphone or accelerometer could be 20 000x that.
I think the supplier of those struts is extremely fortunate that SX did not name them, given some of the remaining struts in store showed 1/5 their design strength.
While you may wish that was the case, reality is far different. You design for failure modes; you verify critical cases, do verification fm with instrumentation in qual and flight test; and eliminate as much instrumentation as possible for weight and cost. The closest you come to your imagination, is a top level failure modes and effects analysis, with a ranking items that cause loss of the vehicle or life (ground or on orbit if they get that far.). There is hardly a detailed fault tree. At the top they have some bright people to connect times and observations but this is too quick and too simple to reflect a detailed fault tree analysis.
And the designer of the strut could only design to the load cases supplied by Space X. Space X is responsible for doing a design review. I don't understand why so many fanboys believe all the PR and give Space X a pass on their responsibilities? Space flight is tough and when you cut corners and reviews this is what happens. Maybe this was a new lesson learned or maybe the bright new engineers missed some basics from past lessons learned.
I see you've just joined the site. You certainly have an interesting posting style. Let's look at it.
"And the designer of the strut could only design to the load cases supplied by Space X. Space X is responsible for doing a design review."
Nice implication that SX did not. Although I doubt you have any more knowledge of what they did than I do.
" I don't understand why so many fanboys believe all the PR and give Space X a pass on their responsibilities?"
Another nice implication that SX don't accept them. NASA got a completely new LV out of them for about $200m. $15Bn later the SLS has still managed to put nothing in orbit.
" Space flight is tough and when you cut corners and reviews this is what happens."
Much the same line from ULA or an SLS contractor. And again note the implication that SX don't do such reviews.
" Maybe this was a new lesson learned or maybe the bright new engineers missed some basics from past lessons learned."
Or maybe the low bidder on the parts contract really did have s**t QC?
When an inventory part tests out 5x lower than it's design load that's a mfg fault, not a design fault.
I'm not an SX fanboi, but I do respect their engineering skills.
And I don't like PR goons astroturfing.
Rocket designers face constraints not present for terrestrial designers. Specifically in this case it's the mass allowance. You just can't have everything engineered to 10x load rating, because that drives the launch weight up, and everything else gets scaled up much more to cover it. Using 3x engineering means a much less expensive operation all around.
The sub-contactor aimed for a certain specific load rating, and well, their QA appears to suck.
The sub-contactor aimed for a certain specific load rating, and well, their QA appears to suck
They probably aimed for the load rating SpaceX told them to aim for. Since the subcontractor was probably uninvolved in designing the rest of the rocket, they'd be unlikely to design the strut or model its required strength. I mean, it'd be very awkward engineering if the hundreds of struts used in each Falcon 9 were designed by someone outside of SpaceX - that's just an engineering nightmare.
So, I'd be willing to bet the sub-contractor was just a machine shop that carved the stainless, and maybe heat treated and finished it. It wouldn't be unusual for an aerospace company to tell a machine shop, "And perform lot sampling to verify tensile strength," which means randomly selecting a certain number (depending on lot size), or using witness coupons, and pulling them in a tensile test rig. The subcon would thus know what strength to aim for and would handle testing.
But the design and final QA is on the aerospace company. The subcon borked up, yes, but the buck stopped with SpaceX.
I do note a comment in the SpaceX update: "...SpaceX will no longer use these particular struts for flight applications." That might mean they ditched the subcon, or redesigned the strut, or both. Unlike some aerospace materials fields, working stainless to aerospace specs has a number of competent suppliers.
"Why are you putting redundantly supplied 5500 psi helium tanks inside another tank that will fail if any of them fails"
There are several reasons:
First, you reduce the number of penetrations, seals, and weldments between the helium bottle and propellant tank. Those propellant tanks are pressurized, too, and thin walled. In a large diameter, thin-walled structure like the shuttle's external tank, the pressurization (up to 35psi) could generate tens of thousands of pounds of hoop stress in the tank skin. Poking unnecessary holes through that taut balloon and welding in a line is not preferable. Welds never have the same properties as the base metals, especially when you're dealing with crap like lithium-aluminum alloys.
Second, it's simpler. The helium tank vents directly into the tank.
Third, the chilled propellant will chill the helium and allow you to pack more into the tank. Of course, some rockets then pipe the helium to the engine, warm it, and pipe it back to the tank so you get more pressurization with less helium, thus defeating benefit #1. I think SpaceX's Merlins do this, but don't quote me.
Fourth, this has been standard practice for a long time, over 50 years. There's a lot of engineering experience behind this and aerospace engineers are loath to fix something that's already working. For example, I know that Saturn main stages all packed their propellant pressurization bottles into their cryogenic tanks. If they had external bottles, it was only for engine restart.
As an adjunct to point 4, I think this is the first time such a failure has occurred. There are a lot of things that can go wrong in rockets with spectacular and exciting outcomes, but (I think) no one's previously managed to have a helium bottle break loose and go zipping around inside a propellant tank.
No, I do not know, otherwise I wouldn't be asking now would I? It's not really reasonable for me personally to be intimately familiar with the design of every rocket ever built, I've spent my time learning other things. The possible benefits are good to see listed, even if the reference to it being a method used by the Saturn V does put an amusing spin to the boasting that the Falcon is a completely new design for the 21st century, etc.
The idea of putting multiple pressure vessels, any of which contains sufficient volume of gas to burst a larger pressure vessel, into said pressure vessel, just isn't something I'd think worth the payoff. Does anyone happen to know even rough numbers for just what the percentage of benefit is? If it's under a 25% boost it just seems a lot of faffing for minimal gain versus just mounting them in the free structure between tanks.
@Drybones re your "new design" comment: you appear to have missed the point that Falcon is a long skinny white-and-black painted cylinder with the engines at one end, just like the 1960's Redstone rocket, which puts yet another amusing spin on things.
Or alternatively perhaps you've missed the fact that you can have an all-new design for, say, a car and still put, say, the steering wheels at the front...
@ DryBones "Re: The question nobody is asking... "
If the presurising gas tank or it's mounting fails the launch fails. It doesn't matter if it's in the propelant tank or outside it, the outcome is the same: launch failure and loss of vehicle.
Inside the tank then an over pressure pops the tank, outside then loss of pressure cut's fuel flow, engines are starved, thrust drops and the self destruct goes off.
If there's advantage to mounting it inside the tank then that's what you do. I guess it saves the weight of some plumbing and tank wall unions. The only disadvantage I can see is the mounting may have to support it against bouyancy (when the propelant tank is full) as well as it's mass (when the propelant has drained below it) but I expect the vibration forces are larger than either of those anyway.
Had the struts been up to spec then there would not be a problem, so it's not a design issue, it's a QA issue.
That's why the delay I expect, they are doing a QA re-validation of every part of the system and that takes time.
" even if the reference to it being a method used by the Saturn V does put an amusing spin to the boasting that the Falcon is a completely new design for the 21st century, etc"
The Zilog Z80 and Intel i7 Core (Core i7?) are both digitial microchips that are electrically powered and use transistors. The i7 is still a completely new design.
Likewise, the Saturn V and Falcon 9 have some commonalities, including kerosene-burning first stages and helium pressurants. But they're not the same rocket, or even using the same technology.
Does anyone happen to know even rough numbers for just what the percentage of benefit is?
How do you quantify simplification of rocket engineering and ability to learn from historical examples?
@Chris: To my mind, if you have a seal or line or interface fail on the helium tank and it's outside the LOX tank there's at least the chance the helium will just vent to the interior and then the exterior of the rocket through the pressure equalization vents, and the other tanks will pick up the slack. Yes, if it goes off like a bomb or your valve sticks open and your reg fails you're still pretty well done, but the chance of a failure resulting in LoM seems smaller to me. I'm still weighing chances/results of structural insufficiency for external vs internal, and wondering if bad steel grain is something that can be tested for acoustically.
@Cray: I was wondering if anyone knew how much more capacity it bought them, versus being outside but near the LOX tank. Something to put on the "put them inside" side of my mental scales. Since the helium is needed for pressurizing the RP-1 tank as well, it seems it still needs at least 1 route out of the LOX tank, so I think the "reduction in penetrations" point doesn't seem valid. I'd imagine they make a single penetration of a certain size, weld a single-piece assembly with multiple penetrations in it there, and route reactants and purges and sensors as designed/needed.
"Since the helium is needed for pressurizing the RP-1 tank as well, it seems it still needs at least 1 route out of the LOX tank, so I think the "reduction in penetrations" point doesn't seem valid."
Why? RP-1 tank could have its own helium tanks inside. That'd avoid a lot of troublesome plumbing.
"Since the helium is needed for pressurizing the RP-1 tank as well, it seems it still needs at least 1 route out of the LOX tank"
Each propellant tank generally has their own helium bottles because they need to be pressurized to different levels*, and thus have some differing hardware.
*Because the pumps on the same engine are designed differently to handle the different fluids.
I have no ill-will towards Musk Enterprises, although I do know that the people claiming it runs without massive state subsidies are fools.
Just as Scaled Composites should have been disqualified from the X prize because of massive cross-subsidies from the US military, Space X runs off massive state subsidies. Private venture it may once have been, if it ever was, it sure isn't now.
I like our supply ship, it seems efficient.
Still, the only thing getting people up to and down from orbit is the good old Soyuz, with many technical (the brilliant auto-docking system, among others) and nagivational (fast path to orbit) innovations.
They are not ineffective and are valuable things. US arseholes in Congress complain, but they will be paying (and already are for Space X) more per launch than for Soyuz. When the US military-industrial complex gets their incredably subsidized thing into orbit, it will cost even more.
I wonder what happened to the space-tourism offer of a slingshot on Soyuz around the Moon. Plenty of arseholes for whom it would have been small change.
Maybe the radiation problem is too great and the Moon landings were fake, I only say that to be irritating, but radiation outside the van Allen belt is harsh, the Moon-bound astronauts had no special protection.
Nobody has taken up the circumlunar Soyuz offer, and I think it is no longer on offer.m
It's important to distinguish subsidies from purchases. SpaceX has not to my knowledge done any cost-plus contracts with NASA or any other gov't agency. But NASA does buy stuff from them, including paying for launches. I expect, but don't know, that NASA may also be paying for additional development costs for features that SpaceX wouldn't otherwise have any use for. That's also not a 'subsidy' but payment for product or service.
Recognize that their fixed price (the catalog price is on their website) being less than 1/2 the price of of the 'old space' firms like ULA has caused ULA in particular to reorganize their entire company to reduce their operational costs so they can compete with SpaceX. This is resulting in huge savings for NASA (notwithstanding the lack of Congressional wisdom.)
Several tourists have paid the $20 million or so it costs to do a stint in LEO on the ISS. But AFAIK nobody has proposed a cost to go around the moon under $400 million, and while some Russian oligarchs and Arab sheiks have paid that much for their new yachts, you can use a yacht for more than a week. For national agencies, just riding in a capsule around the Moon doesn't have enough benefit to justify the cost. A couple of companies have come up with lower cost moon landing proposals - Golden Spike company comes to mind - pricing at $1.5 billion for two people to land, stay a week, and return. But there hasn't been much interest. I think it's just too early.
For Gary and the other poster
Predictably, I get the flood of downvotes for any word questioning Scaled or Space X.
In the case of Scaled, the lifter aircraft (White Knight) was based on a project for the US military. If you bother to read about their history, it is perfectly clear.
Cross-subsidies for the ridiculously named 'Space Ship One' clearly flowed from their military projects.
Various other teams were trying to do genuinely independent designs, but it all came to nothing for the massively subsidised success of Scaled, completely defeating the purpose of the prize.
Win for the US military-industrial complex, loss for innovations in space engineering.
I was excited by the video of the flight, saw it at a big electronics shop, but don't try to tell me that they had anything but a negative effect on the contest.
Will agree that Space X is more interesting than ULA, but the OP's or your '$20 million' (from a state govt.) is bullshit, the subsidies are in the many hundreds of millions, already well over a thousand of them I would think, and Musk seems very shy about spending any of his own gargantuan fortune on the project.
The information is readily available, so I won't be spoon-feeding it to you and other besotted Scaled and Space X fans.
Apart from the one or two successful supply flights, Space X has delivered nothing so far, ergo, is receiving massive subsidies.
@Bleu, it would appear that if there is a fool around, it may be you. Your assertion that SpaceX "runs off massive state subsidies" is unsupportable (i.e. totally false).
It is true, of course, that SpaceX enjoys subsidies: you can easily find a $20M subsidy from the state of Texas, but that's hardly "massive", and it's worth noting that anyone who offers to bring a certain level of investment and employment to the state could get the same deal, which is to say that the "subsidy" has nothing to do with what the business does, just where and who it employs. And in the whole scheme of things, $20M is peanuts.
What you omitted to mention was that, err, Soyuz enjoys a level of state-sponsored subsidy that makes every single US operator look like an amateur. Yes, Soyuz is cheap, but that's because it was paid for by the USSR and still operates under Russian military control.
Oh, and while we're at it, you're also dead wrong about the X Prize and Scaled Composites. The "cross subsidies" that you're probably alluding too were not subsidies, but the use of privately funded developments to sell capabilities to other customers, including the USAF (which, by the way, wanted to rent time on White Knight, not SpaceShip1).
Part of what SpaceX is doing is finding out where the balance point between testing and cost is.
Well, they seem to be doing that the expensive way. I don't know how much that failed launch has cost them, but it would surely have paid for a hell of a lot more testing...
It's really hard to get commercial officers to properly acknowledge risk in all companies. Generally you have to have some sort of corporate disaster before they learn the lesson properly, after which the company might not be around anyway, or they've been sacked, jailed, or whatever. Ask BP, TEPCO,
"Since the Falcon Heavy is more or less three Falcon 9s strapped together, it's no surprise that SpaceX's strut woes have prompted caution."
Three Falcon 9 first stages and one second stage. Unless the problem could affect the first stages as well, the Heavy shouldn't be more vulnerable than the Falcon 9.
Remember, SpaceX tries, as much as possible, to have the same stuff - tankage, piping, pumps, control systems, wiring etc - in the 2nd stage as can be found in the 1st stage (and vice versa). It means everyone is familiar with all (or more) of the systems in the rocket, and more people can spot a potential oopsie in the making.
So those helium bottles will be found in the 1st stage tanks too...
It's a shame that the Falcon 9 (and Falcon Heavy) will be grounded for a while but, hey, better safe than sorry...
I suspect they're not; they just convert it all for their public pronouncements cos that's what everyone in the States uses. Obviously once El Reg rules the world (clearly the commentariat are well on their way to doing so) we will have all measurements in linguine and double decker buses, as any modern person would.
Paris, cos some sizes just seem wrong in anything other than inches
Thank you for a real observation. Having a failure at 2000 pounds instead of 10, 000 pounds sounds like major shortcuts in engineering and understanding failure modes, rather than quality control. Or maybe it is the first item they could assign blame, rather than the results of an indepth fault tree. It certainly keeps the illusion that Space X is the genius organization we are meant to believe. Don't look behind the curtain. And how that translates to a 3 year delay for the heavy lift vehicle seems like a manufactured excuse to cover up a separate underlying issues. Space X works Congress and NASA pretty heavily so that good PR is their lifeblood.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
