From what I have seen, the development methodology they used was not actually 'Agile'. It was one similar in principle to Agile but mostly reserved for tasks relating to student groupwork. Its technical name goes by 'Clusterfsck'.
'Agile' F-35 fighter software dev techniques failed to speed up supersonic jet deliveries
Agile methodology has not succeeded in speeding up deliveries of onboard software for the F-35 fighter jet, a US government watchdog has warned in a new report. The US Government Accountability Office (GAO) said in its annual report into F-35 design and development that software development practices within the F-35 Joint …
COMMENTS
-
-
Thursday 25th March 2021 17:56 GMT Anonymous Coward
Which is often what agile ends up delivering.
I think it has its place if done in a certain way but it's often sold as a silver bullet and then implemented religiously and inappropriately.
I also get a bit fed up being told that traditional "waterfall" projects spent a year analysing what was needed and then came back to the business three years later with something they no longer wanted. What a load of rubbish, I can only think of one government project which came close to that. All other projects kept in regular contact with the business and failures were usually due to issues in the business... moving goal posts, changing management whims, planned resources being put on other work at the last minute etc. etc.
The whole waterfall image also present a false image of one slow thing after another. Again, not true in my experience. You always look at what can be done in parallel, or if things can be moved around if you're blocked on a point.
Oh well, rant over :-)
-
Friday 26th March 2021 07:59 GMT david bates
>implemented religiously
Unfortunately not. Too often its either used as an excuse to cut corners, to wing it, or if it starts off with good intentions to wing it when things start going pearshaped. I've worked on several projects where 'agile' meant 'we're not document - well - anything'.
-
Saturday 27th March 2021 15:29 GMT Robert 22
In my experience, projects implemented using a waterfall model often run into problems because of faulty assumptions made early on. Aside from this, there are usually players who have motivations for producing misleadingly optimistic reports of progress.
Conversely, I've seen very successful projects that were implemented using an iterative development model.
-
-
Thursday 25th March 2021 19:20 GMT Anonymous Coward
Re: Basics
The agile methodology can't fix graft and fraud. The Apollo project succeeded in part because of a genuine will to succeed. The F-35 project was designed to fail by going over-budget. When that became clear and the project wasn't cancelled, they doubled down. The later it is the more money they make on it.
Unfortunately because the lack of urgency isn't just impacting cost, what is being built has slipped in terms of both time and capability. It's decades late instead of it's development being sped up, it can't meet the requirements that were laid out 20 years ago, and by the time it is operating semi-reliably, it will be obsolete in almost every meaningful way. The trade-offs of fighter performance vs obervability by dotcom era radar systems don't look good 20 years later. And it will still be eye wateringly expensive.
By the time the F-35 is combat ready, it's operators would be better served having put the money into combat UAVs and support craft.
-
Thursday 25th March 2021 21:00 GMT vtcodger
Re: Basics
from Wikipedia "The Apollo flight computer was the first computer to use silicon IC chips. ... The computer had 2048 words of erasable magnetic-core memory and 36 kilowords of read-only core rope memory. Both had cycle times of 11.72 microseconds. The memory word length was 16 bits: 15 bits of data and one odd-parity bit."
Which is to say, nowhere near as powerful as an Apple-IIe or the original IBM-PC. Probably coded by one person or a handful of people in assembler. Probably not capable of managing most of the subsystems in a 1999 Toyota Camry much less those in a modern fighter-jet.
We've moved on a bit.
Not that there isn't a lot to be said for keeping things a simple as possible.
As for Agile. Haven't tried it or seen it tried, but creating satisfactory mission critical software is not one of the things I would expect it to be capable of.
-
Friday 26th March 2021 18:24 GMT EarthDog
Re: Basics
Why amounted to adapting to new information as it became available, making things up as you go along, ruthless testing, focused on doing one job well, attracting the best people, and being results focused. You know, being agile. Not "Agile", that's a product, but "agile" which is a paradigm.
-
-
Friday 26th March 2021 08:40 GMT bombastic bob
From the article:
C2D2, or Continuous Capability Development and Delivery. This has been less than stellar
I guess this is 'similar in principle to Agile'. And apparently, JUST as *FRAGILE*. Yeah it was supposed to rhyme...
I'll just say it: Developing software to a moving target is NEVER going to get things done. Neither is meeting mania nor analysis paralysis nor changing directions so fast you could generate electricity with the circular motion.
-
-
-
-
Thursday 25th March 2021 19:53 GMT Anonymous Coward
>>>Air Commodores
SX-64s ?
Here's a more sobering stateside opinion of HMS WhiteElephant:
https://thehill.com/opinion/national-security/542290-royal-navy-in-the-pacific-an-ally-against-china-where-we-need-it
"and U.S. pilots will comprise between a third and a half of the Brits’ strike-capable air wing."
Rule Britannia.
-
-
Thursday 25th March 2021 17:48 GMT TaabuTheCat
Figures
And the more I see the results of "design on the fly" (pardon the pun) in mainstream products like all of the 157 Microsoft portals, it's clear to me "agile", "CI/CD", and all the other cool buzzwords around this philosophy simply hide the fact that no one is thinking about the end before they begin. At least with waterfall you had to consider what "finished" might look like. Now? It's a dogs breakfast with constantly changing UI, levels of abstraction that make no sense, duplicate ways of doing similar things - none of them well thought out - and generally no cohesiveness at all to the product. And that's just O365, but I've seen it in other "go fast, break things" products as well.
Good engineering and design is hard. Being able to change your mind - and the product - every time it becomes clear you haven't thought something through, or living in your own little world and not thinking about how what you're doing interacts with the larger whole, well, that's easy. It doesn't require thought or imagination and it allows you to keep pushing difficult design decisions down the road until there's no way out but to start all over now that you have some clue what you are doing. How about we just admit that "fast" isn't the end-all. Real engineering, real design matters too. And thinking about a usable end state and what it will take to get there needs to be part of the plan.
I used to work with guys that thought of their designs as something that could be "elegant", and I saw that elegance more than once. Haven't seen it out of this new methodology, and I doubt that I ever will. Now get off my lawn.
-
Thursday 25th March 2021 21:51 GMT sbt
Re: How about we just admit that "fast" isn't the end-all.
Agree with the rest of your comment but I'd argue Agile, despite the name is slower vs. waterfall if you know what you want. It is suited to projects where the goals/capabilities/behaviours of the resulting product are unclear at the start. If you know what you need just design and build it without all the iteration and wasted time.
For a fighter plane, surely it would have been quicker overall to decide what it could do and design and implement according to that?
-
Friday 26th March 2021 08:39 GMT Robert Grant
Re: How about we just admit that "fast" isn't the end-all.
That's exactly it. Agile isn't faster. Agile is more likely to give you something useful at the end (and also give you more useful things along the way). If you have the luxury of a huge discovery phase, and/or your requirements are already painfully clear, then you will need Agile much less.
-
Friday 26th March 2021 15:39 GMT Doctor Syntax
Re: How about we just admit that "fast" isn't the end-all.
"If you have the luxury of a huge discovery phase, and/or your requirements are already painfully clear, then you will need Agile much less."
Take out the "and from "and/or" because if your requirements are clear then why would you need a huge discovery phase? So now we have "if you have the luxury of a huge discovery phase you need Agile much less" and you have "If your requirements are painfully clear you need Agile much less". That leads me to the situation that you need Agile if you have unclear requirements and not enough time to find out what they are. Those are projects that are doomed from the start so it's not going to help anyway. So we're left with the situation that a project that's feasible doesn't need Agile.
-
-
-
Friday 26th March 2021 06:48 GMT A Non e-mouse
Re: Figures
The problem Agile is trying to fix is having a project that takes X years to deliver then isn't appropriate because either the customer asked the wrong questions or the requirements changed and you couldn't change the design as it had been locked in.
The problem I've seen with Agile is that a section/module is shipped and then never updated as leasons are learned through the development of the complete project. This, as you mention, can lead to an incoherent product.
All methodolies have good and bad parts (for the project you're working on) The skill is picking the right parts for your project/circumstance.
-
Friday 26th March 2021 13:54 GMT juice
Re: Figures
> The problem I've seen with Agile is that a section/module is shipped and then never updated as leasons are learned through the development of the complete project. This, as you mention, can lead to an incoherent product.
Way back when, I worked for a telecomms which had recently brought in a new CIO, who had decreed that the entire company needed to switch to "agile" methodologies.
And that course stressed the concept of a "minimum viable product". Which essentially equated to "build just what you need to satisfy the current sprint"
(Interestingly, the agile community has made an effort to repurpose/reclaim this terminology to mean "build a prototype which you can use to get feedback from the customer", complete with some snarky comments about how people misunderstood what it meant.... https://www.agilealliance.org/glossary/mvp)
And this caused caused a few raised eyebrows.
In the first instance, the people who dealt with hardware rollouts - such as deploying the network infrastructure to a new building - were a bit bemused at the concept of splitting stuff into sprints, since that's much more of an "all or nothing" process which far more closely aligns to the waterfall model; all your planning needs to be done up front before you start plugging in kit and pulling through cables.
In the second instance, I questioned the principle of only delivering what was needed for the current sprint. The trainer was using the analogy of building a house floor by floor, so I used this to point out that if you want to add an extra floor, then you need to make sure the ground floor is specc'd to handle the weight (e.g. if you decide to go for a bathroom with a built in hot tub on the first floor). And similarly, if you then decide to add a second floor, then you need to make sure that both the ground floor and the first floor can carry the weight.
And so on.
Fundamentally, you should always do some work to make sure that whatever you're creating is scalable, both in terms of volume and features. And that directly conflicts with the concept of "just build what you need for now".
And for an added bonus, no matter how agile you think your processes are, there's a huge amount of intertia involved in going back and rebuilding something that's already been deployed.
Unfortunately, that's something which people often seem to forget when planning agile sprints; the focus is on "now", not "tomorrow".
Arguably, that's not a fault with agile per se, but rather a fault of the people implementing it. On the other hand, if it's something that a lot of people get consistently wrong, then maybe it's time to change the way that the methodology itself is structured...
-
Friday 26th March 2021 16:47 GMT Strahd Ivarius
Re: Figures
I had the same kind of experience (twice) with Agile projects in IT.
As long as you need only to deploy the result of your sprint to servers, you are mostly fine.
As soon as you need to deploy the result to the end-users computers, then you run into troubles because you have lots of external constraints to take into account .
One of the most important being; don't disturb the users when they are working, they are the ones bringing in the money!
-
Friday 26th March 2021 21:09 GMT veti
Re: Figures
The methodology that is usually called "agile" nowadays (more specifically, "scrum") was developed to handle the scenario where a waterfall project had failed. A project had been attempted, but it was over budget or past schedule and hadn't yet delivered anything useful.
In that scenario, there are requirements (although they're probably wrong) and a pile of code that's at least somewhat related to the project. And given that starting point, scrum can indeed deliver great things, and deliver them quickly.
The problem, IMO, appears when it's adopted as a methodology in its own right, without any such starting point. In that case it needs much more structure/specific requirements, and even with those in place it's much less powerful (because it no longer gets to stand on the shoulders of giants).
-
-
-
-
-
Friday 26th March 2021 15:23 GMT Jellied Eel
Re: Figures
Speaking of which, an F35 managed to shoot itself recently.
The F-35's systems are so advanced and intelligent, it developed depression and wanted out. There were some other odd problems, like engine degradation leading to developing problems and heavy landings. Or pilots developing barotrauma due to cockpit pressure changes. Or non-US operators not being able to keep secret details of how they're using their F-35s.
Turning the UK's carriers into drone motherships seems like a much better idea.
-
-
-
Friday 26th March 2021 13:39 GMT Anonymous Coward
Re: Figures
it's clear to me "agile", "CI/CD", and all the other cool buzzwords
Agile or not - CI/CD is not a buzzword. If you have a problem with automated testing to validate quality and want to make manual software releases...
Agile doesn't mean changing your mind or design constantly - in fact, that's a sign that a team or owner is NOT actually doing agile, they're just winging it. In a proper agile project, you need to have a clear goal/vision from the product owner - that doesn't change over the course of the project - and a clear plan from the technical lead and architects that describes in broad strokes all of the technology to be used and the rough order that work will be undertaken in. This can takes months or even years before the agile dev team start working on it.
Where agile then comes in is taking these broad strokes (epics) and revising them into actionable work (cases), putting rough estimates on the cases and determining a timeline. The key points of agile are in continuous revising of these estimates and tracking team efficiency in order to track progress and estimate completion dates.
Done right, its waterfall with slightly less upfront planning - the last 3 agile projects I've worked on delivered on time, on budget and with the features we said we'd deliver. Done wrong (and it often is), the team starts with no clear purpose, the project owner aiming to "discover" what they want going along, the team fails to deliver on time, and the end date for the project continually slips.
-
Thursday 25th March 2021 17:58 GMT a_yank_lurker
UnAgile
Too many forget that you need to think seriously and actually have a solid idea before having code wranglers at a project. I have some nominally 'agile' projects come past me with virtually no specifications were someone has thought through the scenarios and have a general idea of how they should be handled. It makes for code that is impossible to properly maintain when it is found out that major use cases were never considered in the design given to the programmers. Use cases never considered by the programmers because they did not know about them.
Agile does not mean there are not design documents and preplanning but that groups are in contact with each other throughout the process. Issues are dealt with as they arise not after the fact.
-
Friday 26th March 2021 06:53 GMT Joe W
Re: UnAgile
Have one of those ----->
There is a tendency to "make everything agile", by which magmt means... actually: I'm not sure what (if) they think. Then there are some colleagues who were there when dirt was invented. They look at these "new" concepts and then tell me the three or so versions of what came before and was essentially the same thing. They are used to working together with different groups ("stakeholders") and accross teams. They stress that none of the new methods are in fact new, and that the no-nonsense parts are in fact what should have been done in the earlier "methods" as well. Think about the goals, and then formulate a clear project. Continuous delivery just means that you can implement some stuff earlier and some stuff in the next version, and that you can react to new demands (e.g. rules and regulations) in mostly some very well defined spots and that you design it that way, not that you mill about because noone has a clue about what the end product should look like.
Yes, there are the typical animosities and kindergartne sand box fights you have everywhere, but the job bloody well gets done (ok, not all of them, but that's due to... ok, I'll stop and have one of these
----->
as well...)
I need the weekend.
-
Friday 26th March 2021 10:10 GMT Warm Braw
Re: UnAgile
I took a look at Agile recently as lockdown boredom was taking hold and I was thinking about going back to work. I was actually quite taken aback - it's more of a cult than a process and I ended up feeling compelled to write about its fundamental misconception:
It has is own language in which every word seems laden with meaning, though that meaning may not readily be apparent: at times I struggled to work out whether I was being encouraged to be a more productive developer or a higher level Operating Thetan. Curiously, the jargon seems to have been imported from, literally, another field – the sports field. Its sprints and scrums and team room are only a communal shower short of a Gillette commercial .
It seems to me to be the precise opposite of a useful methodology: it takes a simple concept (which, regrettably, is the concept of a software development sweatshop, but we'll let that pass) and obfuscates it with allegorical mysticism. I can't see how it can possibly deliver anything of even modest complexity.
-
Friday 26th March 2021 12:00 GMT Greybearded old scrote
Re: UnAgile
Not liking docs is right there in the manifesto. "Working software over comprehensive documentation"
If you're working with things that go bang or other people's money you'd better have correct behaviour bloody well documented.
(I'm doing payroll code atm, to a spec of "whatever the old code does." I'm grinding my teeth so hard there's likely to be an explosion.)
-
-
Sunday 28th March 2021 10:04 GMT CRConrad
Could just as well go the other way around:
– Now pay us!
– Naah, don't wanna. Betcha it doesn't even meet the specs.
– Show us where it fails the specs, or we'll sue you and then you'll have to pay us anyway, because you won't be able to show the court where it fails the specs... Because you didn't give us any specs.
-
-
Friday 26th March 2021 16:11 GMT batfink
Re: UnAgile
Yes that's the main troublesome bit with Agile. Too many times it's interpreted as "don't bother with documentation", although that's not what it actually says.
So, all that does is store up problems, and we're going to come to the point where managers realise that this is going to cost money.
Comprehensive documentation was invented back in the Good Old Days (yes, I was there), when those holding the budgets worked out they were paying people to spend time working out WTF had been done in the past from first principles and guesses about what the original developer was thinking at the time. They worked out that it was cheaper to accept the extra development time and cost to do it right in the first place, otherwise the TCO was going to blow out.
Unfortunately that's now too long ago, and everyone's forgotten.
-
Friday 26th March 2021 17:28 GMT Warm Braw
Re: UnAgile
they were paying people to spend time working out WTF had been done in the past
It's not just that, though that's certainly part of it. If you have a team of people working on a project, there has to be a shared understanding of what they're doing. They need to agree on a representation of the data. They need to know which bits of the system are performance-critical and be confident that they can meet the performance criteria. That needs to be recorded somewhere. "Black box" testing (which is the only thing you can do if you insist on writing your tests first) is not adequate in the general case. "Refactoring" is, in my view, a sign of failure - that the code was ill-considered in the first place.
I can understand that if you're burning through your seed capital and need to deliver something to convince your backers to keep the taps open, it might be an acceptable approach. But only as long as you view the result as a prototype and not the finished article.
-
-
-
-
-
Friday 26th March 2021 12:02 GMT codejunky
Re: Phew!
@Androgynous Cupboard
"Lucky we didn't design an entire aircraft carrier so it could only fly this type of fighter"
The good news is we didnt. The design was to support a catapult system if we decided to do so. Except BAE claimed an outrageous price to make the modification (which would be done in the US for vastly less, so its all cream).
None of this stuff is done sensibly until there is an actual threat of war.
-
Thursday 25th March 2021 23:27 GMT Chris G
"For over 20 years, we have consistently emphasized the need for organizations to collect and use data about program performance to help inform and measure organization operations and results."
Why the hell would any pork barrel recipient pay any attention to such emphasis?
It could lead to someone being answerable for lack of performance.
-
Friday 26th March 2021 00:07 GMT Anonymous Coward
At some point, somewhere along the line there would have been a standup that went...
> Agile methodology has not succeeded in speeding up deliveries of onboard software for the F-35 fighter jet, a US government watchdog has warned in a new report.
Scrum master (reporting-up the common issues found from sprint reviews): "Quite often, when we push something back a bit we find that dependencies mean we would end up doing something silly like delivering the take-off-mode software and finding that the landing-mode-software has now been delayed until after the first test flight. It's starting to eat up a lot of time as the devs check the dependencies. What if we produced and maintained a list of tasks and their dependencies so we could easily check?"
PM: "You mean a plan?"
-
Friday 26th March 2021 02:20 GMT sanmigueelbeer
the UK seems set to slash its order for 138 jets as defence chiefs start eyeing up domestically built unmanned aircraft instead
Does UK DoD still have money left?
Firstly, RN spent so much money on QE2 that it is unable to fund F35 numbers for QE2 that UK defense had to "rent out" deck space to the Americans.
Now, the RAF is retiring C-130 early maybe to get funds to buy the uber-expensive "drones".
-
Friday 26th March 2021 10:37 GMT Anonymous Coward
Not quite true. The carriers ran a bit over budget, but not hugely, and both of them cost less than the price tag for holding the London Olympics.
The F-35 fleet is shared between the RAF and the RN, and the figure of 138 was only ever a provisional one. The first operational deployment of HMS Queen Elizabeth will have a mix of RN and USMC aircraft, but this joint deployment has been planned for years.
The RAF is to begin retiring its C-130 fleet, but it currently operates two other types of transport aircraft, the C-17 and the Airbus A400, the latter of which was designed to replace the C-130 anyway.
-
-
Friday 26th March 2021 03:09 GMT J27
Agile development methods only really work for things where it doesn't matter if some things break sometimes and you have constant analytics. Like a mobile app or web application. For a real-time system where any small failure kills someone? Madness. No wonder these things are such a disaster.
-
This post has been deleted by its author
-
-
Friday 26th March 2021 03:43 GMT Andrew Williams
Agile as a brick
Understand the problem domain. Understand the problem. Produce a design. Come up with a build plan, with tests and checks etc.
Nah, duck it, let’s code something, anything and see what happens. That’s agile.
Side note. The “agile crew” at work have burnt two months and produced nothing, while the designers are now looking at a successful product launch and follow on systems.
-
Friday 26th March 2021 06:38 GMT Mark192
Non-coders?
Guarantee that the cause of this is poor coding not being punished because it's quick, and good coding being punished because it /looks/ slow.
Root cause? Non-coders managing coders, because management is the profession, right?
My managers used to look 'key performance indicators' as if they were targets because they didn't have enough knowledge to evaluate the actual work.
Result?
- People that should have been re-trained or sacked were lauded as high performers.
- Big problems went unfixed because junior management were hounded by senior managers to get their team hitting the 'targets'. Work snowballed because short-termism ruled.
-
Friday 26th March 2021 13:39 GMT uccsoundman
Re: Non-coders?
I worked in QA and the management metric was "number of test cases written" (not "number of requirements tested"). So the QA contractors wrote a script that auto-generated > 10,000 test cases, each testing exactly the same thing. They were RICHLY rewarded for far exceeding the production goals. Those who tried to write useful test cases were sacked for being too slow. Results: bad software.
-
-
Friday 26th March 2021 08:39 GMT Anonymous Coward
rm -Rf /
No methodology is going to fix code coming from government contractors. The recruiters, the hiring managers, and the employees are incompetent at writing software. It's an endless cycle of incompetent software engineers interviewing incompetent job applicants and thinking they're OK. A PHP team literally told me that SQL injection can't be fixed or avoided because parameterized queries would conflict with approved coding standards. Any accidental hires of competent engineers will run for their lives once they see what their massive pool of coworkers have been doing for the past 10 years.
-
Friday 26th March 2021 08:47 GMT jmch
Standard practice
"developers "routinely underestimated the amount of work needed" "
a) if you're developing something completely new or very cutting-edge, there's no way of even estimating how long it will take. Giving an estimate based on different work previously undertaken won't work
b) In my experience, developers are routinely pressured by management to revise their estimates downwards. Management look more kindly on those who promise earlier delivery in an unrealistic timeframe than on those promising a later, but realistically timed delivery. When, for large projects, delivery times are measured in years, accountabilty for the original estimates quickly disappears (helped on by high turnover in project staff)
c) Related to the above, lots of changes in project team members don't only screw with accountability, they screw with productivity. And you get frequent project team changes when management keeps outsourcing to lowest-bidder sweatshops.
Without knowing any details and looking from outside, of course, but I would hazard to guess that the lateness and cost overruns have very little to do with use or not of Agile* methodology, and much more to do with attempted** cost-cutting.
*of course there ARE issues with Agile methodology, mostly that very few dev teams that I've seen actually follow proper Agile process, and just stick bits of it here and there and call it 'Agile'
**attempted since it looks nice in short-term but usually ends up costing more long-term
-
Saturday 27th March 2021 14:47 GMT Anonymous Coward
Re: Standard practice
Re: *of course there ARE issues with Agile methodology, mostly that very few dev teams that I've seen actually follow proper Agile process, and just stick bits of it here and there and call it 'Agile'
Indeed.
My multinational company customises Agile by taking out the parts it doesn't agree with / like. Then trains everyone that their version is the real thing.
And bastardizes it and calls it Business Agile.
Then uses the Agile as buzzword bingo. After all, it just simply means fast, right?
-
-
Wednesday 23rd February 2022 18:17 GMT dboyes
> You can't agile deliver, with sprints, on what are effectively safety critical systems
This. "Fail fast" application development is *failure* where lives are at stake.
One thing that seems to be missing from modern development is the role of the business analyst (someone who understands the business AND application development well enough to write implementable software requirements). Those people aren't cheap, but they're makable if the business invests in them as a priority. If those people are good, Agile vs waterfall vs ? is a non-issue.
-
-
Sunday 28th March 2021 09:00 GMT CRConrad
Good thing you used quote marks around "agile" in the headline.
From the article:
"The program's primary reliance on the contractor's monthly reports, often based on older data, has hindered program officials' timely decision making," said the GAO
So not actually agile at all, then.Because if it were, the JPO would have had its people at the software contractor, participating in the work every day, in stead of relying on any damn monthly reports.