Fragile Agile development model is a symptom, not a source, of project failure You've got to be light on your toes to keep up with Agile. Some say it's the inherently right way to do software. Others call it a cult, overblown and misleading. One way to tell would be to compare Agile and non-Agile project success rates, and that's just what consultancy Engprax did. The result: at least according to one …
Or even the right tool for parts of the job. As I've frequently repeated, agile and "waterfall" are not mutually exclusive approaches. At the simplest level for example, agile is ideal for UI development because users typically can't envisage how it will work in abstract but can identify good and bad elements of it on sight. On the other hand, application security is very hard to ensure consistently across a big agile project unless a full specification of what is required is presented to all teams -- i.e. a "waterfall" specification has been created up front.
The big problem we face is that methodologies have become more important than results, and not only in the IT sphere. It's become the basis for much of education in general, as witnessed by a recent UK dept. ed. spokesman stating that grade boundary marks should be shifted downward "to maintain standards" (i.e. pass rates).
So this is a cultural phenomenon not a software development one. It just shows up most clearly in software development because we rely on its results so much.
That.
In my line of business we do have certain things that are set in stone, and they should be, considering they are, in fact, laws.And then there are things we should be clear about beforehand, like how quickly do you want a result to be returned from the black boxen living in the underbellly of the datacentre. I'm pretty sure that ten minutes is too long. How about... 30 seconds? I don't know, nobody knows, nobody writes these things down. Would agile fix this? Maybe, if we could get users to interact with the half baked software, play around, and give feedback - that must be heard, not ignored or downplayed (gosh, how often have I seen that?).
From my point of view, if we could prioritise getting the whole stack to be more snappy instead of adding new features, giving the developers half a year time to do some housekeeping, let them pay off technical debt, that would be a Good Thing.
You're supposed to stop coding to add new features when the project hits its budgeted time limit, and ship the most-recent, working version.
Bugfixes and additional features/mods are budgeted separately. (Yes, you can get management which budgets time for the initial creation, but no time for bugfixes. If that happens, you're on the Crazy Train to Failville.)
Yes, you can get management which budgets time for the initial creation, but no time for bugfixes. If that happens, you're on the Crazy Train to Failville
Our agile developers only appear to care about the new build stuff ("Oh, shiny!") so bug fixes, well....
In my (now completed) testing/QA career I often stated if you want it tested for bugs and correct function, then I'm your man. However, if you want it tested for immediate release or to a fixed timeframe... My mantra for upgrades was 'is it better or worse than what is currently in place, and can we live with the new bugs introduced?'
> Sometimes Agile is the right development methodology for software
Ah, now.
https://agilemania.com/is-agile-a-framework-or-methodology
Agile is not a methodology or a framework. It is a mindset that enables organizations to be more responsive to change.
And therein lies a big part of the problem.
Agile means different things to pretty much everyone who practices it, and this is only made worse by companies/management layers which just pick and choose the bits they want. Or when they try to use it on projects where it really doesn't fit well.
For instance, I was once working for a major international company which decided that everyone needed to become agile. And so, agile training sessions were duly scheduled for everyone.
Partway through the description of how you build things in sprints, and with buzzwords like "minimal viable product" and "vertical slice", someone at the back piped up:
"My division installs comms hardware in new buildings. We don't have a minimum viable product. Nor can we do vertical slices of functionality. It all has to work when turned on, within a fixed schedule"
The trainers weren't really sure how to handle that...
> The trainers weren't really sure how to handle that...
Of course they weren't. After all, they'd been sold a bill of goods, and by then were trying to re-sell it to you.
Slightly more charitably, it doesn't always occur to proponents of these methodologies that there might exist projects or situations which don't fit nicely within the boundaries.
Reality can be messy like that sometimes.
This in itself is a big part of the problem.
If agile has any chance of being successful, the entire management chain has to understand what that means for project delivery, and allow everyone involved to work that way. That often means managers needing to adapt their thinking, their expectations, and they can be incredibly resistant to that. They don't really understand the concepts, and they can't / won't give up control to the process. So they pay lip service to it, but never actually do it properly. Still want everything spec'd up front in detail, still want to compartmentalise everything into neat little boxes.
Result is the company crows "We're agile!", but scratch the surface and they're still micromanaging everything. In those conditions it's no surprise at all that agile projects fail more often than non-agile projects. Doing anything half-baked is doomed to fail.
Exactly this.
I think a significant part of it is around the type of organisation and the type of project - Agile is great on a new project and particularly in a start-up type setting where being able to get something in front of users early is a critical success criterion and your understanding of the problem space is growing as you create the product.
When you're adding new features or revising an existing platform or working in a scenario where the problems and challenges are well-understood (and the development team have access to the people with that understanding) then something that offers a bit more clarity on anticipated timelines and completion dates is more practical. By their nature large organisations aren't agile - they're lumbering behemoths that are very difficult to redirect - and an approach to project management that understands that is surely more likely to result in success.
IME it always comes down to the people for the job.
If you have a group of talented motivated folks, with the right skillset, who can and do work together well, there's a pretty fair chance they'll produce good results on a project. Regardless or perhaps even in spite of the development or project management methodology used (inflicted) on them. The very best teams can often work it out on their own and just get on with it.
OTOH if you have a bunch of burned out hacks, perhaps demotivated by incessant management "help", maybe infected with a discontented rogue element or two, odds are they'll miss deadlines, deliverables, budgets, and probably have a miserable time doing it. No amount of management can fix that, likely adding management and fad methodologies will make it even worse.
Great teams are priceless. Bad ones are very expensive.
I think the fundamental problem is inability to make a plan and execute it, and it's getting steadily worse. Buzzwords and slogans are taking the place of planning, while smoke and mirrors are taking the place of execution.
If you can't plan, there is no methodology that will save you.
I suspect that Agile appeals to the buzzwords & slogans crowd, because it appears to provide justification for their deficiencies. You don't have to plan! In fact, it's better if you don't! This is not true, of course, and it only looks like that on the very surface. Unfortunately, that kind of mind does not look beyond the surface.
As a consequence, more Agile projects fail, but those same projects would probably have failed under any other methodology. Correlation, not causation.
I have had the pleasure of working with some 'managers' who have read some parts of some 'how to do x' management books and decide 'this is what we are going to do!' without fully understanding the real workings.
Sadly to upper management these people look good as they like the buzzword bingo. At the other end of the scale the ones doing the work are getting pissed off as timescales are vastly unrealistic and stuff just isn't working.
The best receipe for disaster projects usually is to not do the basics correctly. The easiest way to stack things in a way that no matter what talented individuals do everyone is doomed.
I've seen in industrial engineering projects that some principles similar to be found in agile manifesto are easily available but the management struggles to understand them and that is why someone simplifies the idea into lifecack with zero context that they are able to follow blindly. What could (not) go wrong?
What we know about complex projects (for example oasis of the seas cruise ship) is that you can have easily 1000 different companies working their own thing without significant issues as long as majority of basic things are done correctly from the beginning of the project until it finishes and there is enough resources to manage surprises and there is handful of people who know 24/7 what the status is with everything so they can check the big picture in under 30 minutes easily.
>> You don't have to plan!
And there you have hit the nail right on the head - my biggest pain of the "younger agile generation", they seem to have forgotten the concept of planning....and wonder why it all goes pear shaped later
Yes I am using a fairly broad brush there, so apologies to those of you that I just painted over unjustly...
Exactly. Still have to have a plan. A good plan. All "agile" does is help you adapt when that plan inevitably starts to crack, due to things that couldn't have been known about up front, incorrect assumptions, user feedback etc. (Everyone has a plan until they get punched in the face, no plan survives first contact with the enemy, and so on). You just have the opportunity to discover these things earlier in the dev cycle, giving you a chance to fix them, pivot or whatever is needed before you get too far in.
Way too many people don't grasp that. Then wonder why "agile" isn't delivering on its promises.
My experience is that when allowed to figure it out for themselves, competent people will evolve a system that works quite well for their project. It won't be agile and it won't be waterfall and it won't be any of the other daft methodologies that have been trendy from time to time. But it will be more or less optimal.
Then what usually happens is that some clueless manager will come along and demand that everybody must conform to their particular project management religion. And it's usually completely unsuitable for the project. Then everybody gets fed up and productivity falls. Eventually, after a lot of complaints and to prevent the project from complete failure, the methodology gets incrementally tweaked until it's not far off what was being done before.
And a new manager comes along and we go around the whole cycle again.
> My experience is that when allowed to figure it out for themselves, competent people will evolve a system that works quite well for their project
This is agile.
Not a set of defined processes/outcomes, but a team working to work better for the customer.
Of course that doesn't leave much room for management who want to be in control.
Original: My experience is that when allowed to figure it out for themselves, competent people will evolve a system that works quite well for their project
Reply: This is agile. Not a set of defined processes/outcomes, but a team working to work better for the customer.
No, it's not Agile in the sense of the manifesto. What they could come up with might be similar, but it could be diametrically opposed: "We team members have decided that this won't work well unless we write a full plan and specification right at the start. That's what we intend to do. Then we can build this without getting in people's way". Some projects work best with that. Some projects work terribly with it and can use Agile-type methods. People selecting to do something different than the manifesto are not being Agile, they're making their own decision.
To be fair, the manifesto states:
Working software over comprehensive documentation
Which, to me, means it is better to spend time on making more value, that is, time spent making software work than on documentation no one is going to look at.
If documentation helps you succeed that, then there is value to it so do it. If in your retros** you find the documentation is a hinderance, then you should look into what you can do to improve it.
** retros are important to me as it gives the team the ability and freedom to raise issues and allows everyone to fix them. We rarely have management in ours: just the developers, tester, and product owner. We joke, we laugh, we swear, we even banter on our retro boards. It's light-hearted fun but we achieve what we need to: improvements in our processes, which might not always be better, but at least we can learn why.
BUT: Agile cannot be the same for every team---each team will need to continuously go through a learning phase to learn what helps and what doesn't, and doing X now might be good, but then it might not be good next year.
Working software over comprehensive documentation
Which, to me, means it is better to spend time on making more value, that is, time spent making software work than on documentation no one is going to look at.
I'm afraid this is one of the parts of the manifesto that I take the most issue with, and I'm going to have to use you as an example why. There are many people who interpret that line in the same way. I'm not sure whether that's the same way the manifesto's writers were thinking when they wrote it. I hope not, and I think there's a reason that it might not have been, but as I've said to other defenses of the manifesto, they didn't bother explaining what they meant, so they're either in agreement with or responsible for this attitude when people use the manifesto as justification.
A lot of developers don't like writing documentation. They start finding reasons why they shouldn't have to. Managers also don't particularly want to do it, because it would seem to add time to the development process without making the software any more capable, and if the developers aren't going to do it, then you'd have to hire other people to do it, and that's expensive. Too bad for both groups because it is needed, and they should both know that. I'll tell you who suffers when the documentation's insufficient:
1. The users who read the documentation in order to use the software and get their job done.
2. The users who don't read it, but when they screw something up, can be referred to it, read what they should have done, and use it properly next time.
3. The trainers who need to learn how to use the software in order to teach users, but don't use it themselves. Winging it means training people in at best an inefficient and at worst a wrong way.
4. The support staff, if you're lucky enough to have them, who explain to users how something works even if they don't use it full time themselves. This also includes the tech-savvy colleague who gets a lot of routine questions from the less knowledgeable users.
5. Developers in general when they need to know what behavior is expected, so they know whether a certain change needs to be written differently, communicated to users, or expanded with multiple options.
6. New developers who need some way to learn what this project does and how it does it that's a little faster and more reliable than figuring out where the main function is and trying to read it like a compiler.
7. Me, as an established developer, when I want to help the new developers learn but don't have enough time to hold a complete course on it.
8. Me, as an established developer, when I need to teach someone in great detail or they'll be unproductive and our manager gets unhappy, but the lack of documentation means I have to spend a long time doing this, which means my code production decreases, which means my manager gets unhappy.
Apologies for the delayed response, I forgot about this.
My point is if NOBODY uses the documentation (never gets read) then there really is no value, because no one is using it. It's a bit like creating software no one uses.
But if someone uses it then there is value so do it. We have documentation where it makes sense and we even have some that needs deleting because there's no value in it anymore.
I'm also thinking we need other documentation to clear requirements up, instead of everything being in user stories, which I find confusing.
Crap devs aside, if there's value in it then you should do it.
Also you should most of them times ignore whatever the fuck agile evangelists say—do what works best, but sometimes asking them why they suggest things could be beneficial.
... typically from upper management, can doom any project. Example: manager conceives the Whizzo A/P replacement project. Users don't want it, because their current system works more than well-enough. They know the current system's limits snd glitches, and workarounds to those.
The users don't want a new system. Manager X has Taken a Position; the New System is his/her ticket to glory, and will prove Manager X is an "agent of change", etc.
The team is unlikely to get useful input or feedback from the users, and the project, despite the skills of rhe programmmers, is headed to Failville.
I am of the opinion that agile is not a good methodology and should be avoided. However, I recently did some consultation for a small startup company and realized that they actually have no alternative. So little cashflow and very volatile investors.
So ultimately, the decision is between a terrible development methodology vs an abandoned project and potentially lost opportunity.
That said, often they result in the same thing. One just feeds developers for a little longer.
If we're going to do potted history of methodologies, it's worth pointing out that one of the problems with Agile is that it delivered some significant early success to companies that read the manifesto and put some consideration into implementing the ideas (i.e. they actually thought about it).
Why is that success a problem? In the wake of a much needed shake-up to the industry, Agile went from a way to think about development to a buzzword, and on the back of that came a whole load of consultants and specialists who could (for a suitably large fee) teach your whole company how to do their special, branded version of Agile, complete with their special, subscription based Agile Tools (tm) that would give senior management Agile Metrics (tm), that they could use to improve productivity.
We've probably all sat through expert presentations (given by people who often failed out of software development) telling us how to do our jobs from the perspective of someone who genuinely hasn't got a clue what our company or business unit actually does. There will be graphics, special names to call parts of the process, a painfully complex workflow (but don't worry, you can buy.. erm.. subscribe to the software that will help with that) and more specialist teaching for people who's only job is to repeat the rituals, still without actually understanding what any of the technical staff they are shepherding around actually do. Does your 'Agile team' have more non-technical staff than actual code monkeys (you know, the ones who do the work)? Yeah... that.
So a good chunk of the fightback against Agile is not really about the core ideas and whether they work, but the way the term has been weaponised by an industry of grifters and rent-seekers, and (often willfully) misunderstood by senior management.
I agree with what you've said, and it fits in with what the article says, but probably not they way they meant it.
And so it is with the huge slapdash edifice of corporate IT development, with so much cultural and managerial inertia that change will come not with new ideas, but the aging out of the old guard.
I suspect the problems wont be solved with the old guard brought up on waterfall aging out, but rather it's actually caused by them leaving the coal face. Those are the people who understood why waterfall didn't work and had to work out how to make Agile work themselves, rather than being chucked on a week long Agile course and expected to know it all. They are no longer around to mentor junior engineers with the benefit of real world experience.
Ahhh Jira!! The infinitely configurable tool that can make your life an infinite living hell.
I foolishly introduced Jira at my last job as we had NO bug tracking whatsoever and everything was done from people's memory or going back through the source control.
It grew and grew and grew and more custom fields and more 'can you do...' and ever increasingly complex workflows all in the desperate hope that it could bring into line people who either wouldn't or, more accurately, couldn't follow procedure.
We had a change management system using Jira that mostly worked. Very complex changes were not easy to handle in it but then they are hard to handle in ANY system. So ops kicked up a stink that engineering had failed to correctly tell them how to do their job. So a HUGE project was kicked off to rewrite the entire change process. It was AWFUL! A huge confluence page that spawned jira tasks that spawned more jira tasks and despite many rounds of training it was just too complex. It was a work of art (regexp raw confluence pages to change data, more scripts than I could remember) but was just so wrong. It didn't even meet the criteria that kicked off the bloody thing as it relied on people who were too busy already to extract rows of info from the purchasing system and by the time you got to review it was massively out of date.
Spot on...
My last organisation was a content management system integrator and we used agile for software development but within a time and cost bound structure.
We worked for the worlds largest brands, implementing their front end tools to allow them to build web experiences and were voted by our clients the worlds best at it due to our customer empathy - ie we listened to their business problem and delivered them a mix of training, process, tools and software that addressed that problem. We documented all high level requirements, prioritised them, then did deeper requirements for those that were deemed important enough to go into MVP and through that process we could manage senior management expectations and the rate of change for the end-users within the business who needed to actually use the things we were building. Yes, we did allow for some changes, but that was done through a strict process during sprint planning.
Agile was not what we did. It was the way we did some things.
Having left that place and now working alongside companies that call themselves technology agencies, it's appalling to see methodologies being decided upon before even speaking to the customer. Or, Agile being used to do creative designs.
Agile is not unique in this... ITIL also went through the same cycle with people without experience (ie consultants) and tools providers happy to take your cash to answer whatever question you have with the phrase "ITIL solves that'.
I'm seeing the same now with the MACH alliance. Yet another sensible solution to a specific problem that's now in the hands of marketing and now being spread out in the attempt to gain market share it simply should not have.
Which is potentially a manifesto itself. ;)
"Changing the fig leaf can't change what lies beneath"
Presumably the orchestra and conductor*?
Most projects have received more than enough intimate attention from that particular conductor!
(Possibly the right tool for the job whatever that might be [which is often the core of the problem.])
* recalled from an episode of Home James! ITV 1987-90
I sit down with the client and a white board and go through the complexity of their want. If I don't understand something, I say so. It gives the client confidence and buy in.
Once we start on actual code, I ensure there is a weekly client + user group meet to go through stuff to make sure we're all on the same page... And if not we rework until we're all happy.
In the middle of all this comes the "would this be a better way" or "why do you do that" stuff that no one ever thought about before the kick off.
Repeat, repeat but aim for the delivery date and if it's slipping everyone is aware of why.
Etc etc
Quick, write that down (oh, you have - good start); now, invent a few terms (better yet, re-use some common words) for each stage, train up some consultants and - PROFIT!
Or just get on with delivering working systems to happy customers, if that is the sort of thing that floats your boat, weirdo.
The idea that you deliver an incomplete but working system regularly to users as development progresses, who can give feedback which is then acted upon if necessary, is quite a good one, and can save against death march projects and keep everyone having realistic expectations.
However if anybody utters the word "sprint", run away fast.
I struggle to take anything to do with agile programming seriously - at least the “agile” that everyone talks about. It overflows with buzz words, many of which making the whole idea sounds childish - “sprint”, “extreme programming” (extreme? Really???), “velocity”, “story”, “epic”… it goes on. It sounds like it was invented by a 7 year old.
I know many of these buzz words are not from the original manifesto but have been bolted on by various wankers and tool designers (I use the term loosely) but even so. It’s farcical and people use these terms without seeing them for the joke they are.
Having just escaped this crap and the use of jira in a previous job, I hope never to go back.
In my mind “agile software development” means doing your job properly and thinking ahead. It doesn’t mean write something as fast as possible only to repair and rework it a dozen times later on just because that shows your being “agile”
I don't mind sprints. I got used to them at the end of Python conferences and our local user group holds a couple every year, also called "hackathons" because… But I'm not sure we'd qualify to the gurus who make money selling the books and training courses…
What I've learned over the last 20-odd years is how important it is to set expectations for such focussed sessions: plan for something doable, plan for time to setup and to explain what you hope to do, plan breaks, plan food and drink, plan to finish on time with a review of what you got done, including what didn't work. Anything that looks introduces pressure will increase the chance of failure. And, while all-nights can be fun, they usually don't produce anything more than resuming after an evening out and a (good) night's sleep with thoughts on other things.
Oh, you want to integrate this into a project? Then the right amount of good, clear communication is key. Teams have to be built and looked after and no one was born knowing everything. All the technologies and methodologies are their to support the people doing the work.
> I don't mind sprints ... our local user group holds a couple every year, also called "hackathons"
Doing a single hackathon can be fun - like any event where you push yourself to get it done, the old neurochemicals flow and you get a rush.
Then you go home at the end of the conference and relax.
Unfortunately, you've fallen into the same trap that some Agilistas do (and which is encouraged by the very use of a word like "sprint") and confused the "hackathon" ethic with the "we are going to do the same thing next week, the week after, the week after..." of a work project.
> set expectations for such focussed sessions
Like, if you plan for every "sprint" to be that focussed[1] then you're going to burn out, or, more optimistically[2] everyone will get sick and tired of the word "sprint" and just mutter dispiritedly evey time it is mentioned.
FWIW, Good(ish) Agile also says "go home at 17:30 and forget about work" to avoid this, but...
[1] yah see, "sprints" are supposed to be focussed on which *bit* of the task is to be done; if a distraction comes up ("this lib is broken") don't fix it now, log it on the kamban, stay on task and, because we have a review within a week, that issue won't be forgotten. But "focussed wrt domain" gets muddled with the usage you just gave, where the *person* is focussed on the hack, to the exclusion of all else (hence the need to plan rest breaks, even loo stops!)
[2] from the pov of the devs: dispirited is better than burnt out; curiously enough, in the devs eyes, we aren't fungible!
I haven't fallen into any traps, just relating my experience. We hold two weekend sprints a year and they are a way for people to learn new stuff, try things out, solve problems together. We start at 10 and finish at 17:00 each day with an hour for review.
It's supposed to be coding, but not as part of the day job. However, I suspect that learning to present your idea, gauging the "doability" of a topic, working with others, asking questions from more experienced programmers, and providing a review are all skills that transfer to work.
As usual, coders / tech specialists giving managers a good bashing here. This is a one sided story. Some of my worsed managers have been blinkered software engineers who, for some reason, think they have a monopoly on understanding the customer as well as how to manage a software project the purpose of which is, after all, to fulfil an organisational / business need not to write code.
Geeky coders who, more than likely, are somewhat on the spectrum have no appreciation of the human factor here.
Much as I hate to kind of defend terrible managers, this is often a situation where even higher levels of management (who I am not defending) have configured things so that a developer who is very good and happy as a developer can only get a pay rise being promoted into management, where they are unhappy and make everyone else's lives a misery. A company that respects that some developers don't want to have to work in leadership positions and allows them to grow while staying in their existing role will often end up with better results.
You forget, they are no longer on the books as a developer:
"You're the least-highly-paid -- let's face it: grossly underpaid -- manager we have. To stop you ruining the curve for the rest of us, we have to fire you and promote another dev, but think of a different title without 'manager' in it "
> Geeky coders who, more than likely, are somewhat on the spectrum have no appreciation of the human factor here.
OTOH a really geeky coder on the spectrum knows full well what the human factors are, having built up and - here is the geeky part - analysed their own masking behaviours to gain an intellectual and not just instinctive understanding of them.
Someone like that can even explain to their team what is going on!
On the Gripping Hand, someone with that understanding can also become a truly Machiavellian Mastermind and before you know it the whole Dev Team is plotting to overthrow Marketing and consolidate their power before marching on the Swindon office. But that is just the way life goes sometimes. Onwards, for the Glory of The Project!
"it's the people who believe that adopting a new, preferably fashionable methodology, can by its mere presence fix what went wrong in the past".
My workplace is cursed with the belief that all the projects that failed in the past were because waterfall and moving to agile will magically fix it all. Nope, not shaking out that way at all - build your project on a swamp and your castle will still sink.
......is the "regular delivery of working software" thing....
After all, this probably means that the CUSTOMER for the software is forced into the "regular implementation of process change".
I'm sure there are customers out there with a taste for regular re-training and regular re-implementation.......but I've never met them!
I get the impression* that Agile came out of a start-up culture where you were starting from scratch, your users were often members of the public and visitors to your website, and making new features available over time feels quite a natural thing to do.
This is one reason that adopting it in large corporate situations can run into problems - once you are building a product designed to implement a whole workflow - while it only implements part of that workflow it's not really useful to anyone outside your QA team.
*I don't recall the exact details and don't care enough to research them.
Regular delivery of working software doesn't mean it has to be put into production immediately. It just means that particular feature or component can be ticked off and accepted (or rejected for rework) while the team moves onto the next one. It's also a quality measure - if the delivered software doesn't work then it's not meeting the requirement.
You'd typically wait until you have an MVP (buzzword alert) at the very least before putting into production and changing business processes - it's the customer's choice.
It doesn't necessarily mean updating what people use every month, but having more stuff for them to look at every month. For some types of software, this can produce good results. I've worked on these before and kind of have one now. I make a version, send it to some users, and they tell me what they like or dislike about it. It doesn't do everything because the project's not complete, but I have a list of things that do work and a list of recent changes so they can direct their attention. That lets me fix things more quickly, which means I also have time to deal with opinions that aren't necessarily bug reports. The software won't be put into production until it's all complete, but the intermediate versions show the progress and get user input.
If the software isn't as user-facing, this doesn't work. If there aren't any users willing to keep testing intermediate versions, it doesn't work. Sometimes, this is a great method for getting good stuff fast, which is why people have called for it, but they should, and not enough do, consider where it doesn't apply before trying to apply it to everything.
Interpreting this assertion as 2.68 times the rate of failure; and another bold assertion that 70% of IT projects are doomed to not meet requirements, we have (at least) two interpretations:
1. 70% * 268% = 18760 / 10000, so 187.6% of Agile projects are doomed to failure.
2. 30% off all projects are successful, thus (Success(Agile) + Success(Other)/2) = 30; so Other is 22% and Agile is 8%.
Neither of those interpretations is a point of pride; but given the unlikely scenario of 1, that an Agile Project is able to undermine 87% of a non-Agile Project, maybe [2] offers a somewhat realistic decoding.
The success rate of 22 vs 8 still isn't the whole story.
Projects have a cost.
What is the cost of success for Agile vs Other?
What is the cost of failure for each?
One of the promises of Agile was to weed out bad ideas early, hilariously parodied in the clever Silicon Valley series.
Killing a project early costs far less; especially if a less expensive brand of wizards is dispatched on it before Gandalf and Dumbledore get to start billing.
ps: I am not an Agile fan, other than the delight in watching former Sigma Six Blackbelts hovering over a wall of coloured post-it notes while trying to retain a shred of dignity.
Its too bad the manifesto didn't include dunce caps and funny noses.
I am afraid the study has been debunked as Tulip see here: https://www.linkedin.com/feed/update/urn:li:activity:7204841073804787714/
For people saying Agile does not work, then you need to abandon some of the solid Agile Engineering Practices created by Agile Manifesto authors. Such as TDD, Pair Programming, Technical Debt, and so on.
To be clear, Agile is not a methodology; it is a Mindset described by 4 values, 12 principles and an unlimited number of practices.
I notice many people do not read. They do not read the Manifesto. Otherwise, they would know that Planning is listed as a must-have in the Agile Manifesto for software development. If there is no Planning, then it's not Agile, simples!
"For people saying Agile does not work, then you need to abandon some of the solid Agile Engineering Practices created by Agile Manifesto authors. Such as TDD, Pair Programming, Technical Debt, and so on."
No, I don't, and you already know it. For one thing, just because I reject a philosophy doesn't mean that I have to oppose every thing a supporter ever said. That would make for some very odd worlds (let's see, I don't agree with religion X, religion X says murder is wrong, therefore I must become a murderer). Fortunately for me, as I don't want to murder people, that's not how anything works.
Your examples are bad as well. People have been pair programming longer than the Agile Manifesto has existed. Not to mention that it has many limitations and is another one of those things where, if you do it all the time when you don't need to, you are decreasing productivity and irritating the programmers. Technical debt is not a practice, it's a term, and it will exist no matter what you do. You can keep using those words for the concept without adopting the rest of it. As for TDD (test-driven development in case the acronym isn't known), it's not original to Agile either, can easily be used without it, and has been blamed, sometimes unfairly, for quality issues as it means fewer or no people specifically testing rather than developing. That is one which, given the number of times I've seen it done badly, I wouldn't mind abandoning or overhauling.
Requirements are not meant to change every two weeks
Years of estimating yet no real world evidence of anything getting done faster
"In order to reduce process we add more process" - statement dreamed up by the utterly deranged
Poker is a game, not a planning tool
T-shirt size estimation belongs in the clothing store, not software development
"Points represent complexity, not time" yet you keep on measuring how many points fit in a week?
People who want to "burn down" belong in jail, not run software teams
Same goes for people doing grooming
This is real scrum done by real scrum masters:
start: 50 points
Done: 70 points
Remaining: 100 points
They have played us for absolute fools
Agile is a byword for "winging it".
Personally I think the reason it's gone as far as it has comes down to one very simple thing: Nobody knows what they want. Except the ability to change their minds at any point in the rare event they commit to something.
In a traditional delivery model you might spend 12 months writing a spec for some software before going into another 12 months for development. It's quite possible that even after 6 months some changes in the world have occurred which means the initial idea is no longer viable. But this is where one of the biggest mis-conceptions of non-agile methodologies also lies: You can make changes providing you have sound processes for that! There's some bizarre mindset that unless you're using agile everything is set in stone and completely closed off to change.
The agile process basically says don't spend any time speccing out what's needed. Just get straight into it, and then make changes as and when needed along the way. It's all about the MVP - the minimum viable product - and doing whatever is needed to get to that point. Then iterating to improve that.
But here's the thing, that isn't much different to speccing out a project, making changes as you go, and then doing further releases.
The only real difference (buzzwords and bullshit aside) is you don't spend the time upfront planning anything. You just dive straight in and then consider yourself to be better off because you get that extra time for delivery.
I've seen projects delivered via agile where the end result was total and utter shite. But then, I've also seen other projects that weren't delivered via agile that were the same.
Overall what matters most is the attitudes of the people who are delivering work. Not whatever buzzword methodology they choose to employ. You delivered something great and everyone was still friends at the end? Well done. Please stick to whatever you're using!
I've worked a lot with Waterfall and Agile (among other methodologies like "code and fix"). IMHO Agile is (way) better than Waterfall. The problem is that today Agile is like a trend (or a cult) where everyone wants to jump in without being prepared for it:
- Projects start as waterfall (because it easy to set an initial price, especially for old bussiness people), but customers are not sure about the initial requirements plus they wants an agile development, so companies end up with a mix of problems.
- Project managers acting as scrum masters pushing their own agenda in the meetings (ignoring problems because "we HAVE to deliver value" at the end of each sprint).
- Agile develoment group(s) inside a company, and with customers that are not aligned with agile times (they want a feature "now", so that feature is incorporated into the current sprint).
- Big development groups (12, 15, 20 members)
- Single development group having to split its efforts across multple customers (lot of time wasted in duplicated/triplicated/etc Agile ceremonies)
- Development groups with several juniors members without proper mentoring ("we have to deliver value, we don't have enough time to spend in your learning")
- Zero time in each sprint to tackle the accumulated technical debt, which in turn leads to horrible code in less than a year.
- Zero documentation: yes, "Working software over comprehensive documentation", but documentation (not in the proportions of Waterfall) is still needed.
Agile is almost 25 years old, ideally it solves plenty of problems from old methodologies, but IMO it is necessary to update it, because the software development landscape has changed a lot since 2000.
I've been at this almost 50 years.
I've seen methodologies come and go.
I've seen arrogant programmers and designers and architects with virtually no experience or ability spend millions on projects that a competent all-in-one old-fashioned "expert" could have done in 10% of the time/cost... at most.
I've seen managers/manageresses put their "careers" ahead of the company they work for and stab the people who get in their way in the back because they don't get their big project with the latest fashion.
The only thing I've ever seen work is a group of people who were thoroughly grounded in their field of expertise and who respected each other enough to listen and who had a manager who protected them from the idiocy of the higher-ups and the latest fads.
Fads like Agile cost money... it cost the last place I saw it 11x the cost of the original plan and what was delivered wasn't fit for purpose. And the auditors said so. And the place is now stuck with a legacy system that needs 25x the number of people to support it than the original plan needed (in other words, from <10 to >100).
Millions thrown away on the egos involved and at the altar of the latest fad which promises to replace good people but which fails... like all the others.
I've worked at lots of places that claim to be agile, all but one of them (where I was CTO) had nothing to do with agile.
I've skim read the objections in the comments and they don't represent the values and principles in the agile manifesto.
I really do wish people would read it.
https://agilemanifesto.org/
In my experience agile is far and away the best way to create software.
Here is a good talk from Allen Holub on the sort of 'agile' described by the complaints in the comments above.
https://www.youtube.com/watch?v=HZyRQ8Uhhmk
"I've skim read the objections in the comments and they don't represent the values and principles in the agile manifesto. I really do wish people would read it."
A lot of us have read it. There are two problems related to it which mean that arguments such as the ones you've made are not convincing:
1. People follow the manifesto in a lot of different ways. They take a lot of the principles to extremes, even when they cause problems. They read "working software over documentation" as "working software, and documentation only if there is spare time", and then apply the continuous development principles in such a way that there almost never is any spare time.
2. The manifesto doesn't prevent them from doing so. I can't point to one of the principles that explains that they've misinterpreted it. I can't prove in any way that their misapplication is not what the Agile creators intended. I can prove that their misapplication is causing problems, but they don't care.
In the last topic on this issue, a couple people started responding to any criticism with "if it didn't work, it wasn't Agile", and this almost religious attitude puts people off because it can be quite clear that sometimes, it doesn't work even though it was Agile, and sometimes, it doesn't work because it is Agile. In my comments, I have tried to point out the benefits of Agile, when it is a good plan, and when it needs alteration, but when people have seen supporters who are unwilling to accept any criticism, blame any failures on others, and adhere to a very short and mostly content-free manifesto as if it answers questions that it either doesn't consider or answers badly, it shouldn't be surprising that they react with hostility. This is especially true if one of those adherents was their boss and applied it, correctly or not, in a destructive way. The manifesto is not blameless in this, nor is the boss, and both will have to be fixed if you want to see attitudes changed.
engineer.
My M301(a dread module from the open university) lecturer told us in his first lecture "Software engineering is no different to any other branch of engineering"
As in fix the main specifications before even attempting to make pretty stuff, while its all fun and games to build something quick to impress the customer and for that customer to give feedback, its no good if the foundations are built on sand, and indeed , if you get 1/2 way through building a brick arch viaduct and the customer suddenly goes ' I want a cable stay bridge"
But its important to note that while one style of software creation may need the waterfall method (Aircraft control software for example) Agile may fit another software project better than the waterfall(not that I believe it... not an agile fan)
Although we use both methods depending on the situation, a formal project to make 1000 widgets a day gets the full on waterfall treatment of planning, programming , installation, testing. other times when a customer rings up and says "I need 10 widgets double quick" the job gets thrown out, a template program is loaded, edited and the job is done and thrown at the customer(who then either starts bitching about the price, or asks for another 1000... per day)
What if he was wrong? Here's some differences I can think of:
Software is elastic - through the wonder of digital copies it costs nearly the same to build (as in compile etc) a billion as one.
Software can be delivered immediately, anywhere in the world (with internet) at any time.
Software is easy(ish) to revise (even completely) after it has been delivered - compare that to a bridge or skyscraper.
In software the challenge is designing the first one, in traditional engineering much of the challenge is building the design.
In traditional engineering designs are typically very conservative and the costs of mistakes is high. In software you can experiment for little cost.
And so on.
I fail to understand your points above.
"Software is elastic"? So what? If you copy a poorly designed, bug-ridden piece of software a billion times, you now have a billion copies of a poorly designed, bug-ridden piece of software.
"Software can be delivered immediately"? If you're trying to use that as an excuse for shipping bug-ridden software ("hey, we can immediate deliver a fix") then it's a poor reason.
"Software is easy(ish) to revise (even completely)"? I disagree. I've seen many fundamental flaws in software that go right back to original design decisions and which to "fix" would basically mean starting again. I've seen small pieces of software developed in a few weeks require what would appear to be a minor bug fix which takes a significant percentage of the original development time to retrofit.
The costs of mistakes in software can be enormous. If you've got a large development team and pick up a fundamental flaw in the latter stages of development, the cost can be huge to fix. If it's discovered post-deployment then even more so (see Horizon). Ditto for "In software you can experiment for little cost".
Software Engineering is still engineering. May not be the same as other forms of engineering but then mechanical engineering isn't the same as electrical engineering. In general terms, if you're building a large, time-consuming, complex "thing" then the fundamentals of engineering apply.
I know of no management framework that encompasses the values and principles of the Agile Manifesto for Software Development. If we're talking about software development, then this is the ONLY definition acceptable for "Agile." (That's upper-case-A Agile.) The management frameworks like Scrum and SAFe and many others are straw men that have defrauded consumers by stealing the name before substituting a 120-year-old production efficiency process popularized by an American mechanical engineer working in manufacturing industries in the 1890s. This mechanical engineer, Fredrick Winslow Taylor, created "Scientific Management," or Taylorism.
Software development never entered Fredrick Winslow Taylor's mind.
But managers love the idea of using Taylorism (observations, metrics, empiricism, and standardization) for software development. Taylorism agrees with their most fundamental and underlying philosophy that managers don't need to know anything about a process in order to manage it. Taylorism, as it became known, fell from popularity by the 1930s, although the Academy of Management voted Taylorism and Taylor's book as one of the most influential books about management in the 20th century. (I think we have a disagreement between managers and software developers that goes much deeper than just the purloined "Agile" moniker.
Why do I mention Taylorism? Because Taylorism is Scrum; Scrum is Taylorism. Scrum is what everyone today seems to call "agile."
If you call Scrum or Taylorism or anything that looks like it "Agile," then you are deceiving both yourselves and your audience.
In defense of Scrum, I will never say I've seen "Agile" fail because it was done wrong. I will say that I've seen Scrum fail because it is not "Agile."
Words have meanings. Names are very special words that have intense meaning. Words have meanings. Names have souls.
Be especially careful when you use a name but don't mean the thing the name implies.
As my grandfather said, "You can call a bull a cow, but it's not going to go well at milking time, even if you brought your own stool."
I'm a retired electrical engineer/IT consultant who has worked on both big (tens of millions) and small software development projects. I've seen only one "Agile" project, quite a long time ago, and only from the outside - it was at a company I was doing some consulting for in a completely different area, while they initiated a project to replace their financial system. They had a relatively new CIO, who was full of buzzwords, but as far as I could see with software development he didn't know his elbow from his arse. But great on self-promotion! An external software development firm was contracted to do the development, and they trumpeted Agile as the way to go. But the contract was essentially cost-plus, and progress was measured by how much money was spent, rather than any actual results. And I was amazed to hear one of the customer's SMEs say at an early stage that one of the dev firm's lead developers had mentioned that he'd only finished his Scrum and Sprint training a week before starting on the project!
The CIO must have been smart, for despite being clueless about the details of software development he knew enough to know about impending disaster and so left for an even higher paying job elsewhere midway through the project. Meanwhile the costs ballooned, but the contract had no enforceable performance clauses to ensure that the customer got anything that worked. So eventually the project was abandoned and a few senior staff were fired. What a scam!
For all I know Agile could be a good development methodology, but I have a very jaundiced view of it after what I've seen, at least in the hands of snake oil salesmen.
Over on PMI somebody proposed the idea of daily “standups” where the whole gang comes together in a circle and each participant intones 1-what they did yesterday 2-what’s their roadblock and 3-what’s happening tomorrow. “Nobody owns it,” sez the PMP, “it’s great, you blow 15 minutes but now everybody knows the score.”
Harrumph. I worked in that environment and it’s a sweatshop tactic. The supervisor owns the standup, matter-of-fact, and scheduled it every single morning at 7 sharp (so nobody’s ever tardy, you see). If you’re the buddy of the supervisor you get lots of easy tasks that all take moments to do, so every standup you can rattle off a dozen accomplishments, no roadblocks, nothing but win. If you’re not the buddy of the supervisor, you get long difficult tasks that each take several days to do and have substantial roadblocks, sometimes (shocker!) not even being possible to achieve at all. When you mention the roadblocks then it’s public humiliation time, did I mention the supervisor is a psychopathic task-master? If you got too many troubles he’s liable to pile on a whole lot more & now everything is due tomorrow at 7 am, how do you like that. Can’t cut it, you’re fired (leastwise, that was my personal experience).
Anyway, then if you share this feedback on PMI, you know, “criticize” this extraordinarily lazy presentation this PMP slapped together, it’s “very concerning” to PMI “and how soon can we schedule a telephone conversation so we can discuss” your membership in our privately owned cult organization which firmly worships all agile methodology (and kinda doesn’t tolerate any “conflicting evidence”).
I do have half a mind to take them up on their friendly little offer.. !
According to wikipedia, "waterfall" was first described in a 1970 paper, not by name, but by description, of a process that was manifestly flawed in ways that the paper examined.
That is, it was a straw man. No-one was ever expected to do it that. It was never a methodology. It was an anti-pattern 25 years before the term was invented.
Better than waterfall is not something to be proud of. It just means better than something designed to be crap in as many interesting ways as possible.
It all sounds so nice: just follow the principles by the book and success will follow - alas it simply not true.
All groups of people need leadership and if it is a proper leadership the group will be successful and if not - well, there's the failure people were responding to.
And yes leadership does not suffice in telling people what to do ...
Based on seeing 25 years of these train wrecks up close. They rarely ship. And when they do the product almost always sucks. More so than the usual run of the mill product dev train wrecks. Plenty of those out there.
Confused by all the references here to "waterfall". Apart from Government Work that is so 1970's. Every dev team not doing Government Work I've seen successfully shipping a product that did not "suck donkey b*lls" - to use the technical terminology - used some form of "spiral". Do an alpha / beta release and loop back for next iteration. Kicking all the stupid features / management "must haves" down the road while all key features are implemented. Then at some stage when its good enough its kicked out the door. When hopefully the users dont curse your name too often or appear at the office building front door with burning torches screaming - "Vengeance will be ours.."
And that's how you actually ship software that does n't (usually) suck.
Risks; those known unknowns, are not liked within corporate environments. This led to watefall as you do not pass go until you know you are safe to do so: no risks, predictable outcomes, please proceed.
Agile is touted as having lots of benefits but with it does come a need to manage (and sometimes simply accept) risks - which does not mix the aversion that the waterfall culture has imbued.
So you end up in this WAgile world - sadly with the worst of both rather than the best of them.
Mix that up with people not being able to differentiate value from cost and Agile just becomes FRAGILE!
Arguing about 'Agile' is the same as arguing about how many angels can dance on the head of a pin but here goes.
The following comments are with reference to the front page and principles of 'Agile' which read suspiciously like a piece promoting The Law Of Attraction or any other self-help snakeoil.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2025
