Study backer: Catastrophic takes on Agile overemphasize new features You can have your software fast or in a state where it won't blow up in your face. But getting both at the same time in an era of layoffs and restructuring is, at best, challenging and, at worst, impossible. So says Dr Junade Ali, who spoke to The Register in the wake of the furor generated following a study that found that …
What users want is the wrong question though.
End-users do sometimes demand new features. Especially in immature systems. As stuff gets better, they stop caring about the new stuff and just want it to stay the bloody same, so they don't have to re-learn how to use it after every update.
But in the mondern IT sector, end-users aren't always paying. The IT companies have now got so big and so rich, that they've stopped caring about what the end-user wants and started thinking about how they can monetise their data or sell adverts to companies and worm them into the user-experience. Or they create updates so they can link other products they provide and sell those as well.
Even worse if the end-user isn't buying the software but is on a subscription model or getting it free. Because then they can go full agile and change the user interface every month (I'm definitely looking at you Office 365).
Ask not, "what can I do for my users?" Ask, "what can my users do for me?"
...adults would much rather software worked properly than was chock full of new features. This should be thumped in to every student of software engineering from day 1 !
Along with how to define work properly. Without something more than a fuzzy, hand wavy list of "features" how could anyone demonstrate whether a body of software "worked properly" or not. As it stands I suspect it falls upon the poor sods who configure the software's test suites to define this (ex post facto as it were.;)
I would also guess that for any large application the vast majority of features (existing or new) are never used by most of the user base. Add the volatility of the user interface (especially in the Windows environment but Linux isn't immune) I can easily understand why the average computer user is completely underwhelmed by the industry's offerings.
For the life of me, I cannot imagine the great unwashed needing more than 5% of the features of the Windows (or LibreOffice) word processor - MacWrite would probably cut it even today. If you have observed Excel etc in the hands of the polloi - it's truly scary.
Humungous, incomprehensible bug riddled software in the hands of naive users... what could go wrong?
Except when you find that users actually do want new features. There are many features that get developed for a set that's so small that you might as well ignore them. However, many customers do use features to decide what to buy and complain if something doesn't have them.
Everyone has a level at which they don't really want many new features. If they were very thorough, they'd name four new ones (one that could work for other people, three very specific only useful for them) that they would like added. Everything else that's added just annoys them. However, it takes a product a while to get to that level, and if it hasn't gotten there already, they want the new features and will choose the option to use based on them. As with everything else, you have to manually determine where your product is in this cycle or it will not work.
The thing is - everyone says what they want is a simple program that "just works". But when you actually try to drill down to what that means, they're not all talking about the same thing.
Consider a word processor. 90% of users are happy to let it correct their spelling mistakes, and replace quotation marks with "smart quotes", and the rest of the autocorrect nonsense. But 10% will froth at the mouth and demand the right to customise those features, or turn them off entirely. So you have to give people a way to do that. Boom, a feature that 90% of your customers don't want.
Then there's the ability to integrate graphs from a spreadsheet in your documents. Let's say another 10% of your customers are obsessive about that. And another 10% are really, really passionate about having a word count that's displayed on screen at all times. And another 10% care about structuring long documents with consistent styles and formatting, while maybe 10% want to just make things look pretty as they go along. One user wants to type in a web address and have a hyperlink added automagically to the document, another collapses in horror at the very thought, muttering darkly about security and document portability.
How are you going to make a single product that satisfies all those people? Options. Lots and lots and lots of customisability options, most of which are at best irritating distractions to most people. And that's how we got Microsoft Word. Ask ten people you know what features of Word they hate and would like to dispense with, and I bet you get ten quite different answers.
When someone comes up with a survey and says "What do you want out of your software?", it's not hard to come up with answers. But if someone says "now we're developing this new software, if you pay us $1000 you get to specify a feature we'll include" - then your answer would be quite different. People's "expressed preferences" are not the same as their actual buying decisions, and it's the latter that matter.
It is true. The problem is that "works properly" and "has the bare minimum features to satisfy requirements" are two concepts that can vary wildly from user to user.
Not coincidentally, this is a large part of why I can make a living building bespoke software as a self-employed single developer, outcompete much larger businesses, and have a string of very happy customers - the application I make is going to work perfectly for you, and nobody else. Much, much easier.
Theres a little difference in being considered adult in the field of biology, jurisdiction and lastly and most importantly behaviour...
Seen a 12 year old in an MMO acting a lot more mature than his 60+ year old guild leader. Non internet-afflicted experiences also give this impression. It would be bearable if this issue was limiteed to games and marketing but weve got quite a lot biological and juristical adults in the bundestag who are even less mature than a kindergarten brat that throws a fit every three seconds.
It's the lazy developer's getout - can't get it done in the sprint, despite confirming the story points - just shove it on the backlog with no indication of how important the unfinished story is.
Consider developing a story taking CC payment - testing against the bank obviously infeasible in early dev so you stub it, responding appropriately to the various test cards. The story replacing that stub never gets defined clearly enough to make it obvious in backlog prioritisation that it must be done - so it stays on the backlog.
The end result - sprint prior to go live you get to testing with the bank representatives on site, works with your test cards, they use their cards and ....
Wide ranging government systems transformation, multiple web front ends delivered - very visible, integration with backends deemed too difficult/too long to deliver so front ends just delivered pdf's that were fed into existing processes. Still a benefit in that it eliminated a lot of human error in filling in forms. However front ends had to be limited to the 80% - the simple cases - because it's alll about delivering benefit early and not creating a wide, deep backlog.
Projects now stall - the complex cases cannot be configured into the front ends and the data available won't support backend integration. Basically lots of money spent on delivering pdf generators - which are effectively additional legacy systems.
As the article hints - agile addresses the wrong problem - giving developers free reign is fine for something small enough for a small team and products that don't really matter. If the aim is a system that actually replaces legacy, has to integrate with legacy and needs to just run - agile should be considered harmful. (That's the bulk of enterprise systems out there by the way.)
Product owners simply can't work through large backlogs at each sprint planning session to revisit prioritisation - it's too easy for stuff that's 'incomplete' to be pushed further and further down. In reality product owners have a day job and never spend enough time with the sprint teams - even those that do lose contact with the business.
plus the incomplete story is often not described clearly enough for the product owner(s) to realise it's important ....
'replace test stubs on CC integration' doesn't ring alarm bells loudly enough.
You have the clash of devs who've drank the manifesto cool-aid being unwilling to spend the time to create a clear story to define what they haven't done and a 'business' product owner not understanding the subtleties of devspeak.
In most systems, including basically everywhere I've worked, devs don't get to put stuff on the backlog, and that's stuff that's unassigned. If I took something I had been assigned, didn't finish it, and put the tasks on the backlog to disguise this, not only would I be doing something I'm really not supposed to do, but they're likely to find out quickly and be very unhappy with me. Nobody does that.
Backlogs can be a problem. They fill up with stuff that's never going to get done, and if the project manager or enough devs don't want to do something, it can get shunted into the backlog and nobody will ever see it again. However, it is not the tool for devs who don't intend to do what has already been assigned.
If Product owners don't revisit prioritisation then the priority remains the same. Incomplete stories can't be closed, so they will remain open and in the same position on the backlog.
Product owners own the backlog, they decide what stories go on the backlog and their priority. Developers armed only with a deck of cards take the stories on the backlog and starting from the top assign story points to them. Commonly there's a sprint (sprint 0) dedicated to assigning story points to as many stories as possible.
What may also happen when story pointing is a story is broken down into the various tasks need to complete the story, though the team may choose to do this later when they bring the story into a sprint. But whenever this happens these tasks are not stories they do not get added to the backlog as such. They should remain as part of the story, and until all the tasks are closed development can not close the story.
Product owners prioritise the stories, not the tasks that are steps development decides need to be done to complete the story. 'replace test stubs on CC integration' is not a story and should not be on the backlog. It is a development task that's required to be done before development can close the story and as such should be a task under the story.
Devs do not add stories to the backlog Product owners do. I have certainly not "drank the manifesto cool-aid".
Carrying incomplete sprints forward like this is a bit like an Olympic event which starts as the 100m , the becomes the 200m , the 400m , 4 x 400m etc . If it was just one event this may be manageable but often the whole project starts operating like this
Unfortunately, this is almost inevitable.
Software is not usually commissioned by the people who're going to use it (indeed it's often imposed on staff as a means of reducing their numbers), the requirements are often dictated by people whose knowledge of the way the business actually works is at best optimistic and at worst vestigial and there is often a reluctance to modify existing practices where they're a hindrance to effective automisation.
Above all, the process is usually confrontational rather than collaborative: the management make the (ill-defined) requirements, the software team are expected to implement them and the frontline staff are expected to make it work, despite the inevitable shortcomings in the results - which lead to much finger pointing and little practical resolution.
No-one would contemplate setting up an industrial production line without defining every step, the maintenance requirements, the capital and operational costs and the roles and skills of the people needed to keep it all going. Not to mention clearly defining the roles and responsibilities of external suppliers. Yet software development seems often to be treated like busking from a rough chord chart.
I generally agree with this, but I also see the problem a bit different.
Computers strictly follow logical processes, even the big AI constructs are, at their core, stricly logical processes. The same is true for factory machinary. The difference, however, is machinery has it's roots in engineering, from the start and, generally, businesses have stuck to that trend. The engineers are in the design from the beginning and often determine what the requirements are based off physiucal measurements, observation, expected outcomes, etc. While it has it's flaws, these engineers can typically operate with a good idea of what is needed from the design to fit the process.
However, Humans are not logical. We are capable of logic, but it is not necessary or required from us. So, when we try to integrate these logical computers into a human process, it becomes not so easy. One will quickly find most people have no idea why they do something or even what they are doing. Most folks just know they follow the process they have been taught. Try going up the chain and you will find the processes don't even matter that much, as all the managers are more engaged in keeping customers, higher ups, shareholders, etc in a particular emotional state, not necesarily in ensuring processes run smoothly and standards are being met. This leads to countless little compromises, fudging of some numbers, or just sort of skipping all process whatsoever, because the actual end result is not important nor does anyone actually care about it.
So, when you got to design a system, often there is no actual logical requirements at all. What they want you to design is a system that keeps the right people happy and uses fewer resources to do it (and even that last part is not particularly important anymore). Agile attempts to solve this problem through a sort of 'feel it out as you go' method. While that method is fine in theory, what it has turned into is a way to remove any accountability from anyone in a management position. It means managers don't have to try and come up with a reason why they do anything or risk showing their ignorance by trying to explain what is needed or why anything works the way it does. It also means no pesky people poking around and potentially finding out just how terribly run most things are (which might make them look bad). This means more projects can get off the ground rather than dying in the early phases of waterfall because noone can past the planning or requirements phase.
I am lucky enough to now be working on engineering-adjacent systems. I deal with people who are used to thinking logically in terms of "if this then that else the other". I have spent a long time in corporate IT though and that can be pretty painful. It's no fun being moaned at because you implemented precisely what they asked for.
Taking this a little more broadly, there are circumstances where asking the users results in the wrong answer.
You'd hope that when spending money developing software that you're pre-empting the problems you're going to have as well as the problems you currently have. This may require changes in process, removal of people from process, or even fundamentally removing a process and replacing it with a piece of logic.
Asking the users to define this is often met with the 'but that's not how it works' statement and they're right. And hence asking the turkeys about how great it will be after Christmas doesn't go so well.
So, users are important to an extent, so long as you're in the faster horses line of thinking. They're unlikely to be the ones specifying cars.
"Yet software development seems often to be treated like busking from a rough chord chart." - agree. The problems stem from Software Engineering not really being valued as a discipline any longer. It is a begrudged necessary expenditure whose budgets are to be squeezed at every opportunity and itself one of the original victims of "Agile Transformation." Development teams are made up of "full stack agile developers" where the CVs talk of all the many frameworks they have worked on in the 2 years of their total experience. There will generally be one member of the team deserving of the title "engineer" surrounded by inexperienced (but incredibly well paid) "hackers" whom with no investment in the project or the company and will happily move on when an offer to work on the "next greatest thing" framework or language gets talked up.....
Many were. Many fell down.
Until fairly recently in human history much of engineering and architecture was of the try-it-and-see variety. There's massive survivorship bias because we only now get to see the ones that worked. Hopefully these fields have learnt the hard way what is necessary for success.
Which is why a lot of Victorian structures which are still standing are still standing - their engineers went for the "calculate what we think is required for it to stay up, then double it and add 10%" method. Those that didn't were people like Sir Thomas Bouch.
On reflection I've remembered the box girder bridge saga. Along with those half joint bridges they were one of the go-to type of structure* of the UK motorway expansion. The consequence was the reduced lanes and speeds over the Tinsley viaduct on the M1 and a bridge across a river (possibly the Wye) on the M50 and the circuitous routes round the Warley interchange whilst they were being repaired. How could I have forgotten them?
Bridge engineers are not always a good exemplar of engineering excellence and hang the cost.
* I almost wrote mainstays which would have been unintentionally ironic.
There is certainly a view, and one that I think was prevalent amongst the proponents of Extreme Programming and, later, Agile, that software development is more about art than about engineering and that problems are best solved by providing an environment in which practitioners of the art have maximum opportunity for collaborative creativity. I think that partly stems from the origin of computing as a branch of mathematics that particularly revolved around the development of efficient algorithms by people who subsequently got to put their names on them.
Most contemporary software development, though, is really systems engineering: assembling complex systems from proven parts with the expectation of a determined behaviour. Which is pretty much what building a car involves. There may be scope for creativity and research at the margins, but the idea of the artisan coder is, I fear, a relic of a past era,
I used to ask similar question: Can any other branch of engineering afford to use anything like Agile methodology? Because they could not get away with murder, right? ... And then Boeing happened, first 737MAX, falling doors later, etc. So, the short answer is, nowadays apparently they do.
A while back I spent some time in aerospace and there were some parallels.
- in the initial transition from design to build - for the non-automated components you have your 'best' engineering fitters on the job - they effectively make the translation from design drawing to physical product.
The challenge was capturing what the fitters change in the build, (small things like rounded joins, reordering work steps etc) - traditionally they'd annotate the drawings and subsequent builds would use these annotated drawings, the issue being those annotations never made it back to the designers - the improvement feedback loop wasn't closed. So designers made the same "mistakes" in designing similar components. You also needed to cater for these fitters having custom jigs and "components under the bench" :)
(I saw numerous attempts to close the design engineering/production engineering loop - some more successful than others - I hope things have improved in the 30+ years since.
I've mentioned before an old colleague who's previously worked at Shorts in Belfast. The designer required a square hole in wood to take a steel rod. There was no indication as to how it was to be made; presumably the hole was too long for chisels or their mechanised equivalents. My friend waited until the designer had left, drilled a hole with the diameter of the side of the square, took the rod and a large hammer...
There are at least two phases in an engineering project, and the first few steps can be done using something similar to Agile. When researching what works and making initial designs, you can afford to try something that might work and see whether it does or not. Eventually, you have to stop doing that and decide on a good design for the product that's actually going to be sold. Continuing to use the informal methods when making the product will lead to lower quality. Not using a more experimental approach in R&D is likely to prevent the invention of new capabilities which would make the product better.
As with software, you have to know when it is safe or desirable to use a method and when it is not, then run those things separately. This doesn't work well with people who demand a single solution for everything. They demand that either Agile is a great system and can work with anything or Agile is bad at everything and there is a different one that isn't. Neither is true.
I once worked for a software company that also used it software to deliver engineering projects. The owner was a big agile fan and insisted we use it in engineering. It is however a little difficult to analyse a model when it does not exist, and whilst you can start making very large and expensive castings from a model without analysing it, the rework because it failed is both embarrassing and expensive.
You either end up with a sprint of just one thing as it has to happen first or a bunch of people sitting around waiting for the things that have to happen first to complete or at least be close enough to be useful.
So are dieting fads and homeopathic "miracle cures". And they are also sold in much the same manner, as there is an equally large market of gullible buyers for easy instant solutions to complex problems.
Software projects of anything more than trivial complexity are hard. They require leadership and developers with deep technical knowledge and experience to succeed. Thinking you can get there any other way is doomed to fail.
Yes. However, the real question is: did "Dr Junade Ali (pictured above below)" study botany, or ornithology? (judging by his ink, as detailed in this insightful documentary, broadcast around the world). (eh-eh-eh! Nice tats!)
This post has been deleted by its author
Pfffrrrrt! You're just jealous that my jealousy is so much bigger than yours! In fact, it is so big, that it's been dubbed the "Trump hands" of jealousy, noone has a bigger jealousy than me! It's bigger even than that of pr0n actors, whom I don't have sufficient funds to keep quiet in time of controversial need, unfortunately.
I know, jealousy is the 13th deadly sin, right up there with overindulging in the chocolatey goodness of m-&-m's, but I just can't help it! So, sue me if Junade Ali's cool tats remind me of that delicious documentary ... granted though that in the French (dubbed) version, they've swapped "I studied botany" for "I'm a florist", and cut-out the bit about "shears", ending instead with just "let's get to know each other" ... but, same filmed segment, just dubbed.
Makes me feel a bit cheap TBH, not worthy of a locally-produced commercial for such mouth-wateringly yummy treat that won't melt in my hands, but my jealousy remains big and strong, not sleepy, crazy, some kind of communist, or anything like that (eh-eh-eh)!
One distinction of so-called “professional” engineers is the requirement to learn and to observe some basic ethics. Including some fundamentals like “telling the truth” and “protecting the public safety” such as life & limb, health, property, etc.
These concepts are detrimental to the efforts of the “upwardly mobile”: those seeking influence and working on their persuasion skills. Inevitably you sabotage your career if you find yourself shackled with ethical principles. You can’t propel your own self upwards without exerting an equal/opposite influence on your immediate environment (which you are leaving imminently with your freshly-gained momentum). The trick is to step hard on solid footing, if the structure is too rickety or rotten you’ll crash right through!
(Actual engineers get trapped by ethics into building solid and sturdy infrastructure, which then gets scavenged and looted by hackers who just want to grab & steal & don’t care much about some place they were gonna run away from anyway.)
Hint: Agile has absolutely nothing to say about protecting life & limb, health and welfare, or preserving or securing property. Agile is “hacker” crap, just slap something together quick and cheap, cash the cheque, and get off that rickety platform as fast as humanly possible!
> Agile is “hacker” crap
Oi! Rude!
No decent hacker uses Agile.
Hacking is all about getting systems to do something unexpected and interesting - a Good Hack requires painstaking work, often with long hours of concentration and certainly attention to detail.
Even when "hacking something together in a hurry" you'll see (often very deep) expertise in the bits being glued together - and not a single User Story in sight, let alone a Kanban board!
> solid and sturdy infrastructure, which then gets scavenged and looted by hackers
Nah, they aren't hackers.
You can't do hardware with agile. E.g. a house or a network. That's because real life has dependencies and a definition of finished... Not a backlog you take in any order and refactoring for when you get the order wrong and declare a 'minimum viable product' that isn't.
Agile happened because projects got bigger than 3 people could handle and developers thought they were better than project managers (hint... everybody is the same, just with different work to do.)
Process and teamwork is still needed I'm afraid, not agile cherry picking and oneupmanship at the stand up.
> You can't do hardware with agile.
You can. You just can't do it repeatedly, cheaply or reliably. Nor to a fixed timescale. You end up with one-offs (which is not great economics).
For houses, go watch Grand Designs - some of those go totally off the rails, some are done meticulously, some are done in an Agile fashion like "Mr. Blandings Builds His Dream House". And some beautiful results are frankly hacked together, reacting to what has happened so far, modifying to follow the beauty (strangely enough, those tend to be made in wood, where you can decide to keep a weird projecting bit on a beam and then carve it into a beautiful shape that was never in the original outline).
The big difference between hardware and software is in the reproduction stage: it is totally irrelevant *how* you arrive at a working executable, reproducing it is always the same and is always cheap - just copy the files*. *Some* Agile teams have produced working software, and then been able to get lots of copies out there.
The problem is that not *all* Agile software teams managed to make the trick work. Especially those that were basically told "you are now Agile"...
BTW it is logically possible (but thoroughly impractical) to reproduce an Agile House build - just film every single stage, without missing any tiny little detail, and then replicate what that film shows, in a new location (sans all of the standup meetings etc, just do the "put something physically here" bits). But if you wanted a hundred copies, you'll end up converting the film into normal-type plans, with the steps rearranged for efficiency and then you'll see the reproductions going up and no sign left of the original "Agileness".
* Ok, some software seems to be impossible to reproduce, because the Devs seemed to have forgotten what files are required in the copy, e.g. the Windows Registry settings that actually work have to be dug out with a blunt spoon, but once you have actually found all the bits...
I used to think using house building as an analogy for software was a bit spurious.
Having now seen (bespoke) building work up close I think they're a lot more similar.
Architects make some wonderful, detailed plans which you very quickly find can't actually be built in practice, so the builder has to improvise around them.
Without some builder agility you can easily end up with light switches you can't find in the dark, underfloor heating which mostly keeps the bottom of the furniture warm and no way to take your bins out.
"Agile happened because projects got bigger than 3 people could handle and developers thought they were better than project managers"
Alternatively they discovered just how crap some project managers are. What provoked me into retiring was my last (as it turned out) client's PM. After spending the afternoon before he went on holiday havering as who was going to do what he changed his mind that evening and rung someone up to reverse his eventual (and to my mind correct) decision.
Ironically, before he was appointed the client tried to get me to turn permie to take the job. They didn't realise how close I was to their mandatory retirement age and I didn't tell them. I was more than 2 years over it when I quit.
"The problem lies in the quality of the product. Imagine a futuristic car, a sleek and beautifully designed machine inspired by Gernsback's vision. From the outside, it appears flawless, an immaculate vision of modern design. However, if you look under the hood, you'll find a chaotic mess of interconnected rods, wheels, pistons, and cogs that bug out for no obvious reason. Calling yourself software engineer doesn't make it so.
I retired 5 years ago, but I doubt that things have changed for the better. My experience was that Agile was an improvement over waterfall or whatever else companies used.
But I saw two major problems. First, teams seldom functioned as teams. The teams were deliberately constructed with a senior "leader" and junior people. The leader would call the shots while everyone else did grunt work. This led to the same problems we had pre-Agile: that one person could wreck a project.
The bigger problem was Management. They refused to allow any sort of autonomy, with product owners and scrum masters calling the shots, doing the majority of the talking, and making uninformed demands. Team members who raised concerns were ignored or worse. At higher levels, faceless managers would drive requirements that precluded any focus on understanding the problem, developing a conceptual solution, or creating quality solutions.
I have to agree here. If there's one thing that the manifesto actually does state clearly, it's that the type of mismanagement described is harmful. It doesn't say how to avoid it, but given how a lot of implementations work, I'm not sure it would matter much if it did since management would ignore it anyway. I've worked in places which were better at this, often by having a more informal team structure as Agile recommends. On that point, Agile has recommended the right thing. It should be obvious but sadly is not.
I will say I'm also fed up with modern software pushing what makes the most profit, not actually implementing what users want.
However, to say 'The public were most likely to agree that data security, data accuracy, and no serious bugs were the things which matter to them' is generally meaningless. It's true to some extent, but it does not sell software. Security, accuracy, no bugs, and backup are the minimum standard for software, they do not confer any advantage with most users - they expect that for zero additional cost.
If you want to make sales it's more important to meet customer requirements than to have a perfect system. If you can offer broad or bespoke functionality at a reasonable price you'll sell well, but bespoke functionality tends to lead to technical debt unless you're very careful or highly resourced.
I'd also note that 'Agile' is not the whole story. Yes, you might be able to deliver iterated functionality in a number of sprints, but Agile says absolutely nothing about long term support and serviceability - via decent logging, auditing, in-built support functionality etc. Delivering basic client facing features is only part of the story.
We know now that the massive outage was due to a "program" that needed 21 parameters and 20 were supplied, leading to a general protection fault. It wasn't tested as thoroughly as it should have been.
That doesn't mean the entire industry is completely coming apart and soapboxing the whole thing to point a quivering finger at "Agile" practice is due.
This happens because there are thousands of bored IT journalists and talking heads, not because there are some "fundamental" problems with how IT works
Not that we can't all do better, but this existential hand-wringing helps nobody.
not because there are some "fundamental" problems with how IT works
I would dissent. I believe there are fundamental problems.
I am guessing the only reason the Falcon sensor 20/21 problem actually surfaced was that the code was running in an environment (kernel) which cannot ignore such misfeasance. I imagine if it were running in user space it would have potentially raised a SEGV exception which the application coder would have chosen to ignore, or worse the uninitialised 21st argument contained a residual but valid albeit quite arbitrary address.
In thirty years time I wonder gow many digits will CVEs require - at the current rate at least a (cell) phone number and these are only security related lapses.
This speaks to a key issue in current IT incident management in poor functioning orgs - we talk about the technical factors but not the wider social systems (process, management, oversight, etc).
It’s like talking about the Titanic and discussing the rivets being too weak but not the behaviour of the Captain or the ignored iceberg warnings. Or discussing the Grenfell fire purely in terms of the fridge that set on fire but not the "stay put policy" in the event of fire or the regulatory failings in relation to cladding.
Yet we haven’t learned the same in software.
It may be heresy to say this (but perhaps no better a time given those who have tried to suppress discussion of Agile seem to have lost), but perhaps we need more people to think about the psychological factors behind failures.
“ We know now that the massive outage was due to a "program" that needed 21 parameters and 20 were supplied, leading to a general protection fault. It wasn't tested as thoroughly as it should have been.”
I would say the real problem wasnt the data that was supplied (my reading was that a default value was used for any missing values, so with the default added, there were 21 values). The real problem was that a configuration file reader was not designed to survive any possible input, legal or not, without crashing.
Agile is not a magic wand.
Do not start with user stories defining "requirements".
Start with user stories definining "needs". What is the problem are you going to solve? (the what?). Second: define product features that satisfy those needs (the how?). Keep it simple and stupid. Then, define requirements in order to implement and test those features.
I have seen many "requirement specifications" in my life. None of them was in the end either complete, correct (as in conflicting requirements) or able to describe expectations. Nobody was able to picture what the product/solution would look like, resulting in "but I was expecting it would work like ..."
Especially when windows wants to install an update...
I fear what hoops I will have to jump through to get back the functionality I need and turn off the new features I hate and don't want... all the while wondering when an update will be released that fixes the problems that plague me...
My mother * was so unhappy that her Windows PC would just update itself automatically whenever it liked, changing thing, that I persuaded her to get an iMac on the basis that it would not update if she didn't want it to. She kept the PC for internet things like email and online searches, and the Apple for her photography hobby, once commissioned was isolated from the internet.
*Now deceased
At a security conference in the USA sometime in the 1990's, the speaker was sure that we would rather have an ATC system that was available all the time, than right all the time. I disagree, if the ATC system is only available for 5 minutes of my 10 hour flight it must be accurate. The rest of the time, when the pilots know it is not available they can take other measures to avoid collisions, but a system that is available and not necessarily correct is terrifying in any life critical situation.
The discussion of 'Agile' reminded me of two books about British military operations and government projects*:
Norman Dixon's excellent 'On the psychology of military incompetence' (ISBN 978-0-712-65889-8) and Anthony King and Ivor Crewe's 'The Blunders of our Governments' (978-1-78074-405-6). Both show incompetence of truly staggering proportions (you will wonder how there was ever a British Empire with the likes of Redvers Buller in the army). Indeed, recently the UK politician Chris Grayling, who privatised the Probation Service disastrously, and after Brexit contracted a ferry company for emergency ferries, which did not have any ferries or experience of running a ferry and had many other expensive failures has been nominated for a life peerage**. He was even appointed to the cross party House of Commons committee reviewing security, and when he did not get the chair of that committee resigned (presumably in a huff).
*The theory applies equally to all military and governments, but the authors were British so chose examples they knew best. Find your own for your own country, if you dare ...
** For UK folks, to object, go to https://chng.it/VpWcqdnZZs or go to Change.org and search for Chris Grayling peerage
Any comparison of that nature is so vague that it can't really be used. For instance, ATC that is available for five minutes per ten hours is not going to be very useful even if it is right. What are the chances that it chooses to be available when you need it rather than being available for five minutes of routine flight. It also presumes what "not right" means. In reality, an ATC that is inaccurate probably isn't completely random or guaranteeing wrong answers but has some chance of error. If that error means that a flight makes two turns instead of one because they read out the wrong number, the result is very different than if it is told to taxi at a building the pilot cannot see.
In general, a system needs to be available and right quite frequently or it won't be useful. We then have to consider what harms are caused by unavailability and incorrectness of the type that actually happens to decide how bad those things are. Our goal is of course to have 100% correctness and 100% availability, but our tradeoffs have to be calibrated for the actual outcomes. Often, the incorrectness isn't something we can predict accurately because it is due to irregular events, especially human error, which is why we try to design around frequent causes of error rather than calculating probabilities on something unpredictable and planning with those unreliable results.
Yes, feature factories are usually a disaster, but this reads like another straw man, anti agile argument by people who don't know what being agile is.
Agile is just inspect and adapt. It works because software development is primarily about gaining knowledge rather than defining everything up front (at the point of least understanding). It doesn't stop you from writing reliable, secure code - just the opposite.
And on the subject of 'comprehensive documentation'. That is in the manifesto because software code IS documentation. Unlike stuff written in confluence by people spread all over the world in their second or third language, code is an unambiguous description of what the computer should do. If you employ the right people it's also human readable.
"And on the subject of 'comprehensive documentation'. That is in the manifesto because software code IS documentation."
No, it isn't. It never has been. It never will be. Code is a set of instructions for a computer, not a set of instructions for a human, and definitely not a set of instructions for a user. A user does not care how the program does something, and they do not want to and should not be expected to have to read through in order to figure out how to use the thing.
I've worked at places where that was tried. One team went as far as to put in their readme file (all ten lines of it) RTFC. The result: weeks of wasted time trying to figure out which parts of their library worked and how. Did I mention that one of the people wasting their time figuring this out was on the team that wrote it? Because they did not document, and incidentally did not check, anything properly, their code was impenetrable. To make it worse, their library called out to another library which was written the same way. This is why you have reference documentation. What this function does. What parameters it can take. Otherwise, you too will find yourself spending time trying things over and over again because all the invalid calls return empty sets, but some valid ones have zero results so also return empty sets. Now was this one a failure or success? If only someone had written that down.
And that's in a situation where everyone involved could do that. What Agile proponents often fail to recognize is that most of the people for which software is being written are not programmers. They cannot read the code. They don't want to either, but even if you made them, they would not be able to do it. Some of them don't have the code in the first place, since not everything is open source.
Those who deprecate requirements in the name of Agile are real and the consequences are real.
Hours before this article went live Alan Holub posted: “I keep hearing people talking bout not having vague requirements, but vague is at the heart of agility. You want things as vague as possible until the last responsible moment.”
https://x.com/allenholub/status/1821034221177336067
He’s since double downed: “Requirements, by definition, are limits. They remove choice. They eliminate creativity and destroy agility.”
https://x.com/allenholub/status/1821574628193824971
Martin Fowler, a co-author of the Agile Manifesto, has literally been on stage saying that the goal of creating the Agile Manifesto was to create a process “which doesn’t depend on the requirements being stable”. (You can find the video by looking up a talk “A Retake on the Agile Manifesto” - GOTO 2014)
The person who led Boeing’s Agile transformation was also critical of the fact that Boeing didn’t go further enough to reduce requirements (even after the 737 Max disaster). (Look up a talk “Boeing @ Agile” at “Better Value Sooner Safer Happier”)
And this is just scratching the surface of the cases of this.
It has been one of the consequences of software developers treating themselves as apart from the rest of the engineering world.
> people who don't know what being agile is
I know exactly what being agile is. And it certainly is not following a rigid process of assigning fairy story points and then claiming that you can't do anything else in the current "sprint". Anyone who thinks that the word "scrum" has anything whatsoever with agility has clearly never seen a rugby match. If that's you, here's how it goes: everybody pushes as hard as they can in opposite directions, generally going nowhere until it collapses in a heap, possibly entailing serious injury.
Agile, with a capital A, is not even slightly about agility. It's about managing the potential damage wreaked upon your business by employing a large number of mediocre (but interchangeable) developers instead of paying for (and retaining) some really good ones.
Death to the scrumbags!
-A.
"code is an unambiguous description of what the computer should do"
It's an unambiguous description of what the computer will do, not what it should do. They are not necessarily the same thing and it's from the gaps between that the bugs emerge and flourish.
I remember reading something a long time ago about comments in code that went along the lines of "journeymen programmers say what the code does, master craftsmen say why they did it this way, great programmers say why they didn't do it some other way.".
I always held that development was simply the process of launching a program into the maintenance cycle. Good documentation is going to be essential to the future maintainers. The code itself will never be enough.
The inevitable problem with "fail fast, fix fast" is that end-users, especially of the non-technical kind, spend half their lives in one of the "failing" states and wonder what they did wrong. Nobody has explained that "our software will regularly fail, just suck it up."
The less inevitable (evitable?) result often observed is "fail fast, fix one bug in too much of a hurry and introduce two more bugs".
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
