Agile Development’s Secret Sauce

One criticism levelled at Agile methodologies for software development is that of scale. “It’s all very well for very small projects,” we have been told, “but for anything above 10 people, you can forget it.” So, how true is this? We took this question to the Reg reader panel, and the results make for interesting reading …

[Paul]

Of course

I don't see what's so surprising about all this. Most of the issues people cite as problems with Agile will occur in any method (no ology, it isn't a study).

Things like bad communication, or a poor programme overview, or lack of vision, or crap architecture will kill any project, irrespective of how you go about it.

The difference is that other techniques tend to produce lots of paperwork that makes it look like they are progressing (look Ma, binders!) while all the time the project is going nowhere.

Failure in Agile development tends to expose those problems faster while expending less effort. You can't hide a lack of communication in an Agile team by writing lots of reports or architecture docs or just working the processes.

Something to remember: in many ways Agile is rather like Lean Engineering (the Toyota process) but for software, and Toyota manage to run huge projects using many similar principals.

Paul

[Warhelmet]

Agile vs Lean

@Paul.

Lean Engineering? No, it's Lean Manufacturing. It's the Toyota PRODUCTION System. Tachii Ohno would be turning in his grave.

I have a very jaundiced view of Agile Development because people so frequently q.v. Sick Sigma. Sick Sigma is the revenge of the toolheads and whilst it has had success in very narrow domains, in a broader context it has been a failure. And hey, don't you just need a whole bunch of new tools to do Agile Development? Not, but that's how it's sold.

When Agile Development toolheads q.v. Lean it's because they don't really understand some of the basic premises of Lean. Lean is NOT a toolset, it's a philosophy. But you see it sold as tools.

On top of this, there have been suggestions that what Toyota does is something that has grown up organically and is imprinted in the cultural DNA of Toyota. Whilst other organisation can learn lessons for Toyota, they can not duplicate what Toyota has done.

Toyota, on the project front, works very differently from many western organisations. My understanding (from people who have worked with them) is that they take an awful long time to make decisions, because they look at evidence and try to build consensus. When they make a decision, they act very quickly. This is very different from western styles of management - especially project management - where decisions are often made on a whim, without consultation and not implemented until ages after the decision was made.

Also, I will mention "genchi genbutsu" - literally "go see it for yourself". Having been on the sharp end of being a user of a project output - did the project manager come and see what I did? No. Did any of the developers come and see what I did? No. They just worked to some crappy spec knocked up by an external consultant, who only saw me for two minutes and wanted to talk to senior managers wherear muggins actually did the worm. And I work for an organisation that has, over time done i) Sick Sigma ii) Lean and iii) Agile Development. And BSI stands for "Bull Shit Institute" not "British Standard Institute".

One of the ways that Toyota has achieved it's famous levels of quality is by standardisation of tasks. I've seen arguments that this is not applicable in a service context. It may well be applicable to development if you wish to productionalise development. There are some strong arguments for this, but it rubs many developers up the wrong way.

The arrogance of some developers is frightening. Prima donnas with Aspergers.

Something else that Toyota does is develop it's own talent from within. And promotion takes time. Very different to western thinking that sometimes treat developers as guns-for-hire. Oh, formal methods over creavity as well.

Worse still, I've seen rogue project managers not being brought to book for their failures. They are not asked to "go to 'gemba'". Implement, end of story. It's some else's job to shovel up the sh!t they've left behind. When was the last time you went to a project wash-up meeting? When was the last time that you came across anyone who had a "lessons learnt" database and actually did anything with it.

Project managers with Teflon coating.

Oh, and it's worth pointing out that I have worked in engineering and project management of big engineering projects. I lived through the first wave of ISO 9000. I was there when Prince was rolled out. I'm married to a Prince 2 master practitioner who has done huge building projects. Etc, etc... It's not as if I don't know what I'm talking about.

Do your culture change first, otherwise whatever you think Agile Development is, will wither on the vine.

[Jodo Kast]

It can be done with as many as required

... to get the work done.

But, you must have the right stack in place... a business intelligence layer works perfectly for most application development.

During development, the DLL might return fake data... meanwhile the DBAs are busy working on providing accurate data when a SP is returned. The front-end folks don't care, they just see the data as it comes from the DLL.

Seems like any problem during the SDLC would be someone that's just not doing their job.

[CTG]

Tools

The three most important tools for Agile development (IMHO) are:

1) Printing whiteboards

2) A good chat/IM system that all developers, testers, support and users* are on

3) A wiki for project documentation

Anything else is just fluff. Even a wiki can be dangerous if you don't control the edits, but it is the best way I have seen for storing requirements, design and testing materials. At any point of time, the developers, testers and users need to be able to see what the current state of requirements is, and what the latest build has implemented.

* by users, I mean the specific user "ambassadors" that represent the user base. No users on the project == FAIL.

[Adrian Walker]

Agile Collaboration in Executable English

Excellent article.

There's another agile collaboration tool online. It's a kind of Wiki for collaboratively writing and running content as business rules in executable English.

Results from running the rules are explained, in hypertexted English, at the business level.

It's at www.reengineeringllc.com, and shared use is free.

[John Werner]

re: Tools and other stuff

> 3) a wiki for project documentation

We found that a forum was worked better. The wiki is good in that every one can change it, but without specifically looking for something, you can't always find the "why" of a documented decision. With a forum, you get the history (and discussion) right in there.

Either way, wiki or forum or both, the key is creating a system where ideas and documentation can be kept evergreen and where other people can see what is going on.

There is saying a "You can't get something different unless you do something different." The hard part is getting people to do something different, like using a wiki for forum. I can't tell you how many times I have seen developers, even 2 years after project start, sending around e-mail discussions with 10 people cc'd in. This is exactly where a forum should be used. If there really are 10 people interested or impacted in the decision, then there probably are 10 more people who will be secondarily impacted.

Back to agile scalability. There is no perfect process. Every process has good and bad qualities and good and bad application domains. The real trick is learning what parts of a process make sense in your domain, and being willing to adjust things as needed. Maybe part of the team works well doing pair programming, but others don't. Fine, enable both groups to work to their best potential.

The key really comes down to how you manage development teams. May the question shouldn't be "Can Agile Scale?" but, can your organization develop a process that is adapatable, supports the individual developers needs, and maintains predictability. The further question to ask, is "Is management interested in processes or enabling engineers to do their job?" If it is the first, success is going to be a hard road. If it is the latter, success should be easy.

- John