How does UK.gov fsck up IT projects? Let us count the ways A report from right-wing think tank Freer has estimated failed government projects in the last few years have created delays totalling 34 years and wasted an eye-watering £7.5bn. This was achieved by just nine projects examined by one Parliamentary committee, it said. The paper was written by the think tank's co-chair, Lee …
So, have I got this right? Government is bad at managing outsourced projects, and current bureaucratic oversight is ineffective, so what we will do is outsource the outsourced project's project management and set up a new committee for more buck passing oversight? What can go wrong?
Worse than that,
Firstly it suggested making a permanent secretary fully accountable for delivering projects.
After I stopped laughing my backside off I realised that these so-called ‘informed’ twats are actually serious: they really do think that they can find a senior civil servant to accept responsibility for government wasted spending?
Excuse me while I go out to feed the pigs as they have a long flight tomorrow.
So one of ~2 million email accounts (my guess, I know it was north of 1.5million 10 years ago) was compromised and being used for spamming?
Or was it an NHS system that had relay rights within NHS mail?
Details are important here to judge the effectiveness of the solution.
It's only as secure as the users using it.
If people can't remember their passwords and feel the need to write them down on post it notes or if any of the 2+ users using a shared mailbox falls for a phishing scam I'm not sure how the email system itself can be held to account.
With some many clients accessing it using Windows XP, then you're security posture is compromised to start with. It would be naive to imagine that you'd be thanked for securing an environment in such a way that users were unable to access it via a fat client (I know webmail is available, but try telling that to a 80 year old Consultant who weilds more power than you could imagine in the organisation).
"UK E-envoy Announces Radical Shake-up
In a revolutionary move, the UK Government’s new e-envoy Mr Andrew Pinder, is about to announce the result of a huge two and a half week study into past and future I.T. projects and spending plans. Insider sources, close to ministers, who carried out the study at Neasden Polytechnic (London UK), are believed to have made some remarkable discoveries.
They found that, as predicted by Fred Brooks in his seminal work “The Mythical Man Month”, all IT projects run over budget and are always hugely late. Figures average around 834% in cost overruns, with some jobs being so late that they are never fully implemented, are typical for the public sector. Private sector projects have similar outcomes but the figures are much more difficult to substantiate due to poorer record keeping, better accounting and ensuring that the truth is hidden from the Company Directors.
More recent PFI projects (Private Finance Initiatives) will obviously go the same way, but at least the Government does not have to take the whole blame."
(I wrote this 19 years ago)
One change would be to make projects CoD. The supplier gets the cash when they deliver, on time, deductions for late delivery (not "bonus" payments for stretching it out). Or the money put in escrow and payments made for successfully reaching milestones on time.
Then, maybe, the suppliers would provide realistic timescales and fees in their tenders...
At the moment, the lowest bidder goes in way under cost, then keeps pumping until they get more than the most expensive bid, and it takes longer. As long as the supplier feels no consequences for being late, in fact they benefit from being late, there is no incentive for them to actually deliver.
I don't think that will work as the implementer is financing the cashflow for the government.
Stage payments, maybe? What is unique in the public sector is that even for nominally long-term projects, the executive management (i.e. the Government) could change every 4-5 years and bring in a different set of priorities that means that the goalposts have changed within a particular project's lifespan.
The incumbent Government also won't want to admit that they may not be elected for a succeeding term and so won't plan for the event (although a savvy programme manager might have s secret squirrel Plan B around a phased delivery for this eventuality).
Given the government has no idea how to write a statement of work let alone actually deciding up front what it actually wants, the SI will happily sign any contract knowing that it will then run rings around the government project manager, forcing Change Request after Change Request including early payments etc.
There's a good reason why these big SI like to work with public sector.
You know the old saying: if you owe the bank £1,000 you have a problem, but if you owe £1,000,000 the bank has a problem? Its like that with big projects too.
A big project has work needed on both sides: merely writing the requirements and evaluating bids will be a multi-million pound project in itself. Then there is all the planning, adaptation, training, procedures etc. Property must be purchased, building space allocated. Its not just a matter of waiting for the delivery truck to roll up. And of course in the meantime the problems that drove the original procurement are still there and getting worse.
So when the contractor tells you that there is a problem and the budget needs to be increased you have to choose between abandoning all the work done so far and starting over with a new contractor, or else paying the extra. CoD terms make no difference to this. At best you will get a system that sticks to the letter of the contract but is entirely useless. At worst you will get a decade long lawsuit, at the end of which you might get some money. You certainly won't get a system that actually works.
The biggest issue is that most 'junior' management civil servants don't seem to know what they are doing when it comes to this. I work on one of these projects and since the scope was originally set (5 years ago or thereabouts) there has since been over 670 change requests to the scope ranging from simple minor changes to the way a page looks to massive architectural changes like how authentication should work. When this happens resource already allocated to one task has to be moved to cover it adding further delays. No business can be expected to work for free so I disagree with your statement that the lowest bidder always wins and then sneakily inflates the price. This probably does happen sometimes but when it comes to government contracts - not always. I have seen firsthand the incompetence of civil servants. Some of the demands they make are ridiculous and I could point a finger (not going to) at one individual who more than anyone else is responsible for a lot of the delays.
A few years ago I was contracting for a firm creating an application for an NHS body.
Somehow the NHS body had managed to wangle clauses into the contract that said if the firm made a mistake, it had to be fixed at the firm's expense. However, if the NHS body made a mistake, it had to be fixed at the firm's expense.
The people at the NHS body were spectacularly useless - they didn't know about the clauses and actually thought their jobs were on the line, as NHS england was threatening to close this body if the project failed - and they were still unable to deliver the simplest of requirements until after deadlines were missed or beta versions were delivered. And we are really taking basic stuff - what role of person would be responsible for progressing a document through a workflow, for example.
The firm eventually pulled the plug, after losing about £5m and being no longer able to predict clawing any losses back. Chump change I know, but big money for a smaller firm.
And the guys at the NHS body are still there, and they fucked up the next attempt to get their project implemented, too.
The point being, when the gov body has this power over the firm creating the deliverable, they will fuck it up anyway.
Does the private sector do much better? The likelihood of project delivery on time and on budget decreases with project size, but very few companies are in a position to regularly bet hundreds of millions of pounds on giant projects. Those that do sometimes get it wrong too (the TSB fiasco springs particularly to mind).
When a big company does have a big project go pear-shaped the public don't generally get to hear about it; its not taxpayer money so its not a scandal. Everyone involved has an incentive to pretend that everything is really fine. So we have very little clue about whether government actually does worse than the rest of the world. Its possible that the UK civil service is comparatively good at project delivery.
M'lud I refer to my previous comment:-
"Private sector projects have similar outcomes but the figures are much more difficult to substantiate due to poorer record keeping, better accounting and ensuring that the truth is hidden from the Company Directors."
Note better accounting (for some definitions of "better").
@Paul Johnson 1
"Does the private sector do much better?"
No idea but does it matter? If a private company fails in their project then private individuals lose money for something they tried. Most businesses fail pretty quickly but that is a positive not a negative and why we have cheap, quality and desirable services and products from the private sector.
On the other hand if the gov project fails its a bill forced on the population through threat of violence (aka tax). So if government is doing the few things government is good at then they wont have lots of failing projects because they are not innovators, they are merely custodians. When they throw money on white elephants (millennium dome) and fail miserably at things they should be doing (dispersing free money to the needy) their focus is obviously on the wrong things.
We should leave innovation to those willing to take the risk and the gov should focus on the few things we actually need a gov for.
@Paul Johnson 1
"Unfortunately some of the things we need a government for involve big technology projects. Unless you think the government should still be using pen and paper to administer everything."
The fact that they are very good at throwing money away but not very good at delivering we have probably gone too far. Considering a lot of complaints against government projects is the gov and civil service not knowing what they want or how things work then there is an argument that if they are competent enough to handle a pen the right way around that might be their level of technological capability. And not reinventing the wheel constantly might also help.
Well it does matter -- if private projects fail as badly and as often then there is a global problem with the way projects are managed. Importing methods and solutions from the private sector would make no difference and probably make things worse.
Anecdotal evidence is that most private sector projects come in late and either over budget or under delivered. Private sector managers define "success" as whatever is there when the project ends.
Having worked on a project with a £3 million development cost, we nailed that for delivering the whole thing on time (we actually were 2 months ahead at one point before suppliers cocked up by not being able to keep pace). So big bucks can still get in on time with a strong team.
Annoyingly since then about 90% of the projects I've been on that were delayed were because of 3rd parties (like the ones providing API's or the testing platform falling over).
The best thing you can do is employ understanding people and allow them to set budgets and time scales. What the PHB might expect isn't always possible and they have to face up to this. Once you do everyone can be happier and projects can get made on time and in budget.
Oh and agile helps. Sort of. If only because it devs more say in things.
The one thing that the junior civil servants who get these things dumped on them will do is regard the project as a CV improvement service. They shift the project requirements to suit their career and the poor bloody supplier gets to try to square a dodecahedron. The cretinous lack of oversight of the juniors is where the Sir Humphrey's could actually make a difference, and then you could start worrying about why a supplier was chosen (see 'Door, Revolving, Mark 1')...
that the cause of much of the overruns etc, is the fact that the government fails to produce a detailed spec for many (if not all) of these projects, and then moves the goalposts repeatedly during development. Sorting those two things out would go a long way towards fixing things.
Specifications do get written, only they get approved by someone with absolutely no knowledge of the subject matter.
The spec gets sent to the supplier, supplier does exactly what is in the specification.
The system doesnt work, a full (paid for) investigation is done, a change request raised, an extortionate ammount of money is quoted in the super over inflated impact assesment.
Change goes ahead with the specification for the change being signed off by the same people who didnt understand the initial requirement the reason for the change or what the change does.
Rinse and repeat until the beancounters get fed up and can the project or until someone with a little bit of knowledge queries a specification and gets it right.
"There is little apparent link between successful delivery and the careers and ambitions of the permanent secretaries who are nominally responsible for them."
You still wouldn't be able to get rid of them even if they did make a balls up of it.
It's not just these people that need to be held accountable it's the vendors as well.
Thirdly, it said government should explore the option of outsourcing project management from individual departments to a dedicated central unit. It also suggested creating a new government committee to oversee large projects.
So you want to outsource project management and create yet another committee to add extra levels of bureaucratic nonsense.
Hmmmm...what could possibly go wrong with this?!?!?
If you sue them, you have a long court case and no working system.
If you withhold payment, they sue you and refuse to deliver. See above.
If you refuse to work with them again, you run out of contractors because the number of companies in any area of industry capable of taking on a hundred million pound project can be counted on the fingers of one hand.
As someone who was at the sharp end of the HS2 project (they are putting the line through our property - we've since moved) I can say from first hand experience that the project is ineptly managed, ill conceived and badly communicated with people on the ground. HS2 Limited is very poorly managed by government with negligible oversight and they are just allowed to do their own sweet thing in their own bureaucratic, incompetent and tortuously slow manner. I won't bore everyone with the details, but in the real world if someone agrees to buy your house, the legal process takes up to three months. With HS2 it took over two years, during which time we were stuck in limbo.
A complete rethink needs to be done about all government projects as they all seem to suffer from the same incompetent management by bungling ministers and civil servants, irrespective of which political party is in charge.
They also make up for not having clued up people involved in the specification phase by having many committees full of unaccountable clueless jobsworths. I protest. I think you will find that they have committees full of professional and dedicated people. They just don't actually listen to them, generally because someone senior thinks they know better/ got taken to Wimbledon by the supplier/ is swayed more by the shoutiest person than the expertest.
I salute the British government for their bold plan to see it is possible, through setting IT project delivery dates in conjunction with built-in massive delays in meeting those dates, to actually use the immense delays to force travel backwards in time.
Pretty visionary stuff, if you ask me.
This is on the right track but there is a lot more that can be done
1 - Specialist Skills, having a English or Economics Degree will be of no use in understanding, scoping and running IT projects as a client, they just won't have a clue. France have the École nationale d'administration which provides training to Civil Servants and something similar is needed here, they need to know what an IT project is from the inside out. I mean which initiative does not include an IT angle these days.
2- Providers should be forced to publish all estimate and actual's in time and money for IT projects both Private and Public above a certain amount, this will highlight failures and help to break up the the large consultancies who are effectively mugging private customers who are unable to check history of work. I've seen this in practice and it involves legal documents with NDA's attached post a failure.
Failures should be made into case studies and circulated, as engineering failures (bridges etc..) are today.
The firm of surveyors contracted by HS2 to value my property (they are going to put a railway line through it) said it would take them ten days between visiting my property and them submitting their valuation figure to HS2. In the end it took them six months!
An estate agent who we invited around managed to value our property there and then - the same valuation figure that took HS2's agent six months to arrive at. I wonder how many billable hours HS2's contracted surveyor charged for?
I also gather that there was no service level agreement between HS2 and their appointed firm of surveyors... do the job when you like, take your time and bill us for however much you like, we don't care, the public are paying anyway.
that the civil service/government will implement without a fight
" It also suggested creating a new government committee to oversee large projects."
Mostly because it creates a whole new bunch of civil servants/self important MPs to stick their oars in, and slow each project down even more while it waits for their 'approval'
Or maybe I've watched too much yes minister lately including the staff efficiancy one where a study in the 900 odd positions in the department led to a need to employ 200 more staff....
Government : if you think the problems we cause are bad, just you wait until you see our solutions....
The initial article and the comments all seem to show that something is very, very wrong - and as one poster wrote above - it has been wrong for a long time. There seems a consensus that the proposed 'solution' won't do anything but create another layer of bureaucracy which will have no significant impact. After working with projects for many years - mostly in the private sector but also some government ones it seems as though the whole system for defining, managing and delivering projects is wrong:
1. We simply cannot estimate with any degree of certainty project costs before hand. The bigger and more complex the project, the greater the uncertainty. A budget is a forecast - and like pretty much every forecast the only thing we know with certainty is that it will be wrong
2. Creating detailed specs before a project starts simply doesn't work. Its time consuming and often out of date before finishing and in any case requirements and technology available are changing so fast that spending 6 months or a year to detail things out is, in itself, a complete waste of money although it might give a warm, fuzzy feeling for a while....
3. The current procurement 'best practices' force suppliers to lowball to even get their foot in the door and then manage rigorously for change requests. This is exacerbated by the fact that there are often many sub-contractors with the same issues in large projects
4. Because of the above probably as much time is spent on a project managing the change requests, budget reallocation and increase and arguing about specs than actually delivering the project. Fear of blame / failure is rampant as the price of being seen to fail is far, far greater than the reward for actually getting something done.
What can be done? Project partnerships are one way to go. These have been tried successfully in construction projects - pioneered in the UK but then sadly abandoned in spite of really good results. Accepting and managing through uncertainty - there are quite a few ways of doing this - sadly not really incorporated into MSP / Prince 2. Finally managing for and rewarding outcomes not 'deliverables'.
Is the above a panacea? No. Has it worked in several places - yes with much better outcomes. Will something like the above be adopted - not a chance in hell as although the solutions are relatively simple it provokes too much change and places an increased burden on those comissioning a project to take responsibility and accountability which will never happen in either private or public sectors. Very sad!!
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
