Why do IT projects like the UK's scandal-hit Post Office Horizon end in disaster? This week the incredible scandal that is the UK's Post Office Horizon computer system, which ruined people's lives and drove some to suicide, finally exploded into the mainstream. It's left the public, media, and politicians openly asking: How does such a project go off the rails so hard, why is the computer supplier Fujitsu …
For people outside of the UK, PAYE is Pay as You Earn, which is a taxation system devolved to the employer, so that the employer works out the tax for each of their employees according to what the revenue think are simple rules (although they are defined well enough to be codified to include in a payroll system), and deducts the tax (and National Insurance, another form of tax!) before it is paid to the employee weekly or monthly according to the payroll period. This is then paid/offset against other tax liabilities by the employer to His Majesty's Revenue and Customs, again weekly or monthly.
It's a system that works quite well if someone has just a single employment, and no other significant form if income, and these people do not have to fill in a tax return. Of course, it doesn't take additional employments/other taxable income like stocks and shares or interest on savings into account, and these ought to be declared by filling in a tax return, but I'll bet that a lot isn't (but the Revenue don't really care as long as it is not too much - it may be more costly trying to collect small sums than the tax that would be collected).
People who are self-employed or run their own companies aren't so lucky. Most of the time, they have to fill in a yearly tax return, but even when I used to run my own company (but paid myself as salary rather than salary plus dividends), the Revenue decided that I didn't have any significant additional incomes, so let me off a tax return for several years before I shut the company down. Since then, I have been under a reputable Umbrella company, and have not filled in a tax return for years!
I still remember the first year I did a tiny bit of weekend contracting and, wanting to make sure I paid my fair share, stuck my head above the parapet and told HMRC. The shock when they suddenly demanded 85% of the money I'd earned will stay with me for a long time. I'd put a little under half away to cover any eventuality, but 85% meant I didn't even cover my fuel/car costs for transport to/from the work. Not only that they make it severely complex for a normal person to get it right (easier now than it was a decade ago) and complex to get out. Turns out I needed to run a bunch of spreadsheets in the end to deduct profit for items like proportionate use of car/phone/house and counter for PAYE salary-sacrifice pension malarkey being done by my main employer to get it down to ~40% (which I don't dispute in theory as I'm not here to avoid tax, just that it was insane before). All I wanted was to do a few Saturdays and perhaps take the family away for a night somewhere as a treat.
FWIW I was also always told that if you put in your assessment on the first available day and pay it, the tax office is idle and you'll get investigated (deservedly or otherwise) to the end of the earth, but if you put it in on the last day, it's just a normal submission and subject to less hassle. This then means you get no advance warning of however their wizardry pulls together and the magic number you're likely to have to pay them.
Your comment about UK tax working well with a straight income but not otherwise hits home. I'm the main earner and as such, Child Benefit hits some threshold where I receive it but they then want almost all of it back. If it was assessed on joint income, it wouldn't be a problem. As it is, this is taken forcibly at self-assessment time. How people who don't already self-assess but earn more than threshold are expected to deal with this (not seen it publicized) is beyond me. Moreover, as soon as I knew, I cancelled the Child Benefit as £82/mth is great to put in the child's savings or throw at new school uniform or whatever, but not quite as noticeable as the £800 demanded back with no notice.
The sole reason I had to start Self Assessment was the Child Benefit charge, and IMO it should always have been assessed on the combined income of the parents of the children it's claimed for.
At one point I was just about earning £60k so had to pay nearly all of it back, but my wife was only in paid work part time and only earning £10k (and doing childcare the rest of the time - because working any more and paying for childcare would effectively have had her on considerably less than minimum wage for the hours she was out of the house).
Yet a couple both earning £49k before tax wouldn't have paid a penny back, in spite of their significantly higher household income after tax.
FWIW I was also always told that if you put in your assessment on the first available day and pay it, the tax office is idle and you'll get investigated (deservedly or otherwise) to the end of the earth, but if you put it in on the last day, it's just a normal submission and subject to less hassle.
I did my return right at the start of April last year. Refund paid four days later, no sign of an investigation
>It's a system that works quite well if someone has just a single employment, and no other significant form if income, and these people do not have to fill in a tax return.
One caveat to that. If you have a single employment, but earn over a threshold (currently £100,000pa but looks like it rises to £150,000 soon) you also have to complete a self-assessment tax return.
No, there are a couple of good ones. There is a group of them that talk regularly to HMRC to make sure that they have as little hassle as possible. They mostly don't charge too much, and obey the tax rules, so I have little hassle with the revenue, and am comfortable that I am not going to get any mad scheme as proposed a few years back when HMRC changed the IR35 assessment process and we saw pop-up umbrellas appear and disappear to offer 'tax efficient' come to bite me in the back a few years later.
My only gripe is that I am paid as a basic salary, with the bulk defined as "Commission". This saves a tiny bit of NI, but only a percent or so. Means that to a bank or building society, I don't earn enough to borrow much on tradditional mortgages, but as I'm now mortgage free, that does not worry me too much.
It occurs to me that building software per-se is not hard - we really do know how to do this stuff, and there are companies that excel at it.
What is hard is building anything (train tracks, nuclear power stations, ERP systems) where the project spans multiple years and has multiple, changing owners and stakeholders.
The longer a project goes on, and the more it has to accommodate tweaks and changes and budget shifts and new ideas, the more likely it is to fail. Perhaps one reason large infrastructure projects in China are finished at lower cost and significantly faster is that a project mandated by the CCP *does not change* once it has been decided upon.
No feature creep. No changed specifications. No late discovery. No new management with a "better way to do it". No revised budget.
And in particular, no client who believes that because they are a subject expert in their activity, they are also a subject expert in implementing a software system to manage it.
The fact that there's rampant corruption in China's construction industry is an entirely separate issue. It makes Westerners feel more comfortable that they're not so much of a "threat". You can go back a decade or so and read similar comments about their automotive industry, which at the time was derided for producing cars with 1970's quality. The reality is that they have a genuinely different culture and we can learn a lot from observing where they are successful when compared with Western industry. And there is no question that they have had a period during which they've transformed their industry and society.
When it comes to problems with our own large scale projects, it does actually help to compare with other different approaches rather than making random (usually self-interested) statements about how "we'd do it better".
It might, however, be related to the issue under discussion, namely why they can do them so quickly. Failing to follow safety standards might be another reason for completing it faster, and that works for code as well.
For example, I've just finished writing some code to do a job. It's a smallish script that handles a file in a certain format. I've done the interesting bit where I implemented the logic needed to process the file as expected. It comes to about 150 lines. I am not done, because I need to write the probably about 400 lines of stuff that never touches the file's contents nor the structures I built, but is necessary for this code to be worthy of sending to others. For example, if the file isn't write, it is not sufficient for it to print "Unexpected value in header at 158: 0x16" and exit, but that's what it will do right now. I need to catch classes of errors and report them to a user with a message they understand. I need a method for them to tell it to ignore those errors that aren't fatal and try anyway and documentation telling them not to use it because it's likely to get something wrong eventually. I might even need a way for them to send me the file that broke it so I can improve the code. If all you need is a piece of code to do a job, I can write one quickly. The same is true if you just want a building up quickly. If you want that code to be solid and easy to use or that building to last a long time through whatever typical nature will do to it, you need to spend more time. If standards are different, you can't compare the processes accurately.
But that raises another subject: Are our safety standards proportional and effective?
It's easy to think that other countries are 'below our standards' and are taking 'short cuts' which risk lives but in my experience there are at least some areas where we seem to have gone very far over the top......
That's worth discussing, but it is not relevant to the question of speed. Whether it is our standards that are too high or their standards that are too low, or even if both are the case in some area, it still isn't a valid comparison. We can't say that China builds faster than we do for some organizational reason if what they did faster is not what we would have done.
When I look at the way they're building railway viaducts, it's clear that safety IS a major concern and that they're taking the approach of mechanising to the max to reduce humans being in the danger zone
Those enormous purpose built span layers are doing in hours what would usually take months in most other countries and they haven't skimped on seismic protection
It's clear there are major corruption issues at regional levels due to party members getting into higher positions due to "who they're related to" rather than ability and most of the coverups are aimed at preventing higherups finding out (eg: Covid) , but China doesn't suffer from Russia's "Vranyo" culture and they're actually weeding this stuff out, unlike Russia's potemkin efforts
That's private companies, not public infrastructure
The Chinese have significant incentives (a bullet in the back of the head) for public works not to be as subject to "tofu dregs" as private companies and to be utterly blunt, they're doing far better than my experience of western 1970s-80s government projects in terms of costs or work actually being done to spec
I see this a lot and is the biggest contributor to failings (amongst many others) in my experience. Folk change roles so often they tend to have a self centered short term view of what they should do to make themselves look good. This is often not the same as what should be done to minimise risk and ensure the project is healthy long term. But they don't care about long term as they won't be there, will be in another role.
Keeping the same resources on a project generally helps a lot. Early on near project kick off is less of an issue if realise have not got the right people involved so adjust it.
>Keeping the same resources on a project generally helps a lot. Early on near project kick off is less of an issue if realise have not got the right people involved so adjust it.
The problem is the timescale of the project and the career aspirations of the resources. Do you expect everyone on the project to work at the same grade, salary and on the same role for the full duration of a 3-, 5- or even 10-year project? Ignoring the 50% of the population who could go on maternity leave, everybody has professional ambitions. Some retire early, some leave for better opportunities, some even emigrate.
This! This is what's made HS2 more expensive than it should be. The government is the paymaster, despite this being delivered by 'HS2 Limited'. The funding comes from the Treasury (via the DfT). The amount of time (and money) wasted in the chopping and changing of specifications for Euston (which is in a difficult spot, as any Londoner knows), is absolutely incredible. The rail industry pointed out time and again that the station should have at least 11 platforms for capacity reasons (well, once you build it, you can't really expand), but the government first decided that 10 would be enough (reducing the capacity and hence the benefit ratio by a significant margin), and now, is 'putting a stop' on development at Euston altogether (and driving the lines only to Old Oak Common in West London for the time being), which nobbles the project even further. Let's not add the cancellation of Phase 2b (to the East Midlands) and the nobbling of Phase 2a (instead of to Crewe, shoving more trains onto one of the most congested parts of the West Coast Main Line between Birmingham and Manchester).
Sometimes I do wish we had people in government who said "the project is now set in stone, get on with it" and leaving it be to run to the end without interference, which provides clarity and assurance to suppliers and the industry, so contracts can be costed with less contingency for cancellation, coming in cheaper and making the whole thing more cost effective. But no... we have politicians who like to meddle to show their constituents (or other shouty people in their party) they're doing 'something'.
I'm a little confused about what we are supposed to think about HS2.
When the government was determined to push it through, the media said is was massive waste of taxpayers money that nobody wanted. It is destroying vast tracts of countryside and the only purpose of it was to funnel billions into the pockets of the government's favourite construction companies.
Now that the costs have spiralled and the government are trying to back away from, the media are telling us that is a vital regeneration project and the government is reneging on their promises and screwing over the deprived areas in the midlands and the north.
Seems like they can't win either way.
My own opinion is that Britain has proven itself to be very bad at running railways. It will be expensive to travel on, there will be overcrowding, long delays and cancellations, wrong kind of rain, leaves on the line, industrial action, signalling problems. And on the rare days when you can make the journey in 45 minutes, it will then take you a further hour to travel three stops on the London underground or Birmingham tram network. So it's never going to live up to the "high speed" expectations.
I don't know what that says about what we're supposed to think about HS2, but what it says about "The media" is that they're a bunch of self-interested bastards who only publish stories based on how much click-through revenue they will generate rather than reporting what's actually happening at any given time and letting us make up our own minds about if it's good or not.
HS2 was a necessary project. The WCML is a severely congested route and has no capacity for further expansion, from 1998-2009 about £10bn was spent on improving the capacity from 16tph to 20tph (train per hour). To get additional capacity would require major investment to build new lines - enter HS2. It could be argued the scope of the project expanded too far and maybe should not have had high speed running throughout but the business case is better with faster trains as you automatically get more capacity. I really hope a future government dusts off the plans and continues it because, with some reductions in complexity (less tunnelling!), it could have been a great bit of infrastructure.
I have never been confused about HS2.
HS2 has always been a good project. Why? Because it creates *capacity*, something the Victorian-era railway corridors heading north don't have. If you've ever gone on a rail journey from Paddington, Marylebone, St Pancras, Euston or Kings Cross (or Liverpool Street, for that matter), you'll notice how close the built-up areas are to the rail tracks. Often it's only 4 tracks, maybe you have 6 if Overground or the Tube happen to be nearby. The Chilterns are the only real corridor left with some space (and yes, unfortunately, it's greenbelt land and thus should not be built on).
The media plays everyone like a fiddle. Those complaining for environmental reasons missed the point of the project because the media concentrated on the "look, we'll be in Birmingham 15 minutes faster" speed angle more instead of the environmental benefits like "we'll be there faster, but the towns inbetween get better local stopping services because there's more capacity on existing lines, and boy will we be able to pull lorries off the roads by shoving their cargo on trains!" - The former sells papers better than the latter. The constant whining introduced uncertainty, and yes, the councils and politicians protesting against HS2 crossing their constituencies didn't help much either. Unfortunately, several MPs are of the Tory persuasion and used their shmoozing powers to persuade others in their party to vote against various things, and also persuaded the men with the money to start cutting the cost of it. Bizarrely, there are enough local politicians of both camps up North who fully appreciate the benefits HS2 was going to deliver to their areas, only for them to get shafted by their own MPs in Westminster!!
It's a travesty. We deserve the railways we have because we want the best but refuse to pay for them. No, they are meant to magically manifest out of thin air with no environmental impact and money off the magic money tree!
"We deserve the railways we have because we want the best but refuse to pay for them. No, they are meant to magically manifest out of thin air with no environmental impact and money off the magic money tree!"
Nothing wrong with wanting better for less although it's not so much 'our' problem but rather a systemic problem of government where 'we' the public are lied to by those we elect to represent us, because cosy lies about gross waste, immigrants taking all our jobs, Europe being the cause of all our problems etc. are far more acceptable than cold, hard truth to the average unthinking dullard with a vote.
The understanding I'd built here is that actually it was never about "being able to get to Leeds in 1h20" or whatever the touted headline was. The reality here was that it was about creating a separate line for the high-speed stuff that meant it wasn't running on the same lines as the slower/constantly-stopping local services. In doing so, the vast quantities of cancellations and late running services would be significantly reduced. This would be as they'd not be stuck at the first departure station waiting for a green because a 30-second delay from a London-Edinburgh service that was slightly behind wasn't going to cumulatively mean they'd hit the "Birmingham direct" at their 12th station and have to wait for that, then later be stuck waiting to leave their 20th station because they'd been leapfrogged by the giant Drax freight service that should be 15 mins behind.
It's a capacity thing, but that's not a snappy marketable headline that everyone can seemingly "get" straight away.
Yeah, whilst not a fan of HS2 and quite pleased it's been canned north of Birmingham, it did strike me that the media showed massive hypocrisy with their about turn from stories of doom and nimbies crying tears of privilege because their favourite bridleway might be next to a railway to stories of doom about the north becoming some sort of Mad Max post apocalyptic wasteland because of HS2's cancellation.
"Britain has proven itself to be very bad at running railways"
It has been from the outset, despite them having been invented here
It's worth noting that most existing UK intercity railway alignments aren't based upon the best geographical route, but what was available after canal companies preemptively purchased tracts of land along the best geographical routes in order to stymie the competition railways represented
Another thing to bear in mind is that many of the lines which are _now_ in built up areas weren't when they were built. Railways attract other infrastructure over time
> cancellation of Phase 2b (to the East Midlands
Very happy with that as it would have left a viaduct through the middle if the town (Long Eaton) at the level of our bedrooms.....Mind you it would have been fun during my retirement watching them struggle to build the damn thing at the bottom of my garden, I was going to take daily photo's to build my own time lapse...
Arguably HS2 was also badly impacted by politics - as all large scale projects tend to be. The line passes through areas of the country that are regarded as sacrosanct, the Home Counties. The terrain needs to be protected to satisfy these groups, so you end up with things like the Chiltern Tunnels which cost billions to construct (and end up being protested against by all manner of local residents; even the *emergency exit* buildings are a matter of contention for the AONB board.) The simple fact of the matter is that if you want large scale projects you will necessarily lose a bit of the natural world, but we allow these groups too much control over the project because they are in politically significant areas.
The astonishing thing is that they couldn't be less politically significant in real terms. They're vast tracts of nowhere with bugger all in them and a population of approximately five. In any sane world they'd have been told to sit down and shut up.
Unfortunately all of those five are incredibly rich and vote Tory and thus here we are...
"you will necessarily lose a bit of the natural world"
There is very little "natural" in the chilterns. Even the "ancient woodlands" are plantation forests planted to service the Royal Navy of sailing days
To underscore that point, the Houses of Parliament are significantly older than most designated "ancient woodland" in England
A railway is one of the least intrusive things that could happen there. The NIMBYs are usually one of the worst things that can actually happen to the local environment
My late father was a senior local government officer. He said that the worst NIMBYs were often the last people to move into the village/town. Their attitude was "I have moved in because I like what I saw, now nothing must change". This applied even if their house was new - Now, no one else should be allowed to build a new house. They were always the ones campaigning against the new bypass, in-spite of the fact that most of the rest of the village/town had been pleading for one for years...
Successive governments have done this with railways for several decades and it's got a lot worse in the past few years. Under BR, they would get a multi-year grant and be largely left to get on with it. These days the government micro-manages it to the Nth degree, with the transport secretary this week (which may not be the same one as next week) making decisions often at odds with the advice from those with years of experience who actually know what they are talking about.
Then if there's a public backlash, the government blames the railway companies who were simply doing what the government had told them to do - e.g. the proposals to close all the ticket offices, which was a government initiative, the train operators were told to make plans for it, then when the public pushed back the government backed down and tried to take credit for stopping the railway companies from doing it. And loads of people fell for the lie about the railway companies being responsible for the proposals and the kind, caring government stopping them from doing it.
It's no wonder the railways are such a mess...
The problem with Horizon wasn't so much the software being faulty, despite the huge scale, that can be fixed, yes, feature creep, changing political winds, budgets etc. cause problems but rather it was the way the problems were handled, as is oft said, it's not the crime that gets you, it's the cover up.
Cowardice, people too scared to admit problems for fear of losing their position, money, status, too ignorant or stubborn to understand or even listen to those waving the red flags etc.
It is hard and there will be technical issues along the way with the software and systems, no matter who builds them.
The problem in this case is that the issues were not assumed to be with the software or systems, but with the users. Specifically assuming users were thieves and 'investigating' them, when the vast majority of them were not thieves and the investigation should have been directed to the system, basically Fujitsu were allowed to get away with it rather than being forced to prove that their system was working correctly.
It seems that in this case the S/W problems were known.
It was an extreme case of it being difficult for a man to understand something when his income depends on not understanding it combined with a Watergate effect - trying to cover up a problem and then escalating cover-ups of cover-ups.
it being difficult for a man to understand something when his income depends on not understanding
That's a code-phrase for a two-part human problem: (1) lacking good morals; and, (2) lacking technical competence required to find good technical work elsewhere.
Private companies are BAD at everything, far worse than government.
Government software projects tend to work. The web was better with NCSA Mosaic than with any browser since. The problems happen when, instead of developing in-house, governments outsource projects to private companies.
This was a Fujitsu failure.
Gov't cannot afford large scale mistakes, so it's development moves at snail pace compared to the "move fast and break things" approach the private sector (specially that hedge-funded).
The difference in end result is usually obvious, with one product having archaic interfaces and technology but very stable results, and the other being unstable and failure-prone but running on edge technology.
"Move fast and break things" is a catchy phrase, but it's a worthless development philosophy. That's because when sub-team A breaks things, sub-teams B, C, and D waste time re-writing, and possibly re-architecting, their code to work with the new, life-of-a-fruitfly, interface "standard". You see lots of motion, but get little true progress.
It's not just the browsers, it's the "web standards". The World-Wide Web Consortium ("W3C") has been taken over by industry people, and continues to willy-nilly add new standards to enable the new-and-shiny wet dreams of marketers.
It's telling that Amaya, their free software for editing websites and checking them for W3C standards-compliance, was last-updated in 2012.
If functionality of browsers had stuck at Mosaic level then somebody would have come along with a way of making internet resources that could do more things and that would have taken over because people don't want to deal with exclusively static documents. I don't like JavaScript much, but a JavaScript that's a mostly open standard is a lot better than Microsoft-brand language that only works in one browser and a lot of sites use it, which is what you would have gotten in the 1990s. What you got in the 1990s, in turn, would have carried over to the 2000s, just like JS did, because that worked in old and new browsers. You can't prevent people wanting new features just because you don't need them, and if you refuse to implement them, you'll just end up with someone else's version.
Private companies are BAD at everything, far worse than government.
I would disagree, my experience has been that private companies working to create software for internal use or for sale to other private companies, the software will work. The problem is working for governments (local, state or federal) is hell, it is like herding cats. Scope is completely uncontrolled, you have multiple stake holders each with conflicting agendas or requirements.
I have NEVER seen a government software (or even hardware) project succeed. And I've never seen one continue past a change in government.
My £0.02 worth of ideas:
1 - Lack of specification.
2 - Lack of consistent specification. I'm not saying specifications must never change (that would be unrealistic) but any change in spec will lead to an increase in time & costs. Too many changes will result in a system that's never finished.
3 - Unrealistic timelines. A politician may announce a launch date that suits them (think elections) rather than is practical.
4 - Unrealistic budgets/Value Engineering.
5 - Lack of trust. Public sector doesn't often trust its employees to know their subject area so employ far too many consultants.
6 - Outsourcing everything. Outsourcers have no interest in your line of work. All they're interested in is the fat pay cheque from you. They'll do enough to get paid which may not be enough to actually use the system.
This can be mutually beneficial - for politician and big corporation.
If something is not clearly defined, unrealistic, you can add cheap labour into the mix (but charge tax payer eye watering sums for it) and it means you can make massive stable profit for at least the term of the government, before it gets scrapped. Then when politician in charge quits, gets removed or whatever, just give him executive position as a payment for services rendered.
This is one of the reasons why Tories have changed IR35, to funnel even more money into those cowboys and at the same time Home Office gives out skilled workers and intra company visas like it had a diarrhea, to increase profits even more.
Now it got so grotesque, that we pay highest taxes in living memory and there is nothing to show for it.
Here's the current executive chair of HS2 talking to the Transport Select Committee this week:
There are four reasons why the cost is more than what has been budgeted for. The first is that the cost estimate in the first place and the budget that was set in the first place were too low, in my opinion....Secondly, there have been some changes to scope. Thirdly, there definitely has been some poor delivery on our point. And fourthly there’s inflation,.......It is worth remembering that between 2010, when the then Prime Minister, Gordon Brown, initially launched HS2, and 2019, when the current budget was set, the scope of HS2 was changed significantly by a whole series of Ministers. Much more of it was put through tunnels, which cost a lot more than putting it through cuttings, for example. There have been a whole series of scope changes. Yes, it is true we have not delivered in the way we said we would do.
...The Government and the company decided to let cost-plus contracts, where 99% of the financial risk is with the Government and only 1% is with the contractor, which is extraordinary. That was well before my time, but my understanding is that it was done in order to get these contracts away because they were so huge. This is the civil engineering contracts, not the station contracts.
If you're interested in researching the anatomy of a government cock up, it's well worth reading (or hearing) all of it.
>G. B. the financial genius who sold half of Britain's gold reserves for >10% less than the historically low price gold was at because he anounced the sale date in advance.
A reminder that the loss on the sale of that gold was estimated at around £3bn. Or a fraction of the losses from any of HS2, Covid PPE, Covid loan fraud or any number of government "initiatives" since.
I've seen it many times. Young engineers sleeping in the office under their desks working basically for 18-20h a day, because there is a "deadline". Then the very same people who were promised bonuses, got sacked when new investor came in and brought their "trusted" people in.
I am probably repeating myself too much, but never ever do any sort of overtime, unless the business make this worth it (not by promises, but by something tangible like extra pay there and then or actual shares in the company) or you are the owner.
The moment you are asked to stay longer, I don't know to do this for the team, because we are family, to have everyone's back, you know that you are being exploited and asked to cover up management failure. Never buy into anything like this. They are not your family, they are there to make money off of your work. That's it.
I was drinking with a chap who was a salesman for a major printer manufacturer. Suddenly, he started panicking.
"Get me out of here", he whispered.
I took him out the backdoor and into another pub and asked him "What was the problem?"
He told me that he had just closed a multi-million euro deal based on a product that was barely in Alpha so that management were driving the engineers so hard that their eyeballs were bleeding.
Those engineers had just walked into the first pub.
Thanks to IR35 that limits competition and the fact that big corporations are exempt, they can make massive profits while hiring cheapest labour possible and pay little tax.
The government in bed with those corporations got even more greedy where they started giving out skilled worker visas like confetti.
Such corporation can get overseas developer for a salary just over minimum wage and charge government or other businesses for their services £500 or more a day pocketing the difference and shifting the profit offshore.
Then there is no accountability. These big corporations realistically don't have to deliver anything. There is always going to be an excuse, a sweetener.
Hopefully after Post Office the HMRC and Treasury will be investigated for their lies and participation in this.
Most members of Government (and this one in particular) have absolutely no knowledge of IT, systems, analysis or anything else besides how to get rich, give bungs to their mates, ancient Greek and Latin etc. We're some what short of anyone with basic mathematical or analytical skills up at the top, just the ability to give orders and dodge responsibility.
We're some what short of anyone with basic mathematical or analytical skills up at the top
There are people with those skills, they just don't use them to benefit the voter. If they see corruption is essentially legal and they can set their family for generations or they can actually do the work for the benefit of everyone, they choose the former.
"Most members of Government (and this one in particular)"
It certainly doesn't help by making it party political although if you must do so remember that CW exposed this back in 2009 - the time of the last Labour Govt.
The original contract goes back to the Major govt. Blair confirmed it should go ahead on the grounds that it was important for the PO, for Fujitsu as a major inward investor and for PFIs - that episode has Mandelson's fingerprints on it too.
It's a moot point as to whether the PO were really very persuasive and very good at covering things up or successive ministers & PMs were anxious not to know.
What must be clear is that every PO CEO must have been made aware on taking up the role, if not before, that this was going on and none of them had the courage or integrity to call a halt and try to clean up the mess as it then existed.
Because ministers pick their mates for bungs before picking the most competent people.
Its not just government, its rife in tech in general.
When was the last time anyone here was asked to build the best solution possible?
It always has to be cheap or done quickly...and thats where the problems occur.
Its why a lot of indie passion projects end up better than multimillion pound commercial projects.
It also has to do with hiring as well. These days people are in tech because its a job. During the late 90s early 00s most people working in tech were passionate about it. These days most people are in tech for the pay cheque. Its not an interest or a passion. Its just a job. Add to that the MBAs, accountants and various other suits that exist in the space now and you have a homogenised, sterile, industrial process that lacks any soul or goals outside of sheer profit.
Also add to this the frameworks on frameworks on frameworks and you'll see that we dont really have all that many proper developers / engineers out there. Just a load of juniors painting by numbers.
It also has to do with hiring as well. These days people are in tech because its a job. During the late 90s early 00s most people working in tech were passionate about it. These days most people are in tech for the pay cheque. Its not an interest or a passion. Its just a job. Add to that the MBAs, accountants and various other suits that exist in the space now and you have a homogenised, sterile, industrial process that lacks any soul or goals outside of sheer profit.
There is nothing wrong to want to be paid handsomely for one's talent. The employers used to exploit workers by perpetuating the view "oh it's your passion, you enjoy doing it, so why do you need this much money? It's not a hard work for you" and so on.
Now there is this paradox - the wages in tech are generally higher than in other sectors, however, the pay is no longer something you can write home about.
For people who truly have passion, the only path to wealth is through setting up their own business, therefore most talented people are not interested in employment, because it's where dreams die.
Also add to this the frameworks on frameworks on frameworks and you'll see that we dont really have all that many proper developers / engineers out there.
Because work no longer pays. There is no point pouring your heart out for a big corporation - they don't pay fairly, the tax man taxes massive cut giving virtually nothing in return and if Excel says no, you will be given notice, regardless of your performance or accomplishments. There is literally no reason why one should care about being good employee.
"I agree, but skill without passion is not talent."
Why not? If your building needs running water, do you refuse to accept a plumber who doesn't plumb for fun? If you need something driven somewhere, do you only hire racecar drivers, who clearly love driving, rather than someone who can do the task correctly and does so just to get paid?
I admit that there is a correlation between the people who do the best work and those who have a passion for it, primarily because their passion means they've gained a lot of experience that people who don't have the passion don't tend to have, but there are two provisos to that. The first one is that this is not always the case. Programming tends to have more little details that take a lot of time to learn, and passion helps with that. There are other fields where it doesn't take so much passion to learn the required things to do the job well. The second is that someone is still capable, even in a detail-rich field like programming, to learn how to do something well without having a passion for it. If someone just writes code to get paid for it, but they spent the time needed to know how to write that code well, then they are as capable of making something good as someone who learned that because they enjoyed doing so. I've seen a lot of people who chose tech because they wanted something well-paid, not because they enjoyed it. Many of those people were bad at it and left because the hill of knowledge you have to climb was just too high. However, some of them didn't have any problem climbing it, go to work as programmers and produce excellent output, then go home and do something unrelated with the rest of their time. They are no worse than you or I.
No I wouldn't refuse him, but I wouldn't have high expectations either, I would expect to get exactly or less than I paid for. The guy that doesn't live and breathe plumbing is going to put a pipe exactly where I ask for it. Nothing more, nothing less. The passionate guy though, he's going to look at why I want a pipe there...ah you're installing a new dishwasher, have you got it here? I need to check how high up the outlet is and the depth of the appliance, need to make sure I put the outlet in the right place...also, you didn't ask me to install a waste pipe for it, do you want me to do that as well? I've checked the waste pipe routing and it looks like it might make the job a lot easier if we move the dishwasher to this end of the room, that way I don't have to run any pipes around or near any electrical wiring...etc etc etc.
The guy that does it for a job will do exactly what I ask, which might not be what I actually need, he'll walk away installing a pipe that I can hook up to the dishwasher, but I'm still left with half a solution, because I may not have known that I also need a waste pipe. When I work that out, I have to call him back. Two visits for the same solution. The passionate guy understands what I need and he can see that I've only asked for a partial solution. He will cost me more, but he will sort everything in one visit.
The guy that just wants to get the job done will do exactly what I ask, but as a customer with no plumbing knowledge or experience, I could be asking for the wrong thing...doesn't matter to him. The passionate guy wants to do the best job he can and he knows I'm not a plumber, he'll help me to understand the requirement better and solve the actual problem I want to solve. He won't just install a pipe. That's why he costs more...because you get more.
That's the difference. It's not the capability, it's the understanding and care...you can teach anyone to weld a pipe, but you can't teach people how to understand customers and their problems. That comes from experience and passionate people tend to have a lot more experience because they spend additional time solving problems nobody asked them to solve in their spare time...they innovate.
An amazing craftsman is always a craftsman, an acceptable craftsman is only a craftsman until he clocks off.
There is nothing wrong with wanting to clock off and end your work day...but clocking off doesn't mean you should cease to practice your trade in some way shape or form...a lot of plumbers are also artists, they go home and use their tools to make other stuff, messing around with offcuts and scrap from jobs...so they're always using their tools, practicing, learning, figuring things out...this where the interesting stuff is figured out "huh, that type of pipe doesn't weld very well to that type of pipe, who knew?" or "Man, that kind of pipe welds extremely well to that type of pipe, I'll make a note of that, could be useful" or "that sculpture has been out in the garden for a year now, it's all oxidised apart from that one small length of pipe there, I wonder why...perhaps I should use more of that stuff for underground runs".
If you practice your trade skills outside of your job and do things with your skills that you wouldn't do on the job, you can learn things that you would otherwise never learn and you can become a better craftsman. You will never become a top tier tradesman if you clock off and never use your skills outside of your job.
Doing this is why I know that white orange, orange, white green, blue, white blue, green, white brown, brown (swap oranges and greens around for the opposite standard) is the sequence for an ethernet jack and I will never forget it...as a younger engineer, I used to sit in front of the TV sometimes making patch leads. Practicing getting the length exactly right, testing my crimping, trying out different jacks from different manufacturers, trying out different types of cable, cutting down long 100m cables into 100x 1m patch leads etc etc after properly testing them and verifying them, I used to bundle them up and sell them on eBay for quite a bit of profit, right now a 100m ethernet lead on Amazon can be had for less than £40, but 1m patch leads typically cost more than a pound, RJ45s cost pennies...because I've made so many frickin cables from tons of different types of ethernet, I can tell you by feel how difficult a particular spool of cable will be for a particular job...especially if I've seen the proposed run!
There is no substitute for practicing your skills outside of work. Practice, practice, practice.
You can't really compare plumbing to programming to be fair. Plumbing is a very tactile job and the more time you spend doing it the better you understand your materials and tools etc.
Programming, yeah you can get results being a jobsworth for sure...but if you're the kind of person that builds things for fun, you can not only have depth of knowledge you also create a broader knowledge...meaning you can both get to a solution, but with a broader knowledge base you can also get to a better solution.
The regular guy will build you an email form and it will work. You can fill it in and it will send an email. When you test it, it works...however, over time you'll notice that sometimes the email don't arrive, or they go into spam, or you start getting tons of spam etc etc...the solution you got was the one he learnt ages ago and they will continue to recreate forever...when you get him back in for something else and you mention the issues...he clears the local postfix queue, runs a test ... "there we go, it's working now"...it is 'fixed'.
The better guy will tell you about the various different types of email form he can build, the pros and cons of each, and offer you a choice that better fits your end goal. You will be able to fill in your form, it'll probably have a CAPTCHA, it will link up to some kind of service like Mandrill, which will cost your more, sure, but none of the emails end up in your spam folder, it's solid etc etc etc....when you test it, it works and that is exactly how it will work each and every time it is used...the solution you got was the best possible solution this guy can create using his current knowledge...when you get this guy back in, you won't mention any problems with it, however, he'll have some new refinements to make..."That Mandrill based form I made for you, since then, I've found a way to make it better and possibly cheaper to run, would you like me to update it while I'm here? Won't take a minute...also I found something better than Mandrill!".
5 years later...
Standard guy now has to completely rebuild that form. Loads of stuff is deprecated. His original Stack Overflow snippet from 2006 that he's been cruising on has stopped working...be easier to rebuild the whole site mate...this guy was dirt cheap for 5 years...but now you've got the prospect of a massive project and your site is broken until it's done or some dodgy hacky workaround has to be put in place.
The better guy...his solution still works, because he has been refining it the whole time, every time he learned something new or figured out a better way to do something, he offered it to you. Each time you get him in, all those little tweaks and fixes he did kept the code relevant. He cost you more over the 5 years, but you don't have to completely rebuild your entire site and embark on a massive project because one of his scripts stopped working...he has actually saved you money and you've never had a period of time where your business is limping because of a busted website.
Probably because nobody asks the simple question of what can go wrong. People plan for success and not failure. It's simple thinking outside of the box like that that can make a project succeed. That and proper testing with real case examples. Constant monitoring. Checks and balances. I worked as a data analyst for many years and one if the things I implemented for myself from very early on was to check and verify everything multiple times and every single time a change was made. That meant that whatever I presented I was confident it was right. Not only that but I could prove I was right as well. Sure, it took longer but it meant I was good at what I did.
In this day and age you really shouldn't be getting balancing errors on an accounting system. You should have it locked down so no one can change it afterwards without the highest of authority and good reason. There should be a complete audit trace system. Once they identified the issue reversing should have been straight forward yet even now they don't know because they are getting them to sign a declaration that the reason was Horizon or they get charged with fraud. How can they not be able to identify these issues after all this time? The only thing I can think could be the reason is that the entire system was written with the help of stack overflow answers (don't get me wrong these are invaluable but should be used a guide so you can fix and understand what you did wrong and why it didn't work, not just a copy and paste exercise) by people who list HTML as a coding language on their CV.
This post has been deleted by its author
“The horizon system was based on a messaging system named Riposte used as a database. The interface and business logic was in VB6”
That's Visual Basic 6: Good Grief Charlie Brown!
The EPOSS system was developed in VB5. The sample coded caused much mirth when exposed, but the context was
1. Cash Account desktop Reporting was added as a late requirement, to minimise dependency on the the Post Office mainframe system.
2. It was delivered three months after contract award
3. Without a specification, Post Office staff that worked on the legacy system were recruited.
4. The Dumb ReverseSign function, takes and returns a variant {Integer, Long, Single, Double} - a dumb attempt at polymorphism, without error checking (fails if a string passed). Ignorance of VarType() function was disappointing at the time, but far from the biggest problem with the code.
Fujitsu bought ICL, who had been the UK government's preferred software supplier for decades. The company was gutted, and most development move to Dublin.
Around that time I worked for a company that had also used ICL extensively, and for a new project the internal development team did a proof of concept with PostgreSQL and some well structured Perl code.
The management were impressed by the presentation, mostly because they were unaware there even was an in house team (we mostly fixed the bodgery of ICL/Fujitsu). The ultimate decision was still to go with a Fujitsu proposal that used a terrible database called ODB2, and a weird dialect of C that had been a ICL proprietary product.
ODB2 barely worked, as it was some Japanese academic project that Fujitsu had bought the rights to. I guess they foisted it on companies like the one I worked for to justify their decision to buy the damned thing, and also as a form of vendor lock in - it didn't even use SQL, but its own bizarre query language.
The requirement for government contracts should be that any technology used in the development of the software should be open source. 20 years ago maybe that would be problematic, but today there is no longer a reason to use any proprietary software.
That means if there there would be an issue of maintenance and support, tax payer is not at mercy of one specific supplier, but any competent business could take over.
Well, I happened to grow up around that brand of Christianity! These people are so Holy and Good People that they are forgiven in advance.
Which is neccesary because their version of the 10 commandments are scriptable. Like: "If God didn't want me to $(cheat_on_my_taxes), He would stop me!"
Yes, the problem wasn't that the software had flaws - it was that there was no apparent attempt to review its outputs when there were discrepancies, no right of review for staff fired purely because of a piece of software (maybe not apparent initially, but there should have been proper investigation of incomings and outgoings as part of the disciplinary/legal proceedings, the lack of coordination or records-keeping or whatever it is that would have identified a pattern in these events, the vendor misleading and covering up issues with the software, the PO executives doing the same. Even the lack of enquiry further up the chain - did the ministerial office think that firing 1000s of people for fraud over time made sense? Presumably people who'd be well-vetted prior to their being hired in these highly-trusted roles?
We can get this kind of thing happening with purportedly "well-managed" software - god help us when AI really gets its claws in. Of course there will be crap like this going on all the time right now - AI-driven or not - here's hoping the fuss raises some kind of awareness of creating software with results that can't be human-audited.
I think the root of the problem in the Post Office scandal isn't that there was bug in the Horizon system, but what the reaction to it was. Along the lines "it's not about never making mistakes but how you deal with them".
If at the very beginning, Fujitsu and the PO had gone "oh it looks like a bug, sorry about that, we'll fix it ASAP" this tragic story could have been mostly avoided.
Instead, as we now all know, they double & tripled down, lied, maliciously prosecuted, committed perjury, etc.
While there's plenty of faults with Government outsourcing and these large contractors which results in low quality/unfit software, even a talented and well-motivated team won't guarantee 100% bug-free in all circumstances. It's how you respond that counts.
"what can be done to avoid this level of fiasco in future"
When you work in orgs that promote incompetence to be head of IT time and time again, with no IT experience, this will always happen.
The wrong people will always be in charge, the little underpaid engineers like me will continue to be ignored.
Fujitsu is running software for HMRC that (just like Horizon) they have full-access backdoors to. Basically Fujitsu employees can illegally view AND CHANGE anyones tax returns or tax refunds due whenever and why ever they want.
So you get a family member, just give them a £5000 tax refund. Too small to raise red flags, but then do it for someone else and repeat...taking your cut of course.
"permanent salaries in UK government roles are not high enough to attract anyone good or competent"
Yet the corporate world is eager to sign up these incompetents in their droves post politics. BT took Tebbit (after railroading thru its privatisation of course), A plethora of companies took Ruth Davidson (no idea why but can't wait to see what companies sign up Douglas Ross and Andrew Bowie( nuclear minister - guffaw).
Lets remember that there's a sampling bias inherent in all news reporting. What goes well doesn't get reported,not even in the trade media. As a freelance sub-contractor to a sub-contractor to the main contractor (one of the usual suspects) I've been involved in several that didn't hit the headlines.
A stupid deadline and late feature bloat leading to a compressed development time even more compressed unit, system and user testing. Which leads to stress on the dev and test teams which leads to people leaving which slows development which, well you get the rest...
I was employed as a SME for a payroll system. My role was to manage, test and certify one of its more complex components. I was asked to sign it off as ready. The developers hadn’t even got the UX working properly. Not to mention the numerous unresolved test observations. I said “no” and explained why. A few days later my immediate boss signed it off and I was transferred elsewhere. A rumour was spread was that I had been given “the spear” for not being a team player.
Been There, Done That. Your former boss has more guts than mine, though.
In my case it was that Management wanted me, a lowly project manager 3-4 layers "below" His Lordship, to relinguish the scope of "my" project and sign over 12 MEUR to "Our Favorite Contractor" instead of going through EU procurement, like, the law says. On top of all that, the procurement paperwork were prepared and approved by Procurement so we would waste about 1 years of tedious work.
I told him that it was much better that he did this transfer on his own authority because it was his budget and his scope. He didn't like that very much. Pehaps because the person signing this could be going to jail or at the very least end up in front of an inquest.
He especially didn't like that both Procurement and "Our Favorite Contractor" disagreed and sided with me. In the end, it went for procurement and "Our Favorite Contractor's Minion" got the contract with the proper process. I believe the Minion was pushed in as a shim because "Our Favorite Contractor" had become suspicios of the leadership. They know that working too closely with morons is how your project ends up in arbitration.
The consequences for me were that no work arrived at my desk ever after, which was nice for a while. I ended up leaving.
1st A vendor will bid on a tender knowingly not have the experience or staff to undertake the work with the hope of hiring cheap later if they win it.
2nd If they win they then trawl through all the Linkedin profiles to try to find someone cheap and that who claim to have the experience in doing the work
3rd They bid so low and when they win because it's the accountants who make the decisions they then realise that they went too cheap and have to cut corners.
4th The Company wanting the work done have no idea of what they want or need in the first place.
5th the decision is made on the Golf course or given to one of their pals.
I don't know.
We can make lots of assumptions based on experience and people already have in the comments here.
What those of us with the experience do know, is the following:
* Failure to scope the project correctly in the first instance = impossible to build a good product
* Failure to engage with the client during all phases of development = impossible to build a good product
* Continually changing specifications or very loose specifications = impossible to build a good product
* A waterfall development pattern = impossible to build a good product.
* Poor (no) unit test coverage, poor (no) end-to-end test coverage and poor (no) integration tests = impossible to build a reliable product
* Lack of planned rolling releases, from the very first MVP (minimal viable product), through to the first 1.0.0 release and onward - continuous development.
* Continually changing teams/squads - over a multi-year project, this can murder the best of projects if not handled effectively
* Project rushed out to meet artificial deadlines - usually imposed by someone high up promising the undeliverable.
Many years ago I worked for ITSA which was the state ran IT side of the civil service. The philosophy of the department was quite simple, 'get it right' and we would work on products until they passed the rigid testing regime. Over-runs in time and budget where always inevitable, however the products worked reliably and did what what was required of them.
Then the government outsourced the work to consultancies, to which I was moved, and suddenly the philosophy became one of ‘get the customer to pay the last bill’. This resulted in rushed solutions which we knew introduced bugs, but the current list of bugs would be shown to be resolved and that was what we were required to do.
Often product development starts with a specification which will lack sufficient detail, contain misinterpretations of what the customer requires and generally over looks details that only become apparent as the product develops. Consultancies know this and I have watched, as rather than get the product right (which seems never), will expend a lot of effort with legal teams looking for get outs in the original contracts and specifications so the customer has to either walk away or increase thier commitment to the consultancy. I would laugh if it wasn’t for the damage these lying, cheating flim-flam artists leave be-hide. Plus I will not even broach the blatant corruption I suspected between the customers representatives and the consultancies.
How many Government projects would fail if they were funded from the Politicians Pension fund, rather than the treasury itself? The fund takes a portion of the success, if there should be any and of course it takes the fall if it doesn't work out.
Would those ministers be so eager to make changes to just 'leave their mark' would they be so eager to agree budget increases?
Of course any Government Contract should NEVER be given to any company where a Politician or close family member of said politician is a director of that company, but I am sure there are those on here who quite like corruption as long as its 'their team' doing the corrupting and will mark this down. Yet, HS2 trundles on at £B's per mile yet other countries seem to be able to do everything so much better and so much cheaper. Wonder why that is? Perhaps we should ask Matt Hancocks landlord....
Trouble is that you get some CEO who is total numpty, and believes that a computer system cannot be wrong. They don't believe or listen to a computer "expert" and don't realise that Shit In = Shit Out. Those responsible for the Post Office system should be awarded at least 10 years in the slammer.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
