Re: Why, oh why?
Because that's not how government works. There are many civil servants who are just as frustrated with the constant mess as you or I or any other member of the public is.
Ministers want to show they did something (like Grant Shapps insisting that his name be put on what was Keith Williams' work, i.e. the Williams review of the railway industry and strategy as it stands). They want to implement whatever warped vision their party has of what the world should be like. And when they get their mitts on what they really want to mess with, God help the civil servants, because they're tasked to implement what the minister wants, regardless of whether they think it's the worst idea ever or not.
The permanent secretaries (i.e. heads of department) are there to advise the minister of the sense (or foolishness) of their ideas. That's why you've seen several of them depart (because either they were fed up with being spoken to like children, or felt that they were actively being sidelined) during the current party's rule. These people are there to keep order and avoid chopping and changing things at the whim of some minister who has absolutely *zero* clue about what their brief is, what it entails, and what they're meant to do.
Unfortunately, they also view people who come from the commercial world with a bit of a suspicious eye ("what are you doing here, and what's your ulterior motive"), which is both wise and a shame because while there *are* people who go in to see how much they can score for their friends in the commercial world (these tend to be the ones 'suggested' by the ministers or secretaries of state, or their minions), there are also a lot of people who believe that the department/ministry is getting well and truly shafted and needs help fending off the former. The sad thing is that the latter are in the distinct minority, and they tend to come up against consultancies who like to charge lots of money.
The end result: Chaos and what you could consider mismanagement.