back to article City council Oracle megaproject got a code red – and they went live anyway

Birmingham City Council, a massive local authority in England, went live with new Oracle Fusion ERP software in April last year knowing the system was likely to fail. Since the introduction of the replacement for an ageing SAP financial management system, the council cannot close its books and file financial reports. The …

  1. b0llchit Silver badge
    WTF?

    Oracle declined to comment.

    But they will be sending the lawyers because Oracle has not been paid since the system refuses to pay bills sent through the new system. The city being bankrupt is no excuse and Oracle is eyeing replacement payment in other currencies, such as all the land and buildings of the city centre. In the meantime, a statement from Oracle counsel was received to say: "The radius of Oracleham city ownership will extend for every day we are not properly compensated.".

    The city of BirmingOracleham declined to comment.

    1. jay_bea

      Had Oracle decided to comment, I suspect it might have been "Kerching".

    2. J.G.Harston Silver badge

      Is Birmingham Council allowed to steal loads of other people's property in order to pay off their own debts?

      1. b0llchit Silver badge

        Have you read the quantum dots at the bottom of your contract?

  2. Anonymous Coward
    Anonymous Coward

    A drop in the ocean

    Looking at how much Birmingham have frittered on noisy, slow trams and the wider messing up of Brum city centre, I suspect the Oracle overspend is not really significant.

    1. A Non e-mouse Silver badge
      Mushroom

      Re: A drop in the ocean

      This is my beef with El Reg's articles on the Birmingham Bungle: They present the story as the failed Oracle implementation has brought down the council. My reading of El Reg's articles are that the failed ERP project is just one piece of the puzzle and not the catalyst. If the ERP project has suceeded would Brum council still have gone backrupt? Yes. The £760 million equal pay claim is gonna be a big part of the failure to balance the books.

      1. The Mole

        Re: A drop in the ocean

        Without the ERP system it seems like they have no way to even know if the books are balanced let along work towards balancing them. Its not the only factor but is no doubt a big contributory factor and example of how the council was running everything. As they have said on other stories it isn't the whole but it is the part of the story the technical audience of El Reg are most interested in.

        1. SVD_NL Bronze badge

          Re: A drop in the ocean

          Yup, the cost may be a small contributing factor, but i'd wager not having a functioning ERP system doesn't help with identifying and resolving your financial issues...

      2. Lurko

        Re: A drop in the ocean

        "The £760 million equal pay claim is gonna be a big part of the failure to balance the books"

        The £760m is an estimated current liability, and is on top of £1.1bn in equal pay settlements already made by BCC in the last ten years. In terms of going bust, BCC's £2.5bn pension deficit doesn't seem to have got much of a mention, but would appear to my causal eye to be the biggest chunk.

        1. Graham Dawson Silver badge

          Re: A drop in the ocean

          The irony is that Brum council rather screwed themselves over in the equal pay matter. The city's officials and press office made a huge fuss about equal pay for years, loudly decrying the "inequality" of lower wages for certain, female-dominated employment sectors when compared to others that tended to be male dominated, regardless of the differences in work requirements and risks.

          All of the council's criticisms of other organisations' practices became evidence of their own malfeasance in the claims. They could hardly argue that they were providing equal pay when their own pronouncements said otherwise.

          A few small pay rises and a little more circumspection would have saved them hundreds of millions.

          1. fnusnu

            Re: A drop in the ocean

            ^^^^ would have saved the taxpayer hundreds of millions.

          2. Mooseman Silver badge

            Re: A drop in the ocean

            "loudly decrying the "inequality" of lower wages for certain, female-dominated employment sectors when compared to others that tended to be male dominated, regardless of the differences in work requirements and risks"

            Thanks for clarifying that you have no idea what wage equality actually means.

            1. Graham Dawson Silver badge

              Re: A drop in the ocean

              Tell it to Birmingham council.

      3. aerogems Silver badge

        Re: A drop in the ocean

        Roughly 80m pounds may not cover everything, but it would certainly go a long ways towards covering any other shortfalls. Just sayin'.

      4. sketharaman

        Re: A drop in the ocean

        +1. My heart bleeds to see Oracle being made the Fall Guy here (Disclosure: Ex-Oracle employee).

    2. keithpeter Silver badge
      Trollface

      Re: A drop in the ocean

      Trams? Talk to Andy S not BCC.

      PS: I'm quite relieved at the moderate speed the Metro runs at in the city centre myself. Try Manchester just outside the art gallery one day - but be prepared to jump quick.

      1. Anonymous Coward
        Anonymous Coward

        Re: A drop in the ocean

        "Trams? Talk to Andy S not BCC."

        The trams kicked off back around 1999, the West Midlands Mayorality started in what, 2017? There was of course Centro (Wumpty as was) doing the wider West Mids "thinking", but Wumpty was always dominated by BCC.

        Seems it's a local government thing - Edinburgh, Sheffield, Manchester, Nottingham, Brum, they're all keen on these ludicrocusly expensive railways down the street, as a form of local government jewellery. Birmingham's latest "in progress" acheivement is the East Side extension, building 1.4km or track for over quarter of a billion quid, and that's to connect the city to the misbegotten HS2 station at Curzon Street. Save ten minutes on the journey to London, but only after fifteen minute tram rattle to Curzon Street, genius. Then zip to The Smoke in not time and find yourself at the urban desert that is Old Oak Common......maybe it's not just a local government thing.

        1. keithpeter Silver badge

          Re: A drop in the ocean

          Cost per mile is bonkers agreed. Same pattern as HS2 - start building while consultations going on &c and adjust as you go so more cost.

          The point I was making is that Metro is a combined authority responsibility now. Not everything that happens in Birmingham is BCC.

          Another e.g. Clean air zone = central government mandated air quality standards.

          (HS2: I have always been somewhat sceptical of the journey time argument. The capacity argument used to make sense when the NE and NW segments were still in the plan but now its just embarrassing).

          1. Tim Almond

            Re: A drop in the ocean

            The capacity argument hasn't made sense for years. We know what the predicted growth was in 2013 when the planners drew a straight line from the previous 15 years of rail growth.

            But business rail barely grew and then a few years before COVID it went into reverse. What growth there has been has been leisure rail which isn't worth building for.

        2. Ian Johnston Silver badge

          Re: A drop in the ocean

          Passengers like trams. And while the original Edinburgh system was undoubtedly a Category A, chrome=plated clusterfuck, the Nottingham system was a model of cost-effective planning and delivery. What is it about trams which so upsets Daily Mail readers?

    3. Tim Almond

      Re: A drop in the ocean

      Trams: all the cost of a train with none of the flexibility of a bus.

      1. Ian Johnston Silver badge

        Re: A drop in the ocean

        Much higher capacity, quieter, runs on electricity, more popular with travellers.

  3. BOFH in Training

    Why did they kill off the existing SAP system before Oracle system could fully take over?

    1. Derezed

      Presumably someone’s bonus was tied to the retiring of old and implementing new. Job done!

    2. Doctor Syntax Silver badge

      End of contract with SAP and/or the H/W it was running on?

      Train crashes depend on the momentum of the train.

      1. aerogems Silver badge

        I'd guess more that the version of SAP they're using is EOL and the amount of money SAP wanted to continue supporting it just for them was too high. Now why they went with Oracle instead of migrating to a newer version of SAP is anyone's guess. It probably involved a lot of "business dinners" at really nice restaurants and "business meetings" held at luxury boxes at sporting events.

        1. Lonpfrb

          SAP ECC 6

          SAP ECC 6 on NW 7.5 EOL 2027 so that's not a technical blocker. Intention to Fit to Standard is aligned with S/4 Green field implementation so no worse in general to a Oracle green field implementation.

          So another reason does appear likely...

    3. James Anderson

      Because there is no easy migration path from one SAP release to the next. So its often wisest to delay upgrades as long as possible. The trouble is when your current system is about to go out of support you are three or four releases behind.

      In many cases all the essential customisation has to be redone from scratch.

      Upwards compatibility? "Bitte überweisen Sie die erhöhte Lizenzgebühr auf dasselbe Konto".

      While I applaud the decision to escape the SAP forced upgrade nightmare, it was a case of out of the frying pan into the fire.

      1. Lonpfrb

        SAP Upgrade

        "no easy migration path from one SAP release to the next" is generally an untrue allegation. Rather huge effort has been put into the upgrade path for customers.

        HR line of business will have mandated legal change upgrades so finance is required to align which means no case for delay to upgrades. Configuration will be fine, but custom code will depend on the continued use of sap enhancement concepts to be upgrade safe, as we built it, and not creating technical debt through poor practices.

        However SAP has been very clear that ECC 6 is EOL 2027 and S/4 Hana is the strategic direction but not a succession product with an automated upgrade path. The BCC decision to Fit to Standard is perfect for S/4 green field implementation.

        New infrastructure is required for S/4 but by now the infra would be worn out despite several hardware refresh actions e.g: 20 years / 5 years depreciation cycle.

        Whichever ERP is chosen the implementation partners experience and understanding is key to enable making good (low TCO) decisions. The leadership and discipline to Fit to Standard is not easy, and that failure is how to over run by 4 times.

        1. James Anderson

          Re: SAP Upgrade

          A UK council could not possibly, practically or legally “fit to standard”.

  4. tmTM

    Flip-Flop

    oh look, another large public body that can't make it's mind up when it comes to large IT investments.

    Perhaps if you had a properly researched plan in place before blindly ploughing ahead you might not find yourself in this gigantic mess.

    1. A Non e-mouse Silver badge

      Re: Flip-Flop

      I'm gonna say that it's probably management (or the councillors) that flip-flopped. The article says the original plan was to change process to match Oracle. I bet someone got cold feet at trying to push through change (or someone "important" complainted). So they then swapped to hack customise Oracle to match their processes. It's that core change mid project that probably did it in.

      It's always the mid-project changes that kill you.

      1. Richard 12 Silver badge

        Re: Flip-Flop

        Or even more likely, they discovered that "standard" Oracle doesn't work for anyone at all, ever, and that customisation is absolutely necessary.

        I'm not aware of a single Oracle (or SAP) ERP installation that is "standard". They're all customised to a significant extent.

        1. veti Silver badge

          Re: Flip-Flop

          Indeed, "out of the box" software at that level is a myth, and Oracle should be prosecuted for deceptive sales practises for even allowing the phrase to be used.

          1. SVD_NL Bronze badge

            Re: Flip-Flop

            I know someone who worked as a sales consultant for a certain ERP software vendor.

            He described his job as follows:

            'I just say "yes the software can do that" and then tell the technical guys the software has to be able to do what i promised"

    2. ecofeco Silver badge

      Re: Flip-Flop

      More research than the right family vouched for it and the right brown envelopes were exchanged?

      How plebeian.

  5. Missing Semicolon Silver badge
    Unhappy

    Product not suitable

    It is entirely possible that the people who signed up for the Oracle product actually did not fully understand what it does, and in what ways it does not implement the (probably legally) necessary financial reporting and integrations that the SAP system has.

    So the decision to go customised may well have been because it is legally required.

    In fact, the Oracle system should not have been purchased at all.

    And, "The council's political and administrative leadership" will experience exactly no hardship and no censure for being so incompetent.

    1. Anonymous Coward
      Anonymous Coward

      Re: Product not suitable

      Yes well - therein lies the problem. Being involved in the early stages of prepping for procurement of an ERP trying to get the business to clearly articulate what they do beyond "it's complicated and you wouldn't understand" is an interesting experience.

      1. Brewster's Angle Grinder Silver badge
        Boffin

        Re: Product not suitable

        "...it's complicated and you wouldn't understand..."

        I can do Quantum Field Theory and General Relativity. Try me.

        ("It's complicated and you wouldn't understand" being code for, "it's complicated and I don't really understand it - certainly not well enough to explain it to you and I know I'll end up looking a fool.")

        1. gerryg

          Re: Product not suitable

          Here's a quantum of advice "If you can't explain to a five yr old you don't understand it" Richard Feynman

          1. Ian Johnston Silver badge

            Re: Product not suitable

            Over many years working in a academia, it has been my invariable experience that the more eminent the expert, the more lucidly and concisely they can explain their work to a lay person. Waffle and circumlocution always indicate third-rate minds.

        2. Bebu Silver badge
          Windows

          Re: Product not suitable

          《"...it's complicated and you wouldn't understand..." I can do Quantum Field Theory and General Relativity.》

          I suspect Brumy City Council Accounting Theory isn't renormalizable.

          A distant relation to Douglas Adam's Bistromath perhaps.

          How is it that Boris Johnson and his defective crew aren't responsible for this fiasco?

          1. Brewster's Angle Grinder Silver badge

            Re: Product not suitable

            That explains why they went bust. It if was renormalizable, even Oracle would have solved it by now.

  6. Anonymous Coward
    Anonymous Coward

    On the RAG

    Ahh highlighted as red in the red/amber/green report? That'll be printed in black and white because "colour is too expensive" so will have been missed for many consecutive months.

    1. Mark #255

      Re: On the RAG

      Having reached an age wherein I have run out of fucks to give about how I'm perceived, I now loudly decry red/green combos as an accessibility failure (because I'm colour-blind), with the back-up argument of "what if it's printed in black and white?".

      Red/amber/[yellow]/white are the colours to use, with solid/dashed/[dotted]/no borders

  7. mikus

    Has there ever been a proper ERP implementation that hasn't run 5x or more in cost than intended? I've never seen or heard of one, pretty much anything Oracle or SAP is simply destined to be a disaster when put toward every weird and wacky business or government use case.

    1. old_n_grey

      "I've never seen or heard of one"

      Do you think that might be because no-one ever bothers to report successful implementations? I used to be an Oracle functional consultant and worked at three large system integrators and I can say without fear of contradiction that none of the projects on which I worked went five times over budget (obviously I do realise that the hyperbole was not meant to be taken literally). Did any large projects go over budget? Yep! Sometimes because we, the integrators, got something wrong and others because the client kept changing the requirements - "I know that is exactly what we asked for but it's not what we wanted"! But I also worked on several that went live on time and on budget. But who wants to read about those?

      The other thing about Oracle and SAP projects is that they are likely to be the large projects, with complex requirements and so more likely to be problematic.

      Mind you, I am trying not to defend Oracle the corporation, who used to be only too happy to agree that the system could do everything the potential client wanted. On one particular project, that cavalier attitude to real life caused me an enormous amount of grief when I had to tell the client that the system really did not do what Oracle said it would. Of course, not all integrators were squeaky clean either. Nor indeed were clients who sometimes preferred a lower initial cost based on a set of requirements that had had functionality descoped that would need to be added during the project via change control. And, unfortunately, change control never comes cheap!

      1. ecofeco Silver badge

        Feature creep is the bane and often death of every project, service and product, from tiny to large.

        The Rube Goldberg Effect is the default setting of everything.

      2. A Non e-mouse Silver badge

        Vendor, lie???

        Shocked! Shocked I tell you!

        I was using a vendor's system for about five years and we decided to go to tender for a replacement as the vendor couldn't provide certain features. We made sure to detail these features as mandatory in tthe tender. Our incumbant bid claiming they could do these missing features. I challenged them on this and they said "Ah, we added those features just now." Bollocks they had. I asked legal about this contradiction and legal said there wasn't anything we could do: We had to believe what they wrote in the contract.

        Fortunately the vendor lost the tender due to other flaws in their bid.

  8. cookieMonster Silver badge
    Holmes

    At this point in time…

    You would have thought that after years and years of failure and countless billions wasted, SOMEONE would have figured out that it would be better and cheaper to hire a shed load of developers and build custom solutions

    1. Lurko

      Re: At this point in time…

      If you're not competent to buy and customise software for some very common use cases, what chance of properly managing a self-build?

      If you have the right people it is indeed fairly simple to create a new accounting, resource planning, ERP, logisitics, or similarly complicated systems, I have worked with people who built one of the most modern EPOS systems for large companies from scratch, with five nerds in a garden shed*. But if you're a ponderous local government body? No chance. No chance of getting the business case right. No chance of getting the user requirements right. No chance of mapping the adjacent systems and interfaces. No chance of correctly specifying the reporting requirements. No chance of knowing who to turn to and manage the dev team. No chance of being able to make informed choices when the programme manager has to get direction. No chance of leaving them alone to build the thing. No chance of sticking by the spec.

      * that system is now (whisper the name) in the Oracle stable.

      1. James Anderson

        Re: At this point in time…

        Most of these ERP “out of the box”, “implement best practices” perhaps work out of the box and implement best practices for a medium sized mid-western manufacturer. But for every body else it involves tons of customisation.

        In effect you are writing a bespoke system in a really crap programming language.

      2. Graham Cobb Silver badge

        The world has changed

        And board priorities have changed. I know a company who's board used to be really engaged with the business efficiencies and competitive advantages IT were bringing.

        But in the last couple of years, all they care about is cyber-security. Which is probably right as ransomware is an existence-level threat to many companies. But it means that big suppliers, and big consultancies, are now highly preferred by the board - certainly over innovation. IT has become the new HR - just keep it running and don't bring us any problems.

  9. gerryg

    Theory of the firm, bear with me

    In the private sector eventually the internal transaction costs exceed the external transaction costs, limiting the size of the organisation, (Ronald Coase). That's one reason why there's more than one supermarket.

    Enterprise software is a sales pitch designed to convince large organisations they can lower internal transaction costs. In the private sector a firm has to cope with the decision or die. In the public sector they just keep spending and put it on the tax bill.

    1. James Anderson

      Re: Theory of the firm, bear with me

      Er companies have to spend less on their product than the selling price ….. who knew.?

      1. James Anderson

        Re: Theory of the firm, bear with me

        Not sure why the downvote.

        The original post was just a jargon filled complicated way of saying firms need to make a profit.

        You don’t need to pick obscure wards out of a text book to say this.

  10. ecofeco Silver badge

    They used Oracle?

    That's a red code nothing can fix.

  11. aerogems Silver badge
    Pint

    I feel their pain

    Living through a SAP migration (from another ERP) and it hasn't been quite as bad as this, but it's been plenty bad. Seems like today my inbox for tickets has been blowing up this week, and all for little changes basically because the consultant company they hired decided that, "Oh, you don't need to worry about that, someone else will do it" constituted as training. So, now everyone thinks everyone else is going to do everything for them.

    It's virtual, but it's the best I can offer for my fellows in frustration over in Birmingham. If not for a little thing called the Atlantic Ocean in the way, I'd buy a round and we could share war stories. ===============>

  12. Big Softie

    The fundamental issue with all the public sector disasters we see is that the people appointed to manage them are incompetent. They lack the skills, knowledge, ability, experience, track record, and training to do the job they're doing yet somehow get appointed to the position anyway. So whilst they are the immediate cause of the cock-up, the solution lies in tackling recruitment in the bigger picture. The same happens in the private sector of course but there corrective action has to occur within otherwise the enterprise will go under. In the public sector it's always the taxpayer who takes the hit whilst the mess continues by putting out the message "We're sorry, we made mistakes, we know have to do better, lessons have been learned...." And the band plays on. It's high time that society stopped treating the public purse as a bottomless pit to make the chosen few rich and failing to ensure cost-effective and successful delivery of the projects it bankrolls.

  13. Grinning Bandicoot

    What learning curve?

    Same song, same blues on ITS ONLY A MINOR TWEAK.

    And this to Megalodon.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like