back to article Post Office threatened to sue Fujitsu over missing audit data

The Post Office proposed suing Fujitsu over missing data from its audit trail that could be used in the prosecution of victims of the Horizon scandal, one of the greatest miscarriages of justice in UK history. The details emerged today as the public inquiry into the scandal — which saw 736 managers of local Post Office …

  1. Zibob Bronze badge

    It a wonder why this happens

    "Paula Vennells, Post Office CEO between 2012 and 2019, has previously defended the Horizon system, and said she relied on assurances from Fujitsu that it was "like Fort Knox.""

    *No you see they TOLD me it was safe*

    So you followed that up with an independent check right? Right?

    *No!? They assured me everything was fine...*

    1. David 132 Silver badge

      Re: It a wonder why this happens

      Ed Davey has adopted the exact same defence.

      “They TOLD me it was all tickety-boo, and I trusted them. It’s not like my job as Post Office Minister required me to hold them to account or scrutinize them or anything…”

      1. Anonymous Coward
        Anonymous Coward

        Re: It a wonder why this happens

        Oh no.

        That would have made the expensive catch up lunches very awkward.

    2. Martin-73 Silver badge

      Re: It a wonder why this happens

      These people need to be charged with malicious prosecution. They KNEW the system was faulty, this is documented. Saying 'oh i didn't know' is called bloody perjury!

      1. John Brown (no body) Silver badge

        Re: It a wonder why this happens

        You've heard of plausible deniability, yes? Everything is kept compartmentalised so no one knows too much!

        1. Martin-73 Silver badge

          Re: It a wonder why this happens

          yes but it's not plausible they KNEW

          1. gandalfcn Silver badge

            Re: It a wonder why this happens

            "yes but it's not plausible they KNEW." Indeed they did but rather a lot of people refuse to accept that fact. Both Fudge and the PO have admitted it.

            1. Martin-73 Silver badge

              Re: It a wonder why this happens

              missing comma made it ambiguous but you got it, for clarification, 'it wasn't merely plausible, they absolutely KNEW, fujitsu engineers were screaming about it.

    3. kulath

      Re: It a wonder why this happens

      Paula Vennells seems to have lied and also lied to the HoC committee. But I'm not sure why you think she should have asked for an independent check (do you get an independent check for every weekly report you get from your underlings - you will be out pretty quickly), EXCEPT that in her email asking for reassurance she says something like "I need to be able to tell <them> that the system is robust" - she wouldn't have written it that way unless she had suspicions that it was not robust, and if you are asking a question like that you shouldn't word it asking for the answer you want. In that case, it was the fault of the managers for not establishing the truth and telling her it, but which managers have the courage to do so - the likely outcome is that you get moved sideways to a non-job paying peanuts so you resign. They should all be prosecuted for perverting the course of justice, but their defence (as you can see in the Horizon Inquiry) is that the just don't recall.

  2. Mike 137 Silver badge

    Somewhat elementary?

    "despite a codified agreement stating that the audit trail should have a level of security such that it could not be altered or deleted, Fujitsu could make insertions and amendments into data"

    I always thought that audit trails were by definition supposed to be write once, then read-only. However I do remember a SaaS accounting system I was engaged to deliver the security for, where the developers allowed invoices to be altered and deleted after issue. So maybe there's some business principle I've missed?

    1. katrinab Silver badge
      Meh

      Re: Somewhat elementary?

      Usually it is a copy on write type system though, so the deleted invoice is still there and marked as deleted, along with the date/time and user id of the person who deleted it, and the amended details, if any added as a new invoice, with some sort of cross-reference between them.

      Then, a regular supplier/customer account statement would exclude the deleted items, but an audit trail query would show them.

      1. Martin Gregorie

        Re: Somewhat elementary?

        Here's an interpretation of how HORIZON is built that seems to follow from the contents of "HORIZON: WITN00620100 David McDonnell - Witness Statement_0"

        The components are:

        1) The prototype terminal that ICL showed to the POST OFFICE in 1999 was just an initial version of the terminals that would be installed in all Post Offices: the Witness Statement reads as though no HORIZON-specific mainframe code had been written at this time.

        2) The accounting and stock control subsystems were to be run on one or more ICL 2900 mainframes installed in Fujitsu's data center. I don't think these subsystems had even been specified at that stage (but I may well be wrong about this).

        3) Since the user interfaces will be installed in Post Offices all over the country and both accounting and stock control subsystems are running on mainframe(s) in the data center, the wired network need only be straight-forward BT cabling capable of handling serial message streams fast enough to manage the expected data volume. The Witness statement doesn't mention network encryption so I don't think it was used despite HORIZON being a financial network.

        4) The above setup is fairly straight-forward to implement as three self-contained processes along with two switching processes: one directs incoming messages to either the accounting or the stock control process and a second that returns acknowledgements and responses to the requesting terminal.

        That's job done except, as the Witness Statement says nothing about the messages passing between mainframe and HORIZON terminals being encrypted, we must assume they are NOT. This has consequences:

        a) My financial network designer's background says that you NEVER send plain-text financial messages over a network: even ATM networks are expected to be encrypted..

        b) Seems likely to me that the 'transaction adjustments' that Fujitsu are known to have made became super-easy once they realized that, with plain-text communications, anybody can plug in an extra terminal, say, in the data center and freely alter any cash or stock balance that they feel like messing around with simply by injecting unauthorised messages into the inbound message stream.

        1. Eclectic Man Silver badge

          Re: Somewhat elementary?

          If the messages between the Post Office terminals and the Horizon mainframes were not encrypted, how could anyone possibly claim that Horizon was 'perfectly secure' and 'robust'?

          The 'professional' financial industry (as in regulated by government organisations such as in the UK the Bank of England and the Financial Conduct Authority, and in the USA the SEC) are pretty keen on mandating encryption between separate entities. So one can only assume that because it was considered internal Post Office communication, nobody thought that encryption would be necessary. I wonder if anyone told the comms supplier (Probably BT in the UK) what their network would be carrying?

          The more I read about Horizon, the more convinced I am that it will be one of the main 'horror stories' in Computer Science degree courses.

          1. TV nerd

            Re: Somewhat elementary?

            The messages were secured by a cryptographic signature. They do not need to be encrypted to be robust. The signatures prevent messages from being amended.

            Computer Science courses should teach this far better …

        2. gryff

          Re: Somewhat elementary?

          I doubt that 2900 series mainframes would have been used - they dated from the 1970s and were long out of production and support.

          Might have been a similarly named UNIX server from the Bracknell end of the ICL "empire, but they were all DRS +number..e.g. DRS3000, DRS6000 and also five years off the cutting edge.

          But given the state of Horizon...anything could be true...

        3. TV nerd

          Re: Somewhat elementary?

          You miss the fact that the branches were “offline” and the transaction record there was “replicated” to a mirrored set of servers in a data centre.

          There was no ICL VME as part of Horizon (ICL’s 2900 series having been end-of-life in the 1980s).

          There were links from the Post Office data centres to ICL VME systems at the Benefits Agency data centres - but these systems were not ever part of Horizon.

          Escher’s Riposte system handled the replication process over point to point links (normally ISDN).

          Your points a) and b) are simply incorrect.

    2. elsergiovolador Silver badge

      Re: Somewhat elementary?

      where the developers allowed invoices to be altered and deleted after issue.

      manager> Why can't I edit the invoice?

      developer> Invoices can't be edited, it's in the requirements. You can void the invoice and create a new, correct one.

      manager> Why can't I just edit it?

      developer> For many reasons - audit and compliance, fraud prevention, data consistency and so on

      manager> Are you saying that I could try to commit fraud?

      developer> No, but other users could. It's a possibility.

      manager> So why don't you enable editing just for me and other senior people?

      developer> Because this is not in the requirements.

      manager> Can you add it to the requirements?

      developer> No. That is not my responsibility.

      manager> Okay, I just added it. Can you make it editable for tomorrow?

      developer> I think such change should have legal sign off. Besides, I am in the middle of working on something else.

      manager> Who do you think you are?

      developer> I am a developer

      manager> I am a manager and I just changed the requirements and put the ticket on top of your list. Get on with it.

      developer> I can't.

      manager> Right. You are suspended then under further notice. Go home.

      developer> I am at home.

      1. katrinab Silver badge
        Pirate

        Re: Somewhat elementary?

        Make it "editable", but have it void the old invoice and post a new edited version in the backend.

        The manager is happy because it takes fewer steps to "change" the invoice. The auditor is happy because they can see that the manager did that.

        1. elsergiovolador Silver badge

          Re: Somewhat elementary?

          manager> Whenever I edit the invoice, the invoice number is changing. Sales notified the client for unpaid invoice, but he claims he paid and sent us a link, though I get 404. Can you go to transactions log and edit the payment for 90020124 invoice to be applied to 9002124 invoice that is unpaid?

          developer2> No, we cannot edit the transaction log. We can reverse the transaction so the client gets refunded and then they can pay the correct invoice again.

          manager> Why can't we edit the transaction log? It seems like we've been there.

          1. katrinab Silver badge
            Pirate

            Re: Somewhat elementary?

            Invoice number doesn’t change, transaction number, which isn’t shown on the invoice does.

            1. elsergiovolador Silver badge

              Re: Somewhat elementary?

              manager> Our customer service just sent me a complaint from the client. He claims he paid 90022724 invoice, but apparently our system sent him a reminder that invoice has not been paid fully and his account has been suspended. He sent us a PDF with the invoice and screenshot of his bank statement. I see the invoice is for £1500 + VAT, but his .PDF shows £1400 + VAT.

              developer3> Ah yes, Steph has changed the invoice to £1500 as from the note I see the client is over the promotional period. She must have edited it after it has been sent to the client, but before they paid.

              manager> We shouldn't let people edit the invoices!

              developer3> I heard that somewhere...

    3. Necrohamster Silver badge

      Re: Somewhat elementary?

      ”… where the developers allowed invoices to be altered and deleted after issue. So maybe there's some business principle I've missed?”

      I came across a point-of-sale application about a decade ago that was used by the hospitality industry…pubs and restaurants mostly. One of the implemented feature requests allowed transactions to be removed from the database and subsequent transactions to be re-numbered. The purpose? Tax evasion.

  3. Anonymous Coward
    Anonymous Coward

    Blockchain

    Would have made this impossible.

    #Justsaying

    "despite a codified agreement stating that the audit trail should have a level of security such that it could not be altered or deleted, Fujitsu could make insertions and amendments into data"

    1. Anonymous Coward
      Anonymous Coward

      Re: Blockchain

      Blockchain within one entity just means that the one entity could rewrite the chain from scratch. There isn't anybody else to check that they haven't.

      1. Anonymous Coward
        Anonymous Coward

        Re Blockchain within one entity

        isn't blockchain ...

    2. sev.monster
      Facepalm

      Re: Blockchain

      Once again cryptobros injecting "muh cryptography" into the conversation without offering any sort of viable implementation plan and without the understanding and foresight that there are simpler, less computationally expensive ways to achieve the same goals. You even used a hashtag.

      I worry that this is bait or parody. But one cannot be sure anymore.

      1. katrinab Silver badge
        Trollface

        Re: Blockchain

        The correct way to make sarcastic comments is to use this icon.

        1. Korev Silver badge
          Trollface

          Re: Blockchain

          Or even /s

    3. Eclectic Man Silver badge

      Re: Blockchain

      There was a way of protecting an audit trail by having cryptographic hashes of each entry chained, so that any editing of a previous record would show up when the subsequent hashes were checked. Not called 'Blockchain'.

      1. R Soul Silver badge

        Re: Blockchain

        That may well be true.

        But you miss the point. Blockchain is the magic-pixie-dust-of-the-day that fixes everything.

        1. Anonymous Coward
          Anonymous Coward

          Re: Blockchain

          So very true.

          I got rich just by thinking about Blockchain!

          I lost it all when I woke up though…

    4. Random person

      Re: Blockchain

      Blockchain had not been invented when the Horizon system was developed and rolled out.

      > The British Post Office scandal is a series of miscarriages of justice which, between 1999 and 2015 ...

      > The Horizon accounting system was developed by ICL Pathway, owned by the Japanese company Fujitsu. In 1999, the Post Office started to roll out the new software to its branches and sub-post offices ...

      > Horizon is the outcome of the Pathway project, and a procurement process that commenced in August 1994 ...

      https://en.wikipedia.org/wiki/British_Post_Office_scandal

      > The domain name bitcoin.org was registered on 18 August 2008.[16] On 31 October 2008, a link to a white paper authored by Satoshi Nakamoto titled Bitcoin: A Peer-to-Peer Electronic Cash System was posted to a cryptography mailing list

      https://en.wikipedia.org/wiki/Bitcoin

      Note that both sides of the transaction and the ledger are under the control of the Post Office.

      It is likely that at least some of the problems would not have occurred if the transactions were ACID compliant. Computerphile did a video about this, search for "Post Office Horizon Scandal - Computerphile" for more information.

      If are trying to be ironic, I suggest that you mark your posts as such because it is very likely that cryptobros would say exactly the same but without irony.

  4. Gomez Adams

    The one time I was involved in a "loss" of audit trail records was when about 14,000 transactions appeared to be missing from the audit trail tapes and was detected by the end of day reconciliation runs (which showed they were doing their job). We investigated and found the missing transactions had been written beyond the EOT (end of tape) marker and so were being ignored. It was down to me to swiftly write a quick and dirty utility to retrieve said transactions and build a copy of the reel in question patching the missing transactions on the end plus rebuilding the directory reels to reflect the new tape numbers used (master and backup). Not the longest or hardest all nighter (it took a few hours to run the rebuild while I kipped under the desk) but one to remember.

    1. Andy the ex-Brit

      Looks like a good "On Call" story.

    2. Graham Cobb Silver badge

      Great work! But, surely, you were being made to fix the problem the wrong way. If you could do that, in that emergency, then someone else might have been able to do the same thing to hide fraud?

      Surely an audit trail should have a documented, and highly visible, way to fix errors (with some special type of transaction or something) so that the fix is forever visible in the trail? Isn't that the point?

  5. cantankerous swineherd

    fwiw evidence in the trial was that the arq logs were worm, but the management information system could be mucked about with at will

  6. nematoad
    Mushroom

    Maybe not.

    ...bugs, errors and defects in the system had been known about by all parties

    No, the poor bloody sub-postmasters and sub-postmistresses didn't and yet they were forced to pay up, lost their businesses, homes,their liberty and lives.

    Others got CBEs, bonuses and promotions.

  7. Anonymous Coward
    Anonymous Coward

    So: fraud, perjury, misconduct in public office, false accounting, libel, false imprisonment. That'll do for starters, though as a special for Paula Vennels: bearing false witness. Anything else to prosecute the Post Office and/or Fujitsu boards over?

    1. Martin-73 Silver badge

      The government and post office should be declared null and void and deported to Rwanda, but that is not really an option, the Rwandans have suffered enough

    2. mikepren

      Perverting the course of justice

      1. Anonymous Coward
        Anonymous Coward

        AC here. Ooooh yes, thanks. Missed that off in the excitement*.

        *Most get out more.

    3. Eclectic Man Silver badge

      Re: other offences

      Well, under the Theft (amendment) Act 1996 , obtaining a money transfer by deception is a criminal offence punishable by an unlimited fine and / or a prison term not exceeding 10 (ten) years on first conviction. The on first conviction bit is important because it applies even if the person in question has a previously unblemished record.

      It is also made clear that you do not have to personally directly benefit from the transfer, and the transfer can be in any form (cash, cheques, bonds, direct debit, credit etc.) and does not even have to be of the amount the perpetrator tried to get transferred. Asking deceptively for a transfer of £10,000 and only getting 1p still counts.

      But then the prosecution would have to take this case seriously ...

  8. Necrohamster Silver badge

    ”The details emerged today as the public inquiry into the scandal — which saw 736 managers of local Post Office branches wrongfully convicted of fraud…”

    It’s amazing that nobody looked at those 736 cases of “fraud” and thought “736? This is way higher than any incidence of fraud we’ve seen in the past. Could there be an explanation other than fraud?” Why didn’t the Post Office go through the financials with a fine tooth comb?

    1. Anonymous Coward
      Anonymous Coward

      Senior Post Office management saw it as confirmation of their long-held belief that they were being ripped off by (sub-)postmaster proles. Why else would said managers be missing their bonus targets ?

    2. Martin-73 Silver badge

      The post office is or was govt owned at the time. UK govt is synonymous with incompetent wankers

    3. notyetanotherid

      It is rather worse that 736. That is just those convicted and does not include those who settled the money that it was claimed they owed without it going to trial. Last month on gov.uk was posted this: "The government has today [Tuesday 19 December] announced that circa £138m has so far been paid out to over 2,700 claimants across the three Post Office compensation schemes.".

      For the last few years, the PO has had around 11,500 branches so basically the management appears to have been happy to accept that around a quarter of its branches' managers were on the fiddle without, as you noted, anyone seemingly questioning whether there might be an alternative explanation...

  9. Anonymous Coward
    Anonymous Coward

    Rebuild the old London Bridge

    Buy some new pointy spikes….

  10. anxiousmac

    Fort Knox

    I'm presuming that some people at Fort Knox can access the vault; it's thieves they want to keep out.

    Rather like the senior developers at Fujitsu, though their superuser security protocols seem to have been a bit ..er relaxed.

    Apt analogy, just misinterpreted by those it was intended to mislead...

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like