back to article Scottish cops dangle £6m for help understanding 160TB treasure trove of structured and unstructured data

The Scottish Police Authority is on the hunt for virtual data warehouse and data lake providers as part of a £6m tender intended to help the public body derive some value from the huge volumes of data it is sat on. The authority said it stores around 100TB of unstructured data alongside a further 60TB of structured data that …

  1. Tubz Silver badge

    160TB of data, and % that is unlawful, yet Police ignore UK and at the time EU rules and official guidelines?

    1. m4r35n357 Bronze badge

      My advice:

      1) sit on it, it might hatch

      2) collect more data

  2. john.jones.name
    Stop

    a single rack

    not exactly a data lake if your requirements is a single rack of storage and CPU even if data doubles this is not high amounts of data infact do they not have camera's in those police cars ?

    1. dave 81

      Re: a single rack

      They do, but that video is deleted after 30 days. Or accidentally deleted sooner if someone requests it showing the police doing something wrong. The number of hard disk errors on police computers is a weird and inexpiable anomaly,

      1. Arthur the cat Silver badge

        Re: a single rack

        The number of hard disk errors on police computers is a weird and inexpiable anomaly,

        It looked at me funny, so I Tased it.

        1. Anonymous Coward
          Anonymous Coward

          Re: a single rack

          Arthur the cat,

          More like:

          The screen went *black* so I tased it, then when I was carrying it out for repair it *fell* down the stairs .... twice !!!

          :(

          Joke .... but not a joke really.

      2. Anonymous Coward
        Anonymous Coward

        Re: a single rack

        As is the way that suddenly something isn't a crime or doesn't have a good potential for success when committed by a serving officer of sgt or above....

        (yes I was threatened with violence in the street by a plod scotland Sgt for daring to file a complaint about offensive behaviour in my parents street and yes I had a witness....fiscal closed ranks with the cops and refused to justify their decision "we don't have to explain ourselves to the general public", said same cop "retired" with full pension after battering a suspect in custody and every excuse under the sun was used to get him off the charges....barely)

        First step towards a police state, hasn't happened yet due to lack of personnel, a genderal laziness from rank and file and utter incompetence.....

    2. Danny 2

      Re: a single rack

      They do not have cameras in police cars, except in a few dedicated traffic police cars.

      A British Transport Police car cut me up late at night last year, would've crashed into me if I didn't brake, and I had footage of it and they didn't. I immediately drove to a cop shop to report it. I didn't press charges but was assured by a BTP Sergeant that the driver had been reprimanded as their drivers are trained to be better than 17 year old boy racers.

      tl/dr - most UK cop cars don't have cams

      1. Anonymous Coward
        Anonymous Coward

        Re: a single rack

        That means nada, would have been better making it a formal complaint, otherwise they don't do jack all...and that comes from experience....

        1. Danny 2

          Re: a single rack

          My parents suffered 13 police raids/visits in 5 years because of me. They knew where I was living and that I'd walk into a cop shop if they phoned and asked, but they also knew bullying my parents was the best way to pressure me.

          After the 13th I put in a formal complaint, and they went through the motions but ended up threatening to visit my parents for evidence if I pursued the complaint.

          In my experience the best thing to give abusive cops pause for thought is a Data Protection request. I asked for every time, date and reason they had been to my parents house, that cured it. Some clear figures you can just pass to a Sheriff or Judge.

  3. steviebuk Silver badge

    Most probably junk

    report1.docx

    report1_1.docx

    report1_11.docx

    report1_111.docx

    and so on from all the documents that they were trying to save that had the same name. Or they did a minor update to it.

    1. SloppyJesse

      Re: Most probably junk

      All with high res images of their logo embedded.

  4. Anonymous Coward
    Anonymous Coward

    Doesn't this just confirm everybody's suspicions that these government bodies collect mounds of data for the sake of it, without any plan of how to make meaningful sense of it?

    Their policy seems to be "Record everything and we'll figure out what to do with it later"

    1. Not Enough Coffee

      This reminds me of some police bodies that would operate as "arrest everyone and we'll figure out what to do with them later".

      1. BebopWeBop

        Everyone is guilty of something: Standard Polis Protocol.

    2. Anonymous Coward
      Anonymous Coward

      People are scared to delete, records management isn't always clear on every document particularly the likes of healthcare where whether you delete something isn't just down to whether someone is living or dead but also what was wrong with them, events they were involved in e.g. child sex abuse scandal - all of those are being kept indefinitely as the Scottish Government drags that on.

      It's not an easy task, especially as the people who generated a lot of this have since retired and organisations have merged over the decades etc.

  5. Danny 2

    It's a trap!

    Police forces make for bad clients in the best of times with simple, harmless products. Sticking all their data onto the cloud is just inviting a lengthy prison term when it goes tits-up and a scapegoat is required. Although it would be fun to browse all their data, and you could sell that repeatedly to crims for many times the value of the contract. You’ve gotta ask yourself one question: “Do I feel lucky?” Well, do ya, punk?”

  6. Version 1.0 Silver badge

    This will not help

    It sounds like they want to replace detectives with AI - they think it's easy to use AI to catch the criminals even if AI can't accurately identify them.

    1. hoola Silver badge

      Re: This will not help

      The consultancy and marketing BS lost will be wetting themselves with this. You can do some basic scanning to find out the contents of some documents that may give you a clue as to what is in it or where to move it.

      Ultimately it does not matter how much tech or money you throw at these things the only person who can actually identify the data is the person(s) that put it there.

      All the same 160TB is not that much.

  7. Doctor Syntax Silver badge

    And the moral of the story - structure your data when you capture it. It's easier that way.

    Otherwise, there's always grep.....

  8. Claverhouse Silver badge
    WTF?

    'Police Scotland' sounds like 'Team America '.

    1. Anonymous Coward
      Anonymous Coward

      Well it was your lot the "Scottish National Socialist Party" aka the Natzis who brought them into being and created a modern state police, just like Hitler merged Germany's police to form the Gestapo (Gestadt Polizei), Along with the party loyalists on the SPA who are fuming that parliament took away their oversight powers for...failing to do any oversight and just rubberstamping anything the police asked for and then deriding anyone who sounded the alarm..

      1. BebopWeBop

        The Gestapo was not formed by merging individual German police forces I think you will find. It was formed by Goring combining Prussian security forces (not the same thing) and then extending it

      2. Anonymous Coward
        Anonymous Coward

        Careful, your prejudice is showing.

        (Two or more can play behind the anonymous coward fence)

      3. Kapsalon

        Geheime StaatsPolizei

        https://en.wikipedia.org/wiki/Gestapo

  9. Aussie Doc
    Pint

    Oops.

    "...hunt for virtual data warehouse and data lake providers..."

    My tired eyes read that as 'data leak providers' and thought somebody was going to leave lots of USB drives lying around on buses or something.

  10. Anonymous Coward
    Anonymous Coward

    Out to tender

    Anyone may apply for this tender, as long as your company name starts with C and ends in A, and as long as the gratitude for the tender remains alive and countable until the day after I retire, said a senior PS wonk

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like