Deleting 'innocent' DNA will cost £5m

Removing innocent people's records from the DNA database will cost almost £5m, the House of Commons was told yesterday. MP Diana Johnson asked crime prevention minister James Brokenshire how the batch loading system for getting records onto the national database worked. He explained that DNA samples are first converted into a …

[ShaggyDoggy]

Cheap

That seems very cheap compared with the cost of the database in the first place

[Steve Evans]

And...

We should get Blunkett and pals to pay for it, personally! For having the nerve to think keeping profiles for ever would be acceptable.

Maybe they should have a word with someone who knows some SQL, I'm sure they could do a few joins and delete the offending records.

[Andus McCoatover]

Price of justice, I suppose

Get it wrong and suffer the consequences. Isn't that the precept of the law?

[Steen Hive]

How much could they have saved

By not collecting outrageous amounts of DNA profiles on totally spurious grounds in the bloody first place?

[David Haworth]

is that for all, or each?

as the title says. if it's £5m to remove every innocent person's DNA then fair enough. if it's £5m per person to remove their DNA then I might suggest that someone needs to go back and look again at the bill.

[Scott Mckenzie]

Err...

Surely it's a case of:

DELETE FROM [DNA] WHERE [crime_committed] IS NULL

I'll do that for £2.5Million...

[Anonymous Coward]

better than that...

DELETE FROM [DNA];

problem solved.

[Andus McCoatover]

Would..

#rm -rf /*

do it? less than £5 million/character.

I'd do it for half the price.

('course, Id probably need to be of non-UK origin to pass the stringent security and P.V tests, but...)

Sorry, I've never used that command. I'm told it's surprisingly effective. Especially (given tomorrow's date) to leave it - as a text - on the BOFH computer first thing tomorrow morning...

[Annihilator]

Even better

DROP DATABASE [dbDNA];

[Sampler]

ahahahahahahhaaaa

You believe a government project was set up sensibly in the first place? What about support and maintainence costs, can't make them stupidly high if they could pass it on to any old schmuck!

[Anonymous Coward]

Bargain, but..

...you forgot to add VAT. :-)

But your point is valid....for such a simple procedure, how the fuck does this cost £5m????

Does that also mean that *adding* data to the table(s) cost £5m too???

[Field Marshal Von Krakenfart]

how the fuck does this cost £5m????

£4.28 to write the code

£118.72 to run the code

£4,999,877.00 in consultancy fees

[Shadowfirebird]

No.

Because that would imply that the database was well designed.

Given that this was a government project, it seems unlikely.

[Martin Gregorie]

SQL? What SQL

What makes you think this database is relational?

The only use of the database is to compare DNA profile X against all other profiles and print out the list of matches. This doesn't need a relational database, especially as the natural key, the DNA 'donor' name, isn't needed to do the search: it is only needed to delete profiles and that was never a requirement.

Given all the above, why pay for a DBMS when you can simply stuff files containing the profiles into a RAID 5 filing system so they won't be lost if a disk dies and because reading from it is bloody fast. Job done, and for minimal cost and zero thought except that putting 1000 profiles in each file will save file open and close time during a scan.

[Anonymous Coward]

WTF are they doing ?

Each profile is added to a "batch" and loaded to the database with 1,000 others?

I'm sorry, but this really is technobabble at it's worst.

Does no one else think this is a crock of the highest order ? Mind you, given the perfect storm of technical illitereacy and journalistic credulity, I shouldn't be surprised. Who explained it to Mr Brokenshire (I urge readers to check his official mugshot out) Stephen Fry ?

[dssf]

Must be a massive amount of DNA collecting going on

To input DNA records or samples in batch sessions. Why batch only 5 or 10 at a time? Probably batches contain a couple hundred "contacts", probably done every 2 weeks. Not sure if that is representative of the level of relevant crimes, but I'm just pulling numbers for effect.

The collation of some number of contacts' files and their entry makes me wonder if file errors or contamination can let some suspects off and convert innocent suspects into incarcerated individuals.

And, what the hell is this batch entry got to do with isolating individual records. If a set of exonerating DNA is placed in the database to "extract" a deep-cover operative who needs a convincing case file or rap sheet, then it means that finding this anonymized record will be a major BI*CH to do so and not expose other records. Records requiring privacy sensitivity should be discrete and not commingled in such a way that deleting or viewing a specific record destroys or exposes or manipulates another record.

Either they have a horribly designed database, or the explanation is a crock of shit, or the interpretation of the explanation was maligned somehow.

Didn't the database design team know of CODD's rules, atomicity, and a slew of other best practices or database "laws" that help frame the design of ANY database. Even if it is object oriented, there still must be a valid concept of data atomicity, row and column locking, audit control, recovery, and so on.

Speaking of recovery, what about all of their backups. If there are 10s or dozens of off-site archives not yet purged, how do they know they will retrieve every last cartridge, disk, or other media and the logs that make references to that to-be-purged DNA? Do the references, too, need to be purged? If their logs are incomplete or inaccurate, they will probably never successfully remove each and every should-have-never-been-filed DNA record. If any of it traveled over a wire, across Wi-Fi, or into a cloud, even if encrypted, anyone knowing of the backup schedule could sniff and hope to crack the content encryption at a later date.

[Anonymous Coward]

someone

Someone didn't think ahead.

What if we need to delete a record?

Ahh we'll never want to do that.

Yeah but what if.

Look just make it a pain in the arse so if we do need to do it it'll cost a fortune.

~~~Several years later

You didn't make it expensive enough >.>

[PT]

Verity Stob explains

"The best way to avoid risk is by advising that any activity is technically impossible for reasons that are far too complicated to explain. If that approach is not sufficient to halt the project, then the engineer will fall back to a second line of defense: "It's technically possible but it will cost too much."

[Hayden Clark]

Shows that compliance was never intended.

In other words, the system requirement spec explictly left out any process whereby a sample and it's records could be deleted.

That's why complying with the law is so expensive.

[Alan Firminger]

Mad

What was the plan when someone, everone, dies ? Was the database to expand forever ?

And what was the intention when it was found that an error, or deception, occurred at collection so that a non person was recorded ?

[Geoff Mackenzie]

I suspect ...

... that when everyone dies, recovering disk space will not be a priority. In all (or slightly more) seriousness, dead people should stay on the database anyway, since it's possible they committed unsolved crimes and their records may exonerate other suspects in the future.

Also, at that point, the crime of storing their DNA for (probably) no good reason has become victimless.

[Black Betty]

IIRC the point of retention was FAMILIAL...

...matching. Sure there was some hope that a person of interest who proved innocent "this time" might be kind enough to leave a sample behind if he ever does offend.

However, that is small potatoes compared to the power of mining a comprehensive population wide sample of DNA. Even a relatively small sample would be enough to start getting direct hits on a parent, child, sibling or cousin. Once the number of samples has grown to a suitable representative size, it would soon become possible to say with some confidence: "We're looking for someone who's mother is a Kerry County O'Rourke and father is a Dun Edin McCleod." Which in turn requires a simple "hatches, catches and dispatches" lookup to resolve. Case closed.

Except with such a comprehensive database it would also be possible to frame up just about any member of the encompassing population simply by salting the scene with their DNA.

Big Brotherish enough thankyou that, even limiting the database to convicted criminals, still allows the "over the odds" apprehension of crime "families" with less than thorough police work.

Sympathetic (Voodoo) magic for real, where fingernail parings, hanks of hair, your blood and even your poop can be used to destroy you.

There is hopefully one silver lining. Technology is advancing at such a rate that spoofing advanced forensics like this is within reach of enough people, that conviction on forensics alone will hopefully soon assume a similar status to "only CIRCUMSTANTIAL evidence" as it properly should.

[Field Marshal Von Krakenfart]

Police announcement

"We have compared the sample found at the scene of the crime and we are looking for a relation of miss Eve Mitochondrial who may have lived in east Africa some 200,000 years ago"

[Anonymous Coward]

Wait.....

Are they saying hash the DNA profile, add it to a file with a thousand other hashes and then store this in a blob field?

How the bloody hell can they *find* anything, let alone delete it.....

[Annihilator]

Easy..

With a very very expensive, complicated and messy front-end connecting to this massively proprietary database that is keeping some third party (for want of argument, lets guess Accenture or IBM) in meal tickets for the foreseeable future! No doubt the sheer complexity of the storage was also sold as a security/encryption feature too.

Isn't that how all government databases work?

[Graham Marsden]

Umm...

... so how about taking the initial Batch Files, replacing the data of everyone who shouldn't be in it with a null record and then uploading them onto a new system before purging the old system entirely?

[nsld]

can we assume

If the figures of 1 million samples illegally held is correct then nearly 5 quid a pop to look up a record and delete is somewhat costly.

I cant see what batch uploads have to do with it, if they have to compare samples they dont get results back in blocks of 1000!

Sounds like idiot Gov muppet getting the shaft from IT person that can spot idiot a mile off.

[Steve Evans]

Re: can we assume

So they have to remove 1 million records? So paying minimum wage to someone manually deleting entries would actually work out cheaper than this quote, assuming they can handle the neck breaking speed of 2 an hour?

[annodomini2]

Hmm...

Format C:?

[Neil Charles]

While we're helpfully throwing SQL about

TRUNCATE TABLE [DNA]

By the way, what did you mean by 'just the innocent ones?'

[Anonymous Coward]

I'll charge them half that

After all petrol isn't cheap these days.

<-------

[Tegne]

The IT person missed an opportunity here..

When we knew that we were explaining technical details to someone who had no idea what we were talking about we'd throw in Star Trek references. 'The deflector dish is misaligned' - 'The dilithium crystals are spent.' - We never got picked up on them.

[Khaptain]

Jiggery pokey going on

Who did they hire to delete the data at that cost , the BOFH ?

[Anonymous Coward]

Can't be the BOFH.

He'd be storing them to null to speed up the loading scripts.

[batfastad]

Not fit for purpose

If this is the case, then the DNA DB is not fit for purpose and there should be serious questions asked of the supplier of the system.

Did they just build it thinking that our civil liberties would never ever be restored after more than 10 years of erosion (and bankruptcy) by Labour?

Please tell me it's a normalised relational database and there's some sort of common ID field linking the DNA profile with a human. If it is, then the hard bit would actually be trying to get together all the ID numbers of the innocent humans in the first place. The actual delete should be a simple process after that.

Surely under the data protection act people can just go and inspect this data and request for it to be deleted? Oh I forgot, the law doesn't apply to governments and their gravy train agencies.

[Anonymous Coward]

RE: Fit for Purpose

Given that the purpose was to collect DNA data from everyone who came within swabs-length of the UK and to hold that data indefinitely then the database actually was fit for purpose.

It's the twats who decided we needed a database of this nature in the first place who were not fit for purpose.

[Anonymous Coward]

I've worked on Police and Courts IT systems in the past.

Last time was in about 2007....I had to upgrade their access 97 database that they were using to run their court reporting software...

[IR]

Banning slavery was expensive too

Let's not do the right thing if it might cost some money.

[Joe 35]

Missing the point ..

... only the most god-awful fucked up DB should make it that expensive to delete data, even if you have to wipe all the backup tapes and create new ones.

[Ally J]

DNA records stored on an iffy database?

If it's really that much work to remove records (and I don't doubt it is), surely the efficiency of the whole databse can be called into question?

Does rather sounds like an explanation worthy of a spooty youth in Dixons or Comet, mind you.

"Well, yeah, it'll cost, uh, four point eight million because we have to, uh, load them up in batches of a thousand at a time."

"But these are ones we want to get rid of."

"Yeah, but, well, then the ones we've loaded up won't be in thousands no more, so we have to load them up again. And that needs Barry from Head Office, cos he's, like, better at this."

And so on.

[The Flying Dutchman]

While we're on the subject of databases...

... the root of the problem can of course be illustrated by executing the following SQL commands:

SELECT * FROM [civil_servants] WHERE [location] LIKE 'whitehall' AND [clue] IS NOT NULL;

(0 rows returned)

and

SELECT * FROM [civil_servants] WHERE [location] LIKE 'whitehall' AND [self_serving] = TRUE;

(server crash - result set too large)

[Anonymous Coward]

Any banker out there want to take a lead...

and donate/recycle some of their bonus to a worthy cause.....

The plaudits they would get would be worth the small cost to them, surely?

[Anonymous Coward]

future backout

Bet they 'accidentally' forget to delete these records from the backups

[Anonymous Coward]

you think they keep backups ....

with a system as incompetently designed as this one, it wouldn't surprise me to learn there are no backups ....

[Yet Another Anonymous coward]

Cost savings

How much does it cost to do a DNA profile?

I can't imagine it's cheap, private companies charge about $250 for paternity tests etc. So assuming that $1=£1 and add a 1000% government contractor markup they are paying a few quid to put innocent people's DNA into the system in the first place

If you waited for them to be convicted before doing the profile than you could save the cost of removing the innocent ones pretty quickly

[Anonymous Coward]

Sigh

Seriously? Did the civil servant/politician who received this estimate just blindly accept it, and not do any kind of mental figuring or apply any sort of common sense to ponder just how ludicrous it is? The fact it comes a day after a huge government announcement about how they're going to do things different now, and no longer accept getting repeatedly screwed by the big IT suppliers just makes me despair. The kind of money quoted would pay a team of 20 consultants £750-a-day for OVER A YEAR. There is no way that even a government IT contractor could figure out a way to string out a few SQL deletes that long. Anyway, they seem to know there's a million innocent people on it... rerun that query with DELETE instead of SELECT COUNT(*) and the job is good as done.

[John Smith 19]

So the sooner they get started the fewer they will have to remove.

Let me guess they *still* have not implemented the rules on *not* putting people on in the first place.

[John Smith 19]

But watch ouf for the old "National security" clause

The impact assessment is here.

http://www.homeoffice.gov.uk/publications/about-us/legislation/freedom-bill/pof-bill-ia

It states that "Evidence" from police and security services state national security investigations could last 25 Years (no doubt without *any* convictions). But they wouldn't mind if such a case was reviewed every 3 years or so.

And yes it does think that there are about 1 million people wrongly on the DB so that is c£5 each.

For what I have *no* idea. Perhaps the National Police Improvement Authority (who run the NDNAD) could explain the intricacies to someone from Vulture Central.

It's a very *grudging* start. 3 years is *still* too long IMHO for some not *charged* or convicted.

[Anonymous Coward]

JFDI

FFS

[Sam Therapy]

It would cost a lot more if they did the job properly

Since there will be many innocent people still on the database - those accused of but not found guilty of a "serious offence" - the cost is not a true reflection of how much should really be spent.

Remember, Not Guilty does not mean "Got Away With It" or "Probably Guilty".

As I have said before, this is nothing but bullshit PR and will remain so until *all* records of innocent people are deleted.

[mhenriday]

There is no «innocent DNA» -

only DNA that the services of the state have not yet convicted of a crime....

Henri

[Winkypop]

I started work when index-cards were the norm

Now, managing thousands of hand-written paper cards was difficult.

Imagine having to store, collate, sort and search these all day?

You needed a pretty sharp HB back then.

I can understand £5M for major changes to that kind of chaos.

But £5M to update a computer system?

Computers; we were promised speed and efficiency.

I am also waiting on my flying car!!

[noidea]

why no records management?

Why is this "database" not part of a records managements system such as other public records?

viz TNA/PRO

Then disposition can be managed by policy which is a basic function

[Neoc]

Tsk, tsk, tsk.

First, let me state that I developed a DNA-paperwork-tracking database for the local Police force. The way DNA-profiling works (here at least) is as follows:

System generates paperwork (DNA + Fingerprint, including barcodes to help track the paperwork + samples) which is printed and handed to the person who will be doing the sample. Why? Because as far as forensics is concerned it can be shown that this DNA belongs to the person with this set of fingerprints. BTW, fingerprints are actually a more accurate identification system - twins (for example) will have the same DNA but different fingerprints. It has been suggested that even clones of a person would have different prints. But DNA is easier to get at a crime-scene these days.

Samples are taken from the suspect and the paperwork is signed, and barcode strips are attached to the samples. DNA Samples are sent away to the DNA labs, FP samples are set to the fingerprint bureau (we'll leave the Fingerprints alone now and concentrate on the DNA).

DNA bureau get the barcoded samples and process them to extract a DNA signature. Both the sample and the signature are only referred to by their barcoded number - you would need both the DNA sample data (with a unique identifier *for each sample*) and the paperwork/system data to link sample(s) to a person. (note; I was informed at the time that DNA-ing a person costs ~AU$100 per sample)

Upon requiring destruction, the system issues the relevant destruction paperwork *with unique identifier and barcode* which is sent to the labs. The labs, which store the samples/signatures base on the identifier, retrieve the sample and signature and destroy them (by sterilisation and/or burning). They then sign the compliance letter which accompanied the paperwork and send that back to HQ. (note; again I was informed that destruction of a sample costs ~AU$10 per sample).

The reason it's done this way is that the police force here can (and has to) retain the fact that you *were* DNA'd and the accompanying facts for legal reasons (like FOIs, court actions, etc). But if you are not charged, or are found innocent, the *identifying markers* (i.e., the DNA sample and signature and the fingerprints) which were taken *for that purpose* have to be destroyed. (note: any existing DNA data for instances where you *are* guilty can/will be retained).

So: £5m pounds is ~ AU$7.75m - does this mean the UK has 775,000 "innocent DNA" samples?

[Anonymous Coward]

Probably

<quote>So: £5m pounds is ~ AU$7.75m - does this mean the UK has 775,000 "innocent DNA" samples?</quote>

Could have indeed. Everyone arrested has their DNA taken, and never removed no matter what the outcome. And its been going on for 16 years already. But somehow I suspect this figure of £5m is bullsh1t.

Every figure the government tells you seems to be weighted towards their preferred outcome. Minimum wage? Too expensive, will cost us <£ Big Number>.

Privatisation? Extremely cost effective and good for everyone, will only cost <£ Small Number>

Etc.

[Anonymous Coward]

A Title

Probably just that the profile number is linked to the person with asymmetric encryption keys and the decryption routine is only built to provide the person where a match has been found already.

The bulk of the cost is probably around the need to keep the bulk decryption routines you have to develop and also the decryption key properly secured or you lose the value of having it in the first place.

[Anonymous Coward]

SQL?

You make it seem like the gov't understands technology.

The data is probably stored in an Excel Spreadsheet...

[Reclaim your DNA]

Brokenshire's DNA cost assessment: Cock up or conspiracy?

Stop hold press! House of commons was misled. Removing innocent people's records from the DNA database will save £ 24 M. The minister only mentioned the cost side of the impact assessment. On the benefits side the impact asessment from the home office actually show 10 year savings of £61.8m in todays money. Now what I'd like to know is if this was cock up or conspiracy to withold the facts. Maybe it was a bad April's fool joke?

More on 'Reclaim your DNA on FaceBook' where you can find the links

[TimB]

SQL is only half the story

Sure, deleting the data is probably fairly trivial and cheap. The vast majority of the cost is likely to be getting rid of the physical DNA samples. This is biological waste, you can't just leave it out for the binmen. Add to that the fact that the samples need to be securely destroyed to make sure they don't accidentally end up on some health insurance database somewhere, and this doesn't seem like such a huge figure. For government spending, at least.