Right to contest automated AI decision under review as part of UK government data protection consultation

The UK government has launched a consultation that suggests it could water down individuals' rights to challenge decisions made about them by artificial intelligence. In documents released today, the Department for Digital, Culture, Media & Sport (DCMS) said that the need “to provide human review [of AI decisions] may, in …

[Graham Cobb]

Data economy?

...likely evolution of a data-driven economy and society

Many of us see no need for, and don't want to be part of, a "data-driven economy". I see that as pretty much identical to having a "crime-driven economy" - why would I want one of those?

Concentrate the economy on real stuff and services that we want. Not on processing data about us.

If the services are attractive enough, and really beneficial to people (instead of exploiting them), then there will be no problem getting people to agree to the data processing that supports them. But different people will make different tradeoffs so everything still has to be based on personal agreement to permit the processing.

[Anonymous Coward]

Re: Data economy?

Make no mistake, this is the groundwork for another degradation of the state's obligations towards its citizens. The purpose is to allow a free run for replacement of all those "civil service" decisions on things like benefit entitlements, immigration, asylum, planning, health services, and I don't doubt eventually the magistrate's courts, by AI systems cooked up by Chum-panies. Their execs and shareholders get a healthy fee, and the Government can sack a swathe of public servants.

The illusion of choice will be provided by (probably insurance based) premium service providers who will be able to circumnavigate the AIs for a fee; but that will be set to ensure only the wealthy few will effectively have an appeal process.

[Anonymous Coward]

Re: Data economy?

C***s! Obviously our glorious leadership bought into their pals' story of how applying AI will solve the backlog of court cases, save the children, defeat the terrorist, and make Britannia rule the ways again, all that while saving the Treasury billions...

[gnasher729]

Re: Data economy?

Well, it can solve the problem of backlogs with court cases. But so would giving the judges some dice and five minute training. At much lower cost.

[Warm Braw]

Re: Data economy?

If you drop the legal requirement to treat people fairly, of course it may, in future, not be practicable or proportionate, because you've dropped the penalties for failing to comply with basic human decency.

If there were any doubt about the post-Brexit future off the UK, then this really ought to remove it. The economy is not there to serve the citizens, the citizens are merely the servants of the economy. Except those chosen few who are appointed to be its masters.

Each time I wonder if it was proportionate to escape the UK in the short post-Brexit window, some new shade of its future awfulness reminds me I was right.

[Anonymous Coward]

Re: Data economy?

If you'd kept your comment to the first paragraph, I would have upvoted it. But the rest of the bremain bile means I can't.

[Anonymous Coward]

Re: Data economy?

The Brexit you get is the one that exists. Chosen by Johnson & Co. Abused to introduce this demeaning policy.

[ibmalone]

Re: Data economy?

Let's ignore the most obvious fact that the move is only made possible by brexit and consider the more fundamental background.

You've misunderstood, this was always the project.

Yes, they sold you a story about immigration, or 'Brussels diktats' or benefit tourism or whatever it is you like arguing with guardian readers about. But the people behind it, when they said "take back control" always meant control for themselves, not us.

It annoys old Etonians and their like that despite having 'won the game' in their minds by getting made MP that there were still rules and powers bigger than them. It annoyed the people who used to be able to stick them a few quid to sort things out that somebody might now come along talking about competition law.

And so here we are. Steadily chipping away at hard win democratic rights that were meant to stop people being packaged and sold.

[Warm Braw]

Re: Data economy?

Regrettably, the most serious consequence of Brexit has been that it the consequences of Brexit have to be denied or dismissed.

Brexit was a necessary part of a wider project to reshape Britain in the mould of recent US capitalism (the increasing concentration of wealth and power in small number of hands). It wasn't an end in itself, but it removed the external constraints on such a policy.

It's perfectly reasonable to regard the Brexit event as a watershed moment because pre-Brexit and post-Brexit Britain are two very different places - not because of the economic consequences, but because it has allowed the executive branch of government to "take back control" entirely unfettered by the British constitution or by external obligations. That means there are no defences against a feral government such as the UK currently enjoys.

With respect to this article, it's worth reading the blog from Mariano delli Santi of the Open Rights Group and this from openDemocracy on the new Information Commissioner. It's pretty plain there's been a sea-change in terms of government accountability.

This wouldn't matter so much if it were simply the ebb and flow of the electoral cycle. However, boundary changes and the new laws on voter suppression that would simultaneously extend the ability of rich non-doms to fund one party and suppress the ability of Unions to fund another and put the regulation of election conduct in the direct charge of the cabinet office put democracy itself at threat. And the sinister "war on woke", equally imported from the US, is deliberately undermining the concept of truth because the state those who support the current government want to build can only be built on lies. And of course by undermining other sources of power, whether that be the devolved governments of the nations or local councils.

Brexit never meant Brexit, it was always a proxy for a culture war. And while these aren't the consequences of Brexit, per-se, they're the consequences of the vote for Brexit and the legitimacy it gave the former "fruitcakes and loons". It may not be what Brexiters intended to vote for, but it is what their vote delivered.

Add in the economic consequences - and the apparent unwillingness of the government to even acknowledge them let alone have a plan to deal with them - and the picture is thoroughly depressing. But it's depressing enough without them.

[MrDamage]

Re: Data economy?

So you feel strongly supportive about Brexit, but you're ashamed to put your name to it.

Curious.

[Anonymous Coward]

Re: Data economy?

"replacement of all those "civil service" decisions on things like ... and I don't doubt eventually the magistrate's courts"

The Master of the Rolls stated last week that he expected the majority of court decisions to be conducted by AI, and only for the most serious/complex/difficult cases to actually be seen by a human judge.

[b0llchit]

Community AI project

Lets all put our heads together and make an AI to use a data driven approach with current affairs as input and output the AI decision: UK government, you are fired!

There will be no appeal. There will be no further discussions. The decision is final. Please leave now.

That would sound fair, wouldn't it?

[Doctor Syntax]

Re: Community AI project

But you're not looking at it from the government's PoV. Their AI would be "no need for further elections, keep things as they are". And it's they who make the rules.

[Anonymous Coward]

This is the stupidiest decision that has ever been made

just saying.

[b0llchit]

Re: This is the stupidiest decision that has ever been made

It was made by AI.

[teebie]

Naive

I saw the headline and thought we were going to get more rights to challenge bad decisions made by AI.

The week must have tired my brain out.

[Mike 137]

The Dictator pronounces

'the need “to provide human review [of AI decisions] may, in future, not be practicable or proportionate.”'

If (as the above suggests) it will be deemed impossible to establish the basis of an automated decision, the ability to challenge it and have it reviewed by a mechanism open to discovery as to basis (e.g. a reasoning human) is fundamental to human rights (if not the limited legal ones, then the wider moral ones). Otherwise it's in constitutional terms no better than the whim of a dictator.

"Computer says No" was intended as a jocular warning. If this proposal gets accepted, we have the prospect of it becoming a legally binding fearsome reality.

[Will Godfrey]

Re: The Dictator pronounces

Indeed.

Be afraid. Be very, VERY afraid.

[Eclectic Man]

Yes, Minister?

"

The UK government is proposing to implement “a more flexible and risk-based accountability framework which is based on privacy management programmes”.

“Under this framework, organisations would be required to implement a privacy management programme tailored to their processing activities and ensure data privacy management is embraced holistically rather than just as a 'box-ticking' exercise,” he said.

"

\begin{RANT}

So soon after the data breach by McDonalds (https://www.theregister.com/2021/09/09/mcdonalds_database_credentials_blunder/). the idea of a 'holistic approach' to security sounds like an episode of 'Yes, Minister'.

As for a 'risk-based accountability framework', WTF is one of those? OH, hang on, we had one of those for dealing with the Covid-19 pandemic didn't we? So far about 134,000 deaths, although arguably the dominating risk guiding policy and decisions was Boris Johnson's popularity, rather than the risk to life for vulnerable people in care homes.

I need a drink.

\end{RANT}

[Mike 137]

Re: Yes, Minister?

"organisations would be required to implement a privacy management programme tailored to their processing activities"

Considering that over the first two years since the GDPR came into force, not a single one of the 400-odd organisations we monitored succeeded in achieving legality in the most basic requirement - transparency about their processing - I don't hold out much hope.

However even the EU seems OK with that. Paragraph 49 of the UK Adequacy Decision states: Data subjects should be informed of the main features of the processing of their personal data. [emphasis added], which implies that the full disclosure essential for the exercise of data subject rights is no longer considered necessary even by the originators and maintainers of the GDPR. Nobody wants all those pesky challenges to their business decisions and processes.

[Irongut]

If you're training an AI to make important, life changing decisions for other people, train it to reject members of the Tory party and their supporters.

If Tory voters can't get mortgages then Tory MPs won't get votes.

[Anonymous Coward]

Computer Says No.

And that's it. Jog on. Tough if your life or livelihood depended on it.

The more this government covers, the more I despair. And not fan of the opposition either. And the British public seem to be saying say "yes, you know best, take our rights away, and protect us from life."

Come Elon, take me to moon where i don't have to deal with this any longer.

[ibmalone]

We're fucked

We're all fucked.

[Howard Sway]

Now that we have left the EU, we have the freedom to create a bold new data regime.

Unfortunately, we are an especially dull minded bunch of ideologues, so we will just decide to do exactly the opposite of whatever the EU does, in the deranged belief that this will yield good outcomes.

[JohnnyL]

Be afraid, be very afraid.

There are only two options:

1. Those suggesting removing this safeguard are completely unable to comprehend the inevitable and severe consequences on any form of reasonable accountability that any person in a western society should expect.

2. They do.

If (1) then every reasonable person should be shouting from the roofops to highlight that they are not 'fit for purpose' to be in their current role, whatever that is.

If (2) See title.

[Eclectic Man]

Re: Be afraid, be very afraid.

I think it is a combination of (2) and being entirely confident that these AI systems will never make any decisions about them. They are wrong about the latter, of course, but if, for example, a flawed AI system* was used for crime fighting, it is unlikely to affect 'Millionaires Row' as much as it does areas where 'poor people' live.

Bu then, if a cabinet minister cannot tell the difference between an England Footballer and a Rugby Union player, what chance has AI got?**. (And no, Mr Williamson, Black people do not 'all look the same', or indeed sound the same.)

*https://www.theregister.com/2021/08/25/shotspotter_chicago_report/ ,

https://www.theregister.com/2018/08/13/criminal_justice_code/

** https://www.independent.co.uk/news/uk/politics/gavin-williamson-marcus-rashford-maro-itoje-b1916299.html

[heyrick]

FFS, there is no such thing as "AI".

Let's get this straight - the 'I' is supposed to stand for "intelligence". There isn't any. There is the illusion of intelligence due to clever pattern matching returning better results than lamer software, but it is (as has been pointed out a lot here) liable to pick up on the biases of its creators and training data. A system that competely lacks understanding that is also notoriously incapable of understanding what black people are, for example, is what we're supposed to accept making decisions about us? Decisions that are to be cast in stone?

Fuck off and die, the lot of you Tory tossers.

[Anonymous Coward]

Re: FFS, there is no such thing as "AI".

>There is the illusion of intelligence due to clever pattern matching returning better results than lamer software

At risk of invoking the downvotes, surely you can apply "just clever pattern matching" to any and all processing systems including the human brain?

[Anonymous Coward]

Don't worry, be happy

Clearly this couldn't happen before they extensively verified the AI methodology. Right?

XKCD AI Methodology

[amanfromMars 1]

Don't worry, be happy. The future is cast, the past is done. What presently can/will you do?*

Clearly this couldn't happen before they extensively verified the AI methodology. Right? ..... HildyJ

Be happy that is nothing to worry oneself about, HildyJ, for others than they have long ago mastered that requirement to effortlessly and quite sublimely provide such facilities and utilities for removing faults most comprehensively with any lingering or missed mistakes safe and secure in a failsafe environment where any possible harm from chosen administrative path effects is always beta tested and firstly directed self-inflicted to ensure ignorance does not prevail in any support of almighty arrogance and perverse hubris.

And such has already been told to you, right here on El Reg too, and as recently as only yesterday ...... https://forums.theregister.com/forum/all/2021/09/10/ibms_quantum_rainmaker_jumps_to_quantum_diamond_outfit/#c_4329299

* .... To think to huff and puff to try to blow the house down aint gonna cut the future mustard, is it, whenever contemporary peers and present competition and current opposition are all on much higher levels of operation on altogether greater planes of remote existence and virtual existentialism ....... Advanced IntelAIgently Augmented Virtual Reality, which to be sure, is a Certifiable Alien Space. Of that you can rest fully assured.

All anyone/anything really needs to be worried about, if to worry is your thing, is the remote administrative execution going to choose to be creative and benign or malignant and destructive, although if one can do nothing about any of that, why worry? Que sera, sera.

[Anonymous Coward]

This is a complete catastro-fuck. "Service improvement" as a legitimate purpose. ICO subordinate to a minister. Payment for grievances. No requirement to report. No review of automated systems.

This is not only an unprecedented assault on individual privacy, it will destroy GDPR equivalency. We will be a hermit nation of rapacious, destructive, half-baked tech firms and nothing else.

[Richard 12]

Ending as a Vassal State of the USA

Or possibly Russia or China, it's hard to say for sure.

With a future UK government that has less ability to affect UK policy than (eg) Puerto Rico. Just do whatever they say.

[amanfromMars 1]

Re: Ending as a Vassal State of the USA

Or possibly Russia or China, it's hard to say for sure.

With a future UK government that has less ability to affect UK policy than (eg) Puerto Rico. Just do whatever they say. ...... Richard 12

There are those who will tell you, Richard 12, paths currently earmarked and presently actively embarked upon both practically and virtually guarantee one ending commanded and controlled as if as a vessel in a Surreal State.

And quite whether the likes of a leader of a Russia or a China or a USA or a UK or a wherever or whatever can also recognise and accept that as the vital necessary quantum leap in understanding to be made in order to be able and enabled to effectively significantly contribute to future deliberations with mass media presentations as impact directly upon Earthly nations, is a task before them to successfully complete in order to either compete or oppose in such a situation.

[Geez Money]

Fuck No

Are you fucking kidding me?

[porlF]

G.I.G.O.

Aside from the political arguments for and against this approach, there is a long-standing principle of computer tech in that Garbage In results in Garbage Out.

Decision models based on any kind of acquired data, accurate or otherwise, are vulnerable to flawed interpretations and consequential flawed decisions.

Is the assumption going to be that models are going to be 100% accurate, both in source data and algorithm, and therefore no dispute is necessary? I mean, that does appear to be the implication!

The common sense of the IT and mathematical world should surely know that "AI" will not be 100% accurate and that we need a separate unbiased non-silicon method of arbitration?

Thoughts?

[ethaan]

Let's not kid ourselves; there are already many automated systems that use our data to make decisions without any human interaction. In many of these systems the average consumer doesn’t understand how the decisions are derived.

As with so many things in life those sorts of algorithms can be used to benefit society and drive innovations but just like any technology it’s open to manipulations. The question becomes do we trust the current bodies &the current people government to act in the interest of society... What I would like to see if a strong independent bodies set-up to monitor and regulate the use of AI in all fields of technology.

[Cuddles]

Context

"issues need to be viewed in the context that the use of automated decision making is likely to increase greatly in many industries in the coming years"

Indeed. In the context of increasing use of automated decision making in many industries, the right to have those decisions properly reviewed becomes more important than ever. The idea that a widespread practice requires less regulation than an uncommon one is insane.