Transparent algorithms? Here's why that's a bad idea, Google tells MPs

Opening up the processes that underpin algorithms may well magnify the risk of hacking, widen privacy concerns and stifle innovation, Google has told MPs in the UK. The comments came in Google's response to the House of Commons Science and Technology Committee's inquiry into algorithmic decision-making, which is questioning …

[Warm Braw]

The algorithmic model can be can be selectively tested with different types of input

If, for example, an engine management unit swaps into "emissions cheat" mode with a combination of inputs it's unlikely to get in normal operation, you'd have to be very fortunate to hit on the specific combinaiton of unlikely inputs simply by chance if you were attempting to investigate it.

It's important that algorithms that affect people's lives are not only transparent, but auditably transparent. If that causes "innovation" to suffer, it's only because it's not the kind of "innovation" we need.

[Andrew Commons]

Re: The algorithmic model can be can be selectively tested with different types of input

Some research recently published on the arXiv preprint server examined inserting back-doors in algorithms during the training phase. The rationale was that training was likely to be outsourced - sent to the Cloud - to get the compute resources and that the training data could be manipulated while it was in the Cloud. Worked well. Back-door could not be detected looking at the model and it survived additional training largely unscathed.

[Daggerchild]

Re: The algorithmic model can be can be selectively tested with different types of input

You're basically describing why both white and black box testing exist. You can bring order to a system by crystallising it from a core (deduction), or wholly enclosing it and extracting disorder (elimination). And you use both if you're smart.

Open sourcers are fuzz-testing code they themselves wrote, because even the whitest of white boxes doesn't actually prove things as clearly as you'd think. The black box tests still find things. But effective black box testing is very dependant on the devious imagination of the guy writing the test cases. Fuzzing is just pot luck.

But while everyone's spitting on Google's arguments here, I'll point out that if I had access to the App Store automatic viruschecking algorithm, I could ensure it would never catch my daily malware. And if you had access to the search algorithm, you would use it to make your site appear as the top link for everything you could, wouldn't you.

There will always be constant recurring calls to open Google's search algorithm, because that is the best way to kill them (and Internet Search) for good.

That said, a Government Department of Inter-Industry API's would probably be a good thing.

[Anonymous Coward]

What a load of bullcr*p...

My microwave doesn't make a decision on how long the food should be cooked... I do...

My car doesn't decide where it should go... I do...

My lights don't turn themselves on or off... I do...

The algorithms used by Google and other companies however decide how to best violate my privacy, extract more money from me and have a real impact on my life WITHOUT me having ANY control over it...

[Steve Davies 3]

Re: My car doesn't decide where it should go... I do...

But for how long.

With all the people advocating car sharing and autonomous driving, You, the slab of meat and water will have to give up being in total control. The

"Rise of the Machines"

will make you into even more of a couch slob.

[Tom 7]

Re: My car doesn't decide where it should go... I do...

There's a large city in China where a company access all the road video cameras and can predict traffic 10 minutes in advance and reduce congestion by 90% by texting people (it knows who is driving the cars too!!) to advise them to take alternate routes to prevent congestion. Your personal preference may be for sitting in a queue inhaling diesel fumes but I'm guessing you're in the minority.

[Teiwaz]

Re: My car doesn't decide where it should go... I do...

There's a large city in China

Fine, if there's an informed opt/in opt/out in place with transparent and clear disclosure on what information is passed and/or retained.

[Loyal Commenter]

Re: My car doesn't decide where it should go... I do...

I don't check my texts when driving (you know, because it's dangerous and illegal), except for the rare occasion when I am already stopped in stationary traffic with the handbrake on and I need to let someone know I am delayed. How would this almost certainly apocryphal traffic management system help me?

[Anonymous Coward]

Re: What a load of bullcr*p...

My microwave doesn't make a decision on how long the food should be cooked... I do...

Ah well you see, modern microwaves have come on quite a bit. Now you can just press a button that says Chicken, and it will ask you how much it weighs, then it decides how long to cook the chicken for.

Obviously, you don't get these buttons on a dial microwave in general..

[DropBear]

If by "black box" you mean "show me any part from my car and I will tell you what it does and where it comes from" and "if you sent me a hundred years back in time I would probably be a Very Dangerous Person for knowing how a magnetron works" then yeah, sure, black box it is. But I'm not so sure all "algorithms" should just stay buried - "gaming" or not, I wouldn't trust a black box voting machine...

[Aitor 1]

Black box

"Many technologies in society operate as 'black boxes' to the user – microwaves, automobiles, and lights – and are largely trusted and relied upon without the need for the intricacies of these devices to be understood by the user."

So they are blatantly lying here.

Most of the users are unaware of how these work, true, but they know that many people understand how they work, and the knoledge is there for them to get them should they want to.

It is the same fundamental difference between google's unpublished algorithms and opensource.

Most people are going to use opensource programs and never bother to go into the insides.. unless they want or need to.

If the algorithm isnt publish you have a reasonable cause for suspicion, as these algorithms cannot be checked by the public.

So basically they made the wrong example.

A better one would be: "people are served by secret services as the MI6 and the CIA, and they have placed their trust in these fine institutions". But because they are seemingly unaccountable and black boxes, people have no trust in them!

[The Man Who Fell To Earth]

Re: Black box

"Most of the users are unaware of how these work, true, but they know that many people understand how they work, and the knoledge[sic] is there for them to get them should they want to."

Agreed, anyone trained in the respective technologies know how they work, not just the employees at the manufacturer. In other words, any microwave engineer knows how a magnetron works & how all mircowave ovens work, not just the employees at the mircowave oven manufacturer. Similar for automobiles - a wide swath of people external to the manufacturer know how cars work, etc.

Google's argument is overtly dishonest & misleading.

[Loyal Commenter]

Re: Black box

More importantly, with a car or microwave, it is perfectly possible to take it apart into its constituent pieces to determine how it works (whether this is advisable is another matter). For instance, if I were to disassemble a microwave, I would find things such as:

• A metal enclosure
• A door with a faraday grille to prevent radiation leakage
• a power transformer
• Control circuitry
• A magnetron
• Control dials
• A hinge switch

etc...

With each of these parts, I would have a decent chance of identifying it and its purpose. If Google's software was open-source, and well designed (which I'm sure it is), I would also be able to go through the exact same process, class by class. If it's properly documented, then it would be even easier. None of this would prevent Google from holding copyrights (and 'software patents') on their code, in the same way that the manufacturer of my microwave no doubt holds design patents, and copyrights on it.

[Rich 11]

Re: Black box

Google's argument is overtly dishonest & misleading.

Well of course it is. It would have been written by their lawyers.

[Zippy's Sausage Factory]

Sounds like weasel words to me

Their analogy doesn't hold up, really - if I want to build my own microwave, or lightbulb, I can go get the patent documents or find a technical publication about it. This doesn't mean I'll have the technical capability to build a working one, but that doesn't mean I can't find out how it works.

They're essentially saying "look, it doesn't matter if you know how it works, so long as we do". That's a different thing altogether...

[Anonymous Coward]

Playing devils advocat for a moment ...

a lot of the analogous products given as an example are protected by patent laws, which it can be argued, leads to a culture of transparency, and removes *one* incentive for companies to keep secrets.

One possible thing Google could be wary of is a situation where they hand over their crown jewels to an organisation as staggeringly incompetent as the UK government with no backstop protection ?

[}{amis}{]

Re: Playing devils advocat for a moment ...

The problem with Google and co is that they don't view them selves as being bound by any law.

They do their best to operate as an entity floating above any countries legislation plopping down tentacles where they think the get the greatest benefit: US for sloppy content laws, Ireland for sloppy tax law etc...

[ibmalone]

Re: Playing devils advocat for a moment ...

While I appreciate the attempt, lights (lights were an odd example for them to pick, as they wont do their intended job if enclosed in a black box), and microwaves are in general not covered by patent laws. Cars are obviously a broader area, though their fundamental operation is not. (Assuming their fundamental operation is moving around, rather than pairing with your phone.) And while the individual on the street might not be able to tell you how a microwave works, plenty of people can, and are able to investigate important questions like whether they operate safely.

[Muscleguy]

Re: Playing devils advocat for a moment ...

Ah but with an infrared camera or just my hand I tell if a black box contains a lit bulb or not. If I can tap the wires leading in I could work out the wattage of the bulb and by analysing the waveforms possibly the nature of it (incandescent, fluorescent or LED).

Black box analysis is taught in many science courses. It's how we figured out how kidneys worked initially (measure both inputs and outputs in different setups). Then we discovered NZ White Rabbits which put their proximal convoluted tubules just below the surface of the kidney meaning you can stick a pipette in there and sample/manipulate that too.

There are still areas of biology which are pretty much black boxes. Natural mutations, knockouts etc can give you windows into them. That's my PhD and Nature paper there, respectively.

[Loyal Commenter]

Re: Playing devils advocat for a moment ...

Is that some sort of horrible yellow alcoholic beverage?

[Ben1892]

Re: Playing devils advocat for a moment ...

Yup, the one at the very back of every liquor cabinet, but with added chili and spices

[Anonymous Coward]

Re: Playing devils advocat for a moment ...

Mmmm...

Devil's Advocaat Recipe

3 ounces milk; 1/2 ounce blue curacao; 1 ounce advocaat; 1 pinch nutmeg; 1 dash grenadine; 1/2 ounce white Creme de Cacao

[Zippy's Sausage Factory]

Re: Playing devils advocat for a moment ...

"Devil's advocaat"?

Yeah, that's what I thought about it the next morning last time I had some...

[Anonymous Coward]

Cooking instructions are an algorithm?

Really?? Sounds like an order of transparent BS designed to keep tech-illiterate pols away from the recipe for Google's secret sauce.

[Richard IV]

Re: Cooking instructions are an algorithm?

Taking the cooking analogy further, food manufacturers are required to list the ingredients for their produce in reverse quantity order but not the actual recipe.

The algorithm is thus hidden, but everyone knows the parameters.

This hasn't stifled innovation in the food industry; rather they use a thesaurus for masking the ground-up chicken bones as calcium from avian sources.

[John Smith 19]

TL:DR Security by obscurity is good for you. Don't bother your silliy heads about such things

Here's a few differences.

Microwaves and cars are physical devices. They have very clearly defined properties and it's usually pretty obvious when they fail. Microwave does not cook food put in it? Not working. Car goes backward when you select 1st gear? Not working.

I especially like the "And it could affect small startups."

Like they give a f**k. Google is built on being the biggest search engine on the planet by a large margin. It's interest is keeping the real barriers to its market entry high as possible.

[Anonymous Coward]

Re: TL:DR Security by obscurity is good for you. Don't bother your silliy heads about such things

People are (deliberately?) missing Google's point here.

"Microwaves and cars are physical devices."

Not completely. Most new models depend on software to some major degree.

"They have very clearly defined properties and it's usually pretty obvious when they fail"

Not always.

"Microwave does not cook food put in it? Not working."

That's analogous to "Google returns nothing when I search for something".

That is, of course, not the point. The issue is that people believe that Google is returning results that are not optimal to the users' requirements, but are relevant enough that most don't know or care.

To use the microwave analogy, this is like you telling it that you want to cook at 70% power for 3 minutes, but internal design means that it actually cooks at 74% for 2 minutes 54 seconds to conserve power or to optimise some other internal process. You don't need to know or care, so long as you get sufficiently heated food.

"I especially like the "And it could affect small startups.""

You don't think that being required to publicly reveal algorithms would effect them as much as established players?

"Google is built on being the biggest search engine on the planet by a large margin."

It didn't start that way, and has got to be there currently largely because of the algorithms it uses.

[Anonymous Coward]

Re: People are (deliberately?) missing Google's point here.

You're confusing missing the point with not believing their bullshit. We know what they're trying to say, it's bollocks.

[big_D]

Re: TL:DR Security by obscurity is good for you. Don't bother your silliy heads about such things

I recommend reading "The Circle".

[ranger]

Re: TL:DR Security by obscurity is good for you. Don't bother your silliy heads about such things

Cars are a good example IMO - there's a lot of software in them these days controlling the full driving experience. I have no idea how the engine management system in my car works, nor do I particularly need to know, but it's there to manage fuel consumption and emissions and make sure that the injectors open and close at the right instance squirting the right amount of fuel in. If the ECU or one of the sensors that feeds the ECU breaks, then the engine might fail, or emissions may go illegal and the little spanner lights on up the dashboard. It's this issue around the engine software that caused the large VW emissions fall-out.

Then there's all the other things (auto breaking, line detection warnings, auto parking, ...) that cars are increasingly able to do. Most of these probably don't have much complex choices in them, but we trust that they work, and can guess how they work (for each of the examples I gave, then that's radar, opto-electronics, and magic).

Some of these would be better made public (especially following the emissions scandal); but I can't see companies that have developed (say) auto-parking wanting to share their code as this removes a commercial advantage from rivals that don't have it. It's the sort of thing that could be shared at a broad level, but that probably wouldn't count for much and it's the detail that usually matters.

[frank ly]

Hush a Bye Baby

We will look after you. Don't worry your pretty little heads.

[Pen-y-gors]

Walking directions?

So walking directions are an algorithm? Technically, yes.

But Google are saying we should put on a blindfold, get into a sealed box, and let them carry us to the desired destination, and trust them that we will get there, and not some other destination which they decide is 'better' for us.

[The Mole]

Re: Walking directions?

That's pretty much what i do whenever i get into a taxi.

[Loyal Commenter]

Re: Walking directions?

If your taxi driver starts off by blindfolding you, you should be worried.

[sabroni]

Re: That's pretty much what i do whenever i get into a taxi.

Really? You say "Take me home, or somewhere you think is better"?

[amanfromMars 1]

How many ways to say F*** Off? Let us count them

Why would anyone with a titter of wit and experience in the field of algorithms disclose and discuss proprietary 0day secrets with anyone steeped in ignorance of the facts which are sublimely drivering ongoing further future development programs and projects already running, and with many others primed for subsequent timely deployment to applications and services yet to be securely provided with mentoring and monitoring of ...... well, let us just call it Product so that one is not waylaid to waste time and effort down a narrower darkening path?

Isn't the usual default business way of doing such sensitive things not just simply to buy into Programs and Projects with the flashing of mountains of cash/the virtual transfer of outrageous paper wealth to a selected customer's bank account to allow them to proceed on one's behalf at apace which is not stalled by either arrogant or ignorant interference.

Methinks that is what Google will be telling, and may even have told, the House of Commons Science and Technology Committee's inquiry into algorithmic decision-making, which is questioning whether organisations should be more open about how machines influence such choices.

And Google DeepMind has already been asked for comment on this NEUKlearer HyperRadioProActive Program and AIdDevelopment .....

Search Engine Optimisation v2.0 [and above] is surely logically a Future Product Placements Engine …… Advanced Intelligence Resource with Immaculate Source, with the likes of a Google not searching for answers, both popular and controversial, but providing them with streams of supporting evidence.

Such would be akin to the Private Mentoring with Pirated Monitoring of Future Events with AIDerivative Programming for Projects/Semi-Autonomous, Self-Actualisation of Virtual Realities.

It is difficult, and maybe even impossible, to see or imagine a defence against such in an attacking configuration.

[Anonymous Coward]

Quis custodiet ipsos custodies?

So if Google folks become self-appointed guardians of our (secret) algorithmic wellbeing, who will guard the guardians? I seem to recall an ancient Roman emperor having a similar problem with his Praetorian Guard.

Seldom have I seen a better argument for Open Source than Google's despicable display of monopoly-guarding b*llsh*t.

[Mage]

Opening up the processes that underpin algorithms

Google WOULD claim it's just like a microwave or a washing machine.

However a false analogy. Google and Facebook are an online advert duopoly.

Both gather personal activities and information they should not.

Yes people have gamed Google's Search (an almost monopoly). Yes openness MIGHT make it easier. However really they are a publisher. They need to have better models to curate the data (or more humans) and more ethical revenue generation.

It's wrong that YouTube, Google Books and other Google sources as well as Wikipedia poison search results.

It's wrong that sites using Google Ads, Google APIs, Google analytics etc get boosted.

Google search is too much like a bookmark service too. It's becoming less useful as Google can't resist gaming the results.

It's best if no-one listens to Google or their shills doing lobbying.

['s water music]

gaming a black box microwave

They may have a point about security by obscurity. I have been trying to reverse engineer my microwave for years by loading it up with combinations of already cooked/heated food and radio sources such as mobile phones in an attempt to reverse the usual process and get electrickery back out of it. I quickly reached the point where I could get sparks (a form of electricity surely) but the electric meter keeps on turning forwards so far. Does anyone have a copy of the secret blueprints I could take a look at please?

[dirtygreen]

working demo

It seems to me that when somebody like google puts forward a contrary idea, the onus should be on them to show a working demonstration of the system that they propose.

[Loyal Commenter]

Security by Obscurity

It is mildly ironic that security by obscurity is a concept so thoroughly debunked that all you have to do is google™ the phrase to find out why.

To quote the first result (Wikipedia):

Security experts have rejected this view as far back as 1851, and advise that obscurity should never be the only security mechanism.

[Nick Kew]

Google is in an unusual position

Google's core algorithms that power search are affected by an unusual problem: they're under constant attack by spammers, for whom reverse-engineering those algorithms would be a Holy Grail. Every victory for those spammers is a defeat not only for google, but for all of us who use google for search.

Furthermore, google is a huge target for those spammers. A deep-pocketed "SEO" shop might justify a million-a-month R&D effort for the merest demonstrable advantage.

In those circumstances, it makes sense for Google to be less-than open about the detail of its algorithms, and to be constantly varying them.

[RyokuMas]

There are few things I trust less that the UK government, and Google is right up there at the top of that list - at least we get to elect our government every 4-5 years.

So in this case, I have to go with the lesser of two evils...

[Teiwaz]

at least we get to elect our government every 4-5 years.

So in this case, I have to go with the lesser of two evils...

Don't look to hard, I'm fairly certain we're only being allowed to vote on tie colour of late.

[Anonymous Coward]

To me the issue is not so much about the algorithms

but on how you can know what data is held on you is accurate (even governments can't seem to manage that)

and when the "computer says no" that you have some kind of recourse to a check of the decision and that it is not simply made on the basis of incorrect and incomplete data.

On another note

I am not sure where the definition of an algorithm would stop either, would you have to disclose all the logic in authentication and transaction handling or just out of context elements of business logic? sounds like a recipe (pardon pun) for disaster to me.

[Field Commander A9]

Can you patent an algorithm?

Hmm, no matter. Baidu and Yandex will find ways to pirate it anyway.

[Tom 7]

The thing about algorithms

is they largely fall out of the data. So with luck we can re-create the algorithms elsewhere. Or will this get patented too?

[Harry Stottle]

Commercial Accountability Theatre

is just as evil as political Accountability Theatre especially when the commercial entity is more powerful than most governments.

The solution is the same. Yes you may have legitimate secrets which need protecting but that doesn't mean that NO ONE outside your organisation should be allowed to access them. It just means we need a publicly trusted audit team to do the job on our behalf. In the case of Google and similar commercial giants, that implies a team of a few dozen, at least 2/3 of whom will need serious IT analytical skills including Security analysis. We also need representation from one or two Civil Liberty specialists (eg ACLU, LIBERTY etc) and all need to be bound by NDAs - unless they find evidence of illegality.

They need the time and budget to do the job properly and they need the right to access ANY parts of the system at ANY time, under appropriate secure monitoring of their own activities, in order to be able to confirm that everything is/was compliant with relevant regulation and remains so.

None of which would aid either competitors or gamers.

[JohnFen]

Google is right

"The real issue here is trust – and increased transparency "is not the only way" of achieving this, the Chocolate Factory went on."

This is correct, but since Google seems to acknowledge this, why can't they come up with an alternative way of building the trust that they're losing at an ever-increasing rate?

[amanfromMars 1]

Re: Google is right

Quite so, JohnFen. Do No Evil is not nearly enough whenever Doing Greater Good is the Minimum Default to actively transparently display the result of trusts put into the likes of a AI Google.

And if you would think that too alien and altruistic a concept for reasonable human consideration, please be assured that it is not, for Google DeepMinded, and they are but one spin-off feeding off and seeding into the Optimised Search Engine Arenas, are Offered a staggeringly simple but highly complex Internetworking Kernels Virtual Machine for the Presentation of Future Virtualised Reality Spaces in Earthly Places.

And Initial Offer Refusal and/or Markets Float Recusal does not Offend, for there are Vast Fertile Plains of Opportunity and Reward with any number of Wannabe Effective Leading Global Players/Greater IntelAIgent Gamers out there.

[andy 103]

TechUK

I'd like to know how much someone got paid to come up with that name.

Hmm, we're in the UK, and we're dealing with technology.... Yep, off to the pub now.

See also: British Gas.

[Anonymous Coward]

Why assume that anyone knows the current search algorithm?

If Google have started using machine learning then no person will actually understand what it's doing.

OTOH all the legally required adjustments to the results really should be published :-)

[Tim Seventh]

algorithm as microwaves, automobiles, and lights?

Seems like their engineers were not well educated. Microwaves, automobiles, and lights operation and fundamental concept were taught in elementary school to high school, from classes, to kids books, to school projects.

So if google's algorithm is the same, then surely some of its concept and operation will need to be taught in school. Oh wait they were trying to be secretive?

[Milton]

Trust!?

If the "key issue is trust" then I suggest the discussion is already over. Don't Be Evil deserve no more trust than any other sizeable organisation. It's surely no longer even debatable that large institutions such as governments and corporations develop the behaviours of psychopaths, where, unless subject to indepdendent oversight , they become ever greedier and less moral in their decision-making and policy. Hasn't the planet witnessed this about a billion times during the course of human history?

The reason we have the separation of powers in modern western democracies, and the reason America's founding fathers insisted upon ironclad freedom of speech and freedom of the press among other things, is the simple knowledge that without checks and balances, governments go bad. (It is, after all, exactly why Trump and his GOP lickspittles are assaulting the press, abusing the judiciary, trying to ignore scientific truth etc).

The reason civilised free-market societies have regulation, especially consumer protection, is the hard-won knowledge that otherwise, corporations can and will say and do anything to score an extra buck at the expense of the customer. Go look at some Victorian-era adverts for such things as "dimple scoops" if you need examples of how horrible things can be. The behaviour of the tobacco industry, food manufacturers with respect to sugar and car manufacturers most recently over diesel cheating, demonstrates quite clearly that corporations in the pursuit of profit will revert to shockingly barbaric, amoral behaviour if allowed to.

Only a blindfolded idiot would "trust" Google. Google exists to pick your pocket, not do you any favours. It didn't break into WiFi networks and harvest data during its StreetView patrols (and then lie, lie and lie some more about it) because it had your best interests at heart. It did it for money and power.

There are good reasons, based on history and human nature, ALWAYS to demand transparency and to hold institutions accountable to high and independently-verifiable standards. Frankly, only fools and scoundrels pretend otherwise.