back to article Oh dear! Amazon's facial recognition is racist and sexist – and there's a JLaw deep fake that will make you want to tear out your eyes

Here's a roundup of this week's other AI news. In short: experts continue to snub Amazon's facial recognition service Rekognition, and there's a new deepfake for you to stare at in horror. China and the US are miles ahead: A study compiled by the UN World Intellectual Property Organization found that China and the US both …

  1. Adrian 4 Silver badge


    Since I've no idea who either of them are, the mashup looks perfectly reasonable.

    Maybe you could do someone well known - Trump and Hillary, perhaps ?

    1. jake Silver badge

      Re: mashup

      I agree with this post.

      The two must be outside my demographic, as is usually the case with pop culture.

      1. Sandtitz Silver badge

        Re: mashup

        "The two must be outside my demographic"

        Your loss. If you ever decide to start watching movies, Steve Buscemi has had several great roles in several great films such as Reservoir Dogs, Big Lebowski and Fargo.

        1. Craig 2

          Re: mashup

          While your (presumably intentional) omission of Jennifer Lawrence's notable movies is humorous, Silver Linings Playbook is also a great film. :)

          1. Timmy B

            Re: mashup

            Winters Bone is an excellent, but certainly not happy, film. If you don't think she can act watch this.

        2. Stevie

          Re: mashup

          Yet you fail to mention his Magnum Opus: Willy in Grown Ups.

          Steve Buscemi is one of those rare actors who can commit to just about any project and come out with no dinging-up of his considerable and well earned reputation.

          Oh, and I'm going to name-check Boardwalk Empire, which if you haven't seen you should.

    2. Anonymous Coward
      Anonymous Coward

      Re: mashup

      It looks like Frank Skinner in a frock.

    3. H.Winter

      Re: mashup

      How do you not know Steve Buscemi? He was the guy who (spoilers) got fed into the wood chipper in Fargo (the movie).

  2. Nolveys

    Rekognition was being sold to government agencies and could be inaccurate - especially when trying to identify people with darker skin tones.

    Reminds me of an episode of Better Off Ted in which they install facial recognition to activate everything from elevators to water fountains. The only problem is that the system can't see black people. Instead of admiting the system is a failure they install separate water fountains for black employees as well as designating them to specific elevators...

    1. Anonymous Coward
      Anonymous Coward

      Rekognition was being sold to government agencies and could be inaccurate - especially when trying to identify people with darker skin tones.

      My conclusion form this article, from its own links, and from *cough* some of the "new content" is that although the likes of IBM, Google, Uber etc are truly useless at making AI do ANYTHING well, a few enthusiastic miscreants have actually done something rather well from a technical viewpoint (whether anybody else approves or not).

      So let's ask the difficult question: In particular directed to Commentards who are not white - are non-white individuals disadvantaged or advantaged by the fact that the crappy big company AI can't tell them from rather too many other non-white individuals? Speaking as a white male, I see ZERO advantage to me that face recognition works well on my ethnicity, I'd value your views how all this works from your perspective.

      Assuming that other people don't think the same way as I do, and still WANT people with darker skin tones to be as readily identifiable (and presumably "manipulable") as lighter skin tones, because somehow that's "fairer". then combining the two points above, the way forward is clear: Ignore big companies, and just engage the same people who made deepfakery work for pr0n, rather than leaving the matter to the tech-sector's pompous, self righteous and probably overwhelmingly white milennials. I suspect there's enough thought there to be thoroughly contentious and start at least one civil war, but having read it several times, I think it's all balanced and reasonable - speak out if you disagree.

      1. jake Silver badge

        Point of order requested.

        Do you actually mean "non white", or do you really mean "non caucasian"? Makes a difference to some of us ...

        1. W.S.Gosset Silver badge

          Re: Point of order requested.

          Yeah, "caucasian" is one of those anthropological terms which has been warped to mean something else. "Caucasian" is actually a skull shape, not a skin colour. Most Somalis, for example, are caucasian.

          "Semitic" is another one. Most jews are not semitic, most arabs are.

      2. Dabbb

        Ha, I knew all blacks and asians look the same, now I have a proof !

        To be honest whites also look all the same to them, so I guess chinese facial recognition system would have exactly the same issues with caucasians.

      3. BigSLitleP

        He's a really, really, REALLY obvious disadvantage for black people.....

        "I wasn't even in the CITY, let alone at the place that got robbed!"

        "Well, this AI thing says you are a perfect match and we all know it works brilliantly so it's off to jail for you!"

        Another case solved.....

      4. Stevie

        "... the likes of IBM, Google, Uber etc are truly useless at making AI do ANYTHING well ..."

        No, I won't have that!

        IBM's "Watson" has slowed the loading time of The Weather Channel's app to a crawl while at the same time adding nil to the positive user experience side of the equation.

  3. jake Silver badge


    "Oh dear, since there is currently no governmental regulation around the use facial recognition, the police can, technically, use it however it wants."

    And anybody who erroneously gets arrested based on the flawed technology can (and will!) sue the department which made the mistake of using a clearly not-ready-for-prime-time system.

    Of course, it'll be settled out of court. Probably with a high six figure or low seven figure payout, depending on the law firm retained. Throw in a non-disclosure agreement, so it never gets picked up by the press. Lather, rinse, repeat. Lovely use of your local tax dollars, citizens. Going to get off your fat asses and say anything about it?

    1. A.P. Veening Silver badge

      Re: Brilliant.

      "Throw in a non-disclosure agreement, so it never gets picked up by the press."

      Here in the Netherlands non-disclosure agreements were judged illegal (in a completely different problem). The press had a field day. I wonder how long it will take elsewhere for non-disclosure agreements to be judged illegal. Seems to me those already should be for anything involving government, either directly or indirectly as the public does have a right to know.

  4. Rajiv_Chaudri

    Are Americans this stupid in their assessment of AI? Nature, physics, & natural selection are by definition "biased" and can be misinterpreted as "racist".

    Nature doesn't care about "equal outcome." Nature provides equal opportunity and lets natural selection bias weed out the weak. Nature is harsh, but fair to all. Nature doesn't give a sh!t about your feelings.

    1. jake Silver badge

      "Are Americans this stupid in their assessment of AI?"

      No, Rajiv_Chaudri, just extremists like the ACLU, the extreme right wing, and other non-scientific bodies, like Congress and the Senate and the Executive Branch and the Judiciary. The vast majority of us have clues. Kind of like in your own country, whichever that is.

    2. Anonymous Coward
      Anonymous Coward

      Where to begin on your post? How is physics biased? It is what it is. If you show me an EM field that discriminates between black and white electrons, I'll pay attention.

      The main problem with your post is anthropomorphism. "Nature" doesn't exist as an entity. You repeatedly anthropomorphise it. That's a view which had some kind of credibility up to about 1853, when an English clergyman put the kybosh on the idea of a God running the world and somehow supervising the arrangement of living things. Nowadays even the Catholic Church has to dance around the handbags a bit on the subject.

      Organisms are not provided with equal opportunity - they exist in an ecosystem which is what it is, and they rise and fall according to "fitness" - how well they are able to make use of energy sources and reproduce. There is no "weeding out of the weak" because (a) there is nothing to do the weeding and (b) it doesn't work like that - the big dinosaurs included some pretty top level predators but a sudden attack of climate change drove them to extinction while the mice,voles, frogs and birds survived. Natural selection is merely a descriptive placeholder for a very complex phenomenon indeed, one which leads to wasps using caterpillars as living food for larvae - but the caterpillar population continues to exist because despite being predated by the "stronger" wasps, they are capable of outreproducing the wasp predation. The race is not always to the swift nor is the battle always to the strong.

      It is misconceptions like yours that provide a lever for the Creationists, and you shouldn't repeat them.

      Meanwhile, back on topic, and by contrast, Amazon, Facebook and the like do exist, they are commercial organisations supposed to function in societies which very properly have laws against discrimination, and they are failing to do so. Handwaving about a nonexistent posited entity behaving similarly is of no significance whatsoever.

      1. Yet Another Anonymous coward Silver badge

        >If you show me an EM field that discriminates between black and white electrons,

        We do live in a world incredibly hostile to "differently charged" electrons

        Positive electrons need our support, at the moment they can only live isolated from the main stream.

        Remember anti-matter particles matter !

        1. Lyle Dietz

          > Remember anti-matter particles matter !

          They don't matter though, they anti-matter...

          I'll just be leaving before I get mugged by some positrons...

      2. BigSLitleP

        I don't know what's worse:

        1) The fact that the OP doesn't understand racism, evolution or society in general

        2) So many people upvoted him, showing their ignorance too

        3) The downvotes you received, showing people don't like to admit they were wrong

      3. P. Lee

        And so the feminist dream inches closer!

        >they were misidentified as men 19 per cent of the time


        But to other points, "fitness" in evolutionary terms has a circular definition. It's fit if it survives and survives if it's fit. Evolution as a theory does not appear to have any predictive capability. A pointless theory in my view.

        With regard to nature and equal opportunity, the point is that all organisms face the environment on their own terms, rather than having the deck stacked against them by politics. White people do worse in hot deserts and black people do worse in northern latitudes, but nature doesn't prestack the deck by raising the survival bar in order to favour one group or another. Unlike the fire service and some American universities, Nature provides the same survival threshold for all.

        If the anthropomorphism triggers you, think of "nature" as a contraction of "natural forces".

        1. Rich 11 Silver badge

          Re: And so the feminist dream inches closer!

          Evolution as a theory does not appear to have any predictive capability.

          In the early 1950s Alexander Fleming predicted the risk of increased antibiotic resistance, caused by evolutionary pressure arising from a widespread mis-application of antibiotics.

    3. Anonymous Coward
      Anonymous Coward

      "Nature is harsh, but fair to all. Nature doesn't give a sh!t about your feelings."

      That's why in India they ensured that a strong caste systems takes care of giving a subset of the population a clear advantage over the others, again, based on skin tone...

      Bias and racism are not a problem in US only....

    4. Warm Braw Silver badge

      stupid in their assessment of AI

      Punched cards are not racist, but it didn't stop them being used by Nazi Germany to facilitate their ethnic identification of the population*.

      The problem with technology being "neutral" is that if you are lazy or unscupulous, you can feed it with biased data and then claim the outcome is unbiased because it was created by a machine. Technology does not exist out of the context in which it is operated - you have to consider the whole system, including its social context, to understand how it really works.

      In this case, all that is being claimed is that the training data is not adequately representative of the general population and consequently the recognisition results are, at best, patchy.

      It's probably also likely that to recognise minority characteristics of any sort in a large population that you would need samples that significantly over-represent their occurrence in order to have equivalent levels of training data for those specific characteristics. That might simply be impossible to achieve in practice given that the technologists seem to find it hard to find data sets that even approach a true proportion of minority samples. That's doesn't make the technology racist, but it does mean it simply won't work. The problem then becomes a human one - we have technology that doesn't work but we want to recover our investment: let's find some people who'll buy it regardless and to hell with the consequences. That's when it becomes racist.

      *I think a Godwin exception is permissible in this case.

      1. jake Silver badge

        "I think a Godwin exception is permissible in this case."

        I don't think what Mike wrote in 1990 means what you think it means. For your education, here it is in it's entirety: "As an online discussion grows longer, the probability of a comparison involving Nazis or Hitler approaches 1.".

        Godwin himself further wrote "its purpose has always been rhetorical and pedagogical: I wanted folks who glibly compared someone else to Hitler to think a bit harder about the Holocaust."

        Nowhere does it suggest never mentioning Hitler or Nazis. On the contrary, in fact. All it suggests is that you think before posting.

      2. Baldrickk

        There may also be technological issues beyond any bias in the training data. It could be because imagery of people with darker skin naturally ends up with less contrast, making it harder for the system to pick out the relevant details.

        Think Polar Bears in the snow.

        1. Carpet Deal 'em

          Maybe not "polar bears in the snow", but setting up a camera that can give you a good look at all skintones at once can be kind of difficult.

          1. Baldrickk

            I was exaggerating, but then again polar bears are also not snow-white ;)

  5. Anonymous Coward
    Anonymous Coward

    Deepfake entertainment potential

    I keep meaning to try to put David Cameron's face on the video of Jonathan Pie telling Theresa May that it's time for her to go.

    I know I never will, but it would be hilarious. Someone do it?

  6. Anonymous Coward
    Anonymous Coward

    There are two main problems with facial recognition

    Firstly inaccuracy, where it gives a false positive. This combined with racial bias gives the "all you darkies/asians/wogs look the same, and you're all crims anyway...", so traditional police work, brought into the century of the fruitbat.

    Secondly.. accuracy, where it empowers the surveillance state, and allows intrusive and pervasive monitoring of citizens. The powers that use monitoring have displayed no interest in limiting their own behavior - in evolutionary terms they have found a vacant niche and will exploit it until something stops them.

    So, facial recognition is bad when its wrong, and bad when its right - and the people selling it, and the people using it dont particularly care.

  7. Anonymous Coward
    Anonymous Coward

    "pictures of women - they were misidentified as men 19 per cent of the time..."

    Bearing in mind that gender is now considered a social construct, I would say that's a pretty good hit rate! I certainly couldn't do any better.

    1. LDS Silver badge

      Re: "pictures of women - they were misidentified as men 19 per cent of the time..."

      Good, they will have an opportunity to get a better paid job and be promoted over those identified as women....

      And which is the percentage of men identified as women?

  8. Skribblez

    So techbros produced a system that can only recognize techbros? Is anyone really surprised?

  9. Anonymous Coward
    Anonymous Coward

    That is a pretty good "deep fake" for someone in their spare time (I presume it was their spare time). Just think what a large sophisticated organisation with a lot of resources or a government agency could do.

    However I don't think the thing to worry about is that significant people could be shown doing or saying something that they didn't. I think the fact that they can get plausible deniability for something they actually did do and get away with anything where there wasn't more than a few witnesses is more worrying.

    It's all fake news > it's just a deep fake.

  10. SVV

    We do not set nor do we utilize a confidence threshold

    This is the problem with AI facial recognition tech : not the fact that it it doesn't work accurately, but the fact that politicians and police forces are so desperate to have it and use it, that they are doing so even though it doesn't work accurately.

    1. W.S.Gosset Silver badge

      Re: We do not set nor do we utilize a confidence threshold

      Yeah, DNA tests are a good example. The standard police DNA test in the UK is much less accurate than that in the States, but is accorded the same "power"/certainty in Court.

  11. Anonymous Coward
    Anonymous Coward

    Rekognition performed worse when analysing pictures of women - they were misidentified as men 19 per cent of the time - and results dropped even further when the women were black.

    Since any bloke can now claim to be a woman - keeping the beard, hairstyle and male clothing is fine - it's hard to see how any system could tell who is what.

    1. Claverhouse Silver badge

      Since any bloke can now claim to be a woman - keeping the beard, hairstyle and male clothing is fine - it's hard to see how any system could tell who is what

      Now we need security tech that can provide windows into men's souls to determine reality; or to determine what the subject 'thinks' is reality; or what the inquisitor 'thinks' is reality; and what do we mean by the concept 'thinks' anyway ?

  12. Eddy Ito

    Not impressed.

    The skin tone change at the neckline is an obvious tell.

  13. The Nazz

    30degree skewed

    Is it just me, or is it a recent trend that all the photos of women i've seen lately** now show the women facing the camera at an angle, not quite side on? Make no wonder they're difficult to recognise.

    **Frequent on Twitter, but also shown this way in many recent BBC "news" articles?

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2022