back to article Amazon, Apple, Google, IBM, Microsoft speech-to-text AI systems can't understand black people as well as whites

Speech-recognition software developed by top tech firms struggle to understand black people compared to white people, according to research published this week. The study, led by academics at Stanford University and Georgetown University in America, probed five “state-of-the-art” cloud-hosted automated speech recognition (ASR …

  1. Elii

    Racist, really?

    Not sure how a computer can be “racist” exactly. Maybe there were more white voices sampled than black voices, so it’s inherently more accurate with white voices. That sampling may just reflect the demographic of the population. It could be that someone writing the software specifically wrote or such that it wasn’t able to recognise black voices as well, but I think that is pretty unlikely.

    Unfortunately with the current ridiculously over-PC atmosphere, if someone has the slightest opportunity call something or someone racist, they’ll take it.

    1. cornetman Silver badge

      Re: Racist, really?

      Indeed. Many intersectional feminists say these days (although obviously they are full of it) that racism is all about power. I'm not sure what kind of "power" a speech-to-text system can be said to have. It's just a program.

      I can't understand a lot of Scots. Does that make me racist?

      1. diodesign (Written by Reg staff) Silver badge

        "I can't understand a lot of Scots. Does that make me racist?"

        I can't understand most northern UK accents, I must admit. That doesn't make me a racist.

        However, if I was training a speech-recognition AI and left out a load of accents because I don't care for nor understand them, didn't think about them, or thought it would just work without them, then you might well be able to call me biased.


        1. big_D Silver badge

          Re: "I can't understand a lot of Scots. Does that make me racist?"

          It is the same with German. A lot of regional dialects are very difficult to understand. Platt has some English pronunciations, but also a lot of Dutch, so I understand more of that than many other dialects.

          Platt, Ostfriesisch, Schwäbisch, Bayerisch, Hessisch, Ruhrpott, Österreichisch, Schweizerdeutsch can be very difficult to follow, for a non-native speaker and also for many native German speakers. You will often get subtitles when people with regional accents are interviewed on TV, for example. Some of it is strong accents, but a lot of it is regional words and grammar that only exist there and have no bearing on "German". I suspect the same voice recognition engines would have similar problems in Germany, without even taking into account people of different race.

          Heck, Alexa was totally hopeless for me. About a 20% accuracy rate. It was usually quicker to "type" a search into the FireTV with that dreadful remote than it was to speak to Alexa. My British accent probably didn't help me, but my wife and her daughters are native German, but come from a Platt region and it didn't have much success with them either.

        2. Cederic Silver badge

          Re: "I can't understand a lot of Scots. Does that make me racist?"

          No, I'd call your work incomplete.

          Did you include people from rural locations in Tennessee too, because they have a very distinct accent and speech pattern. I won't even mention their skin colour because - get this - it just isn't relevant.

          Which is why it's a valid comparison.

          1. big_D Silver badge

            Re: "I can't understand a lot of Scots. Does that make me racist?"

            Or get a couple of people out of the Louisiana Bayous. They can be very difficult to understand.

            1. Anonymous Coward
              Anonymous Coward

              Re: "I can't understand a lot of Scots. Does that make me racist?"

              Watching some Canadian TV... I noticed one particular area, was not hard to understand, just had a habit of speaking very very quietly. :P

          2. jospanner

            Re: "I can't understand a lot of Scots. Does that make me racist?"

            It's not just 'incomplete', though, is it?

            Certain groups get favourable coverage in these things for a reason, and others get ignored.

            It's because you didn't think to include them. And you didn't think to include them *for a reason*. It's because you didn't consider it to be that important, if you even considered it at all.

      2. Anonymous Coward
        Anonymous Coward

        Re: Racist, really?

        I can't understand a lot of Scots. Does that make me racist?

        No, but it does mean you have a lot in common with voice-activated lifts or elevators...

        1. onemark03

          I can't understand a lot of Scots.

          Pedantry on:

          Sorry but your example is a bit off.

          The dialogue in the Scottish lift skit is Scottish English, not Scots, which - depending on your point of view or which textbooks you have read - is either a dialect or a variant of English or even a separate language. (I will not go into that debate here.)

          Pedantry off.

      3. jospanner

        Re: Racist, really?

        Yes, racism implies a power relationship.

        The words "<Ethnicity> sucks" have a very different function depending on the power of the person saying it, and the power of the ethnicity it is being said about.

        You might not *like* being challenged, but it doesn't equate to someone having legit worries about their security or safety.

        1. Anonymous Coward
          Anonymous Coward

          Re: Racist, really?

          Cool, so if we drop David Duke in the middle of Nigeria, he's suddenly not a racist?

          This is breathtakingly obvious nonsense, and I mean that in the sense that it's ridiculousness sucks the oxygen out of actual important racial issues.

    2. katrinab Silver badge

      Re: Racist, really?

      If you only recruit white people for your sample data, then that is probably racist.

    3. a_yank_lurker

      Re: Racist, really?

      I would not say racist as much idiotic. There are numerous North American accents. Some are difficult to understand the first time you hear them if you are not a local. The Left Coast accents are pretty bland as US accents go but some regions have a very strong accent even for whites. Now add in all the other English accents and I would like to see the error rates for say New Zealand, Wales, or Scotland.

    4. Mage Silver badge

      Re: Racist, really?

      Not just "black" people, but the entire Tech Industry tends to be White, Male and US Centric in the testing, default options, possible configurations etc.

      The USA is less than 13% of the world population. A lot of their citizens speak Spanish. Why does the stupid one key less USA layout exist, resulting in badly designed non-US keyboards on laptops, BT and USB mini UK Keyboards that are really the USA one overprinted, poor non-USA support till recently on Android external keyboards, only Latin-Roman fonts on early Kindles, though that was a problem solved 10 years earlier on the underlying OS.

      Absolutely everything having US Region, US Dictionary and Letter Size paper by default.

      So the fact that all these USA developers ignore the rest of the world and only train their speech matching (it's NOT recognition) on a stupidly narrow subset of speakers is no surprise at all. After all it would cost much more to use proper training data.

    5. jospanner

      Re: Racist, really?

      Do you also think that segregated drinking fountains can't be racist? You know, cos they're just drinking fountains?

      1. Imhotep

        Re: Racist, really?

        It's hard to believe you are being serious. Yes, I do think drinking fountains can't be racist. I do believe that people who mandate white only drinking fountains are racist.

        1. The Oncoming Scorn Silver badge

          Re: Racist, really?

          I remember South Africa opening its first mixed race swimming pools.

          The whites get the top half.....

          With apologies for ripping off Not The Nine O'clock News archives for that one, who also brought us Constable Savage.

      2. Another User

        Re: Racist, really?

        Looking at your registration date I tend to think that you are fairly young. This means you do not have first hand experience of segregation. You saw this in the Apple movie ‘The Banker’.

        Cool deduction with limited data available?

        Here is something regarding facial recognition

    6. Imhotep

      Re: Racist, really?

      In my experience "black" speech is mainly a function of environment - especially country - and not race. In college my macroeconomics professor was from Nigeria, and I had a hard time understanding him at first. His enunciation was not the problem, but the rhythm of his speech. Even later in the course, it took a couple of minutes for me to get in "sync" with him to understand what he was saying.

      I have noticed in my lifetime that the regional differences in speech are going away somewhat - perhaps because the media is bringing us to a more universal accent.

      1. Anonymous Coward
        Anonymous Coward

        Re: Racist, really?

        I’ve met some blank people from Africa that actually speak better than a lot of white people I know!

        1. Imhotep

          Re: Racist, really?

          Well sure, if you start with a blank slate....

    7. General Purpose

      Re: Racist, really?

      The word "racist" isn't in the article. The journalist didn't use it, the academics didn't use it. The only one taking "the slightest opportunity call something or someone racist", as you put it, was you.

  2. Dr.Flay

    The study is flawed as it assumes race creates an accent and speech patterns.

    The researchers should have directed their attention to the many hilarious videos in youtube, of mostly white people in the UK that do not have a BBC English accent, where they are pleading, shouting, screaming and swearing at Alexa, Siri or Google.

    1. diodesign (Written by Reg staff) Silver badge

      "it assumes race creates an accent and speech patterns"

      Sure, there are plenty of people, millions, many, many millions, who are non-white, but due to their upbringing, speak and sound exactly like the surrounding white population.

      This study isn't about people like them. It's about AI not having enough training data to recognize a *range* of accents. And it happens to be that the accents lacking coverage are those from black communities. There's the bias.


      1. Anonymous Coward
        Anonymous Coward

        Re: "it assumes race creates an accent and speech patterns"

        I think the point is that there are an awful lot of black Americans in the US, whereas the counter examples people are coming up with are small, isolated populations. Louisiana, for instance, has under 5000 people in 3/4 of its towns. Getting training data on potentially hundreds of different accents with tiny numbers of speakers would be hard.

        I will just note one example of the kind of problem that can arise. A colleague was visiting Kentucky in the 1990s on a QA assessment for a supplier. I get a phone call.

        -"They're talking about 'layered wire'. Does that make sense? I can't find the spec sheet."

        I dig out the spec sheet.

        -"It's lead."

        -"Led what?"

        -"Lead as in atomic number 82."

        Sudden illumination.

        -"Gotcha. Now I think of it, yesterday someone said I needed to 'hayung a rayut'."

      2. Mage Silver badge

        Re: "it assumes race creates an accent and speech patterns"

        Like the "black" Kenyan girl bullied in a USA school by "blacks" for "speaking white".

        A lot of what is called racism is plain bigotry and ignorance. Really so-called "racism" is a subset of prejudice, bigotry, xenophobia, ignorance and simply people that want to bully or control anyone not like them. So they belittle them. Race doesn't actually exist.

        1. Anonymous Coward
          Anonymous Coward

          Re: "it assumes race creates an accent and speech patterns"

          "Race doesn't actually exist."

          I agree with everything you wrote but race does actually exist, and I hope it continues so. As a heterosexual, it's very nice to have many races.

          1. Mage Silver badge

            Re: "it assumes race creates an accent and speech patterns"

            There is negligible difference genetically between a tall fair, blue eyed Scandinavian and an African bush person. There are more genetic differences within Africa.

            The tint of the skin or your height, eye or hair color or kind of hair is irrelevant to Speech Recognition. The very idea of Race was only invented to justify exploitation of various ethnic groups by more powerful groups.

        2. jospanner

          Re: "it assumes race creates an accent and speech patterns"

          Just sounds to me like people are more upset about being called racist than actual racism.

        3. Drew Scriver

          Re: "it assumes race creates an accent and speech patterns"

          How about we refer to homo sapiens as the human race and stop calling each other black and white?

          Look at the gentleman in the picture above the article. The device in front of him is black. The color does not match his skin.

          On a related note, I resent being labeled "white". When I put my hand on a piece of white printer paper it becomes quite obvious that my skin is not white. According to a paint scanner at the home improvement store it's a hue of red, actually. Yes, I did put my hand in it.

          This division of people into different groups called "races" is scientifically wrong-headed, especially if you look at the genetic make-up of different groups. It can also be exceedingly dangerous, as the theory of evolution dictates that different groups would have evolved differently and possibly at different rates. And yes, I'm sidestepping the debate about species.

          "I have a dream that my four little children will one day live in a nation where they will not be judged by the color of their skin but by the content of their character."

          Dr. Martin Luther King Jr., August 28, 1963

    2. Anonymous Coward
      Anonymous Coward


      Scotts Voice Recognition Lift.

  3. TheProf

    'I say, my good man, could you tell me where the bathroom is?'

    On holiday in America a few years back I discovered that I got a better response from people if I used a phoney American accent when asking them a question

    I'm not sure who was being racist.

    Also: I have a voice controlled model Dalek and that only seems to understand me if I do my Captain Picard impersonation.

    1. Sandtitz Silver badge

      Re: 'I say, my good man, could you tell me where the bathroom is?'

      "Also: I have a voice controlled model Dalek and that only seems to understand me if I do my Captain Picard impersonation."

      Shaved head, Picard Maneuver and everything? You could try Jar Jar Binks as well.

      1. TheProf

        Re: 'I say, my good man, could you tell me where the bathroom is?'

        "Shaved head, Picard Maneuver and everything?"

        I have the Picard haircut courtesy of nature. There's nothing natural about JJ Binks.

        1. The Oncoming Scorn Silver badge

          Re: 'I say, my good man, could you tell me where the bathroom is?'

          The still not quite ex Mrs Oncoming Scorn, had loads of fun* did trying to get On-Star voice recognition system to recognize her accent** in her car.

          *Well the rest of the car's occupants did, especially when it finally recognized what she said & asked for confirmation & the youngest shouted out No!

          **A mix of Essex & Devonshire.

  4. Sawari Ma

    They forgot to load the Jive language pack.

    1. Anonymous Coward
      Anonymous Coward

      Obligatory Airplane! clip

      1. The Oncoming Scorn Silver badge

        Re: Obligatory Airplane! clip

        I saw a rastafarian, crying on a street corner, bemoaning the fact his woman had left him, he looked depressed so I suggested he see a Doctor.

        A few days later he was in the same corner, dressed in a smart suit, full of confidence.

        I asked him why the change, what did the doctor say & had his lady returned?

        "No mahn.. I went to de doctors, he say I waz impotent. Hiz words made me tink, so I changed me clothing & life, because if I am impotent, then I gotta dress & look impotent."

        Accent from the Jim Davidsons book of politically incorrect material.

  5. The Nazz

    Racist , really (part ii)

    And what were the scores for other groups, dare i say the Browns, the Yellows, The Reds etc? NOT the Greens, they oft speak with plummy accents.

    Asians, Latinos, Hispanics, real Africans etc.

    Omitted completely.

  6. cornetman Silver badge

    > Black speakers are more likely to use African American Vernacular English (AAVE), a style of English that has its own grammatical rules and vocabulary.

    So, erm not English then.

    I'm sure if I used a lot of b*llshit words instead of well established and understood ones, it would have a hard time understanding me.

    1. FIA Silver badge

      So, erm not English then.

      There is no one definition of English.

      I'm sure if I used a lot of b*llshit words instead of well established and understood ones, it would have a hard time understanding me.

      What's a 'bullshit word'? You seem to expect everyone to talk like you do, the world doesn't work like that. Regional/cultural variations exist; they need to be dealt with.

    2. Anonymous Coward
      Anonymous Coward

      It is a matter of infinite regret to this interlocutor that thou thinkest that the manner in which thou expressest thine intellectual processes verbally is the only nice and accurate formulation of the language utilised by the people of England, and which is further required to be used semper, ubique et ad omnibus.

      It also peeves me that you think the only form of English allowed is the one you happen to have learnt.

      (Most Portuguese speakers neither live in Portugal nor speak with a Lisbon accent. And most English speakers do not use the Home Counties RP I grew up with either. But speech-to-text systems can generally understand me without any problems, other than the one the other day that repeatedly misheard my carefully articulated "ess" as "z". That is unusual; more often due to that bit of American weirdness they tend to hear "see" (third letter of alphabet) as z which they pronounce "zee". "Zed" is much simpler for an automated recognition system.)

    3. joeW

      Yes, proper words only. Such as "erm".

    4. jospanner

      It is English, you just didn't bother to consider it cos your idea of English comes exclusively from white, middle class English southerners, because you are actually quite ignorant.

      1. Anonymous Coward
        Anonymous Coward

        Third form of what I wrote, above. Except that I doubt he's a white middle class English southerner because (insert joke icon) we write better than that. And we know that the Register doesn't have a nanny censor for words like "bullshit". (Chaucer refers quite happily to a "shitten shepherd").

  7. joeW

    Based on my experience

    I'm guessing that it would still do better with Black American english than with my Irish english.

    1. Nige

      Re: Based on my experience

      Same here with a South Wales Valleys Welsh accent.

      I've given up on all the voice recognition phone lines when I try to call my bank or credit card providers. Its easier to just repeatedly pressing '0' or '*' or find the menu option to actually speak to a person.

  8. martinusher Silver badge


    Its not race, its patios, accents and figures of speech that these systems have yet to be trained on. I'm not black but I quite often have trouble with my friend in the corner, she's OK with American and RP English but if I slip into my native dialect (Cockney) then she gets really muddled. It could be that the folks at Amazon have designed the system deliberately to discriminate against me but I suspect its more to do with my failure to enunciate my words clearly.

    Learning to interact with these devices is like learning the rules of the road so you can drive. Its a nuisance and may not be the rules youi originally learned when you grew up elsewhere but its how we all get along and -- hopefully -- avoid colliding with others.

    1. TheMeerkat

      Re: Sigh.....

      It probably the same as with Black faces - lack of contrast makes the recognition more difficult.

      Some accents tend to have better contrast between words and sounds. I guess it makes “posh” accents easier to understand - it came by in a part of society that moved around more than the rest and, consequently, learn to speak in a way that more people understand them.

      1. This post has been deleted by its author

    2. Edward Noel

      Re: Sigh.....

      I don't think it's fair to blame patios. Patois on the other hand...

    3. jospanner

      Re: Sigh.....

      Speech tech is designed by people who speak like that. Why should we accept that there is only one "valid" accent because the designers were lazy and were used to everyone else bending over to suit them?

      1. John Brown (no body) Silver badge

        Re: Sigh.....

        It's the Apple and Microsoft way. Their way or the highway!

    4. John Brown (no body) Silver badge

      Re: Sigh.....

      "Its not race, its patios,"

      Patios? Aren't they the things you bury the bodies under?

      (don't you hate it when your fingers try to anticipate what your brain wants!)

  9. Randy Hudson

    Siri: Cut me some slack, Jack! Chump don' want no help, chump don't GET da help!

  10. TheMeerkat

    Accent is not an attribute of race. It comes with cultural background.

    1. Giovani Tapini

      Now, what about

      Boomhaur from King of the Hill..

  11. Draco

    "Black speakers are more likely to use African American Vernacular English (AAVE)"

    Is this true in Nigeria, Kenya, South Africa, Barbados, etc or is it only more likely in the USA?

    One can argue that the data set is insufficient broad or biased. One might also argue that a minor accent or dialect has less representation in a sample. There might also be a discrepancy in who is willing to give a voice sample. I was recently validating audio clips from Common Voice - Mozilla's initiative to sample as much human voice as possible and of the 120 audio clips I evaluated: 20 were female, 97 were male, and 3 had no voice. Given this is an open-source project and open to anyone who can navigate to it, why aren't more women contributing? I did not track accents and dialects - which are, mostly, beyond me.

    I am also willing to bet these speech-to-text systems fare poorly on English spoken with an accent - whether indigenous or "foreign" like Russian, or Chinese, or Filipino, or Spanish, etc.

    1. Mage Silver badge

      Re: "Black speakers are more likely to use African American Vernacular English (AAVE)"

      Most of the darker skinned people in the World are not USA Citizens.

      Most actual Africans, or non-Americans of African origin do not use African American Vernacular English (AAVE).

      Lots of Black people in the USA don't seem to use AAVE either. At least not the ones I met. Perhaps it's particular urban socio-economic groups in the USA that use AAVE.

      Personally I think Speech recognition has hardly improved since the 1990s and over 15 years ago when it was on devices locally. I suspect a major reason for the Speech Recognition using the remote server now is the invasive capture of your behaviour. Maybe it's really much better for a certain narrow class of White Americans.

  12. Mark192

    Conflating accent and race

    If they didn't control for accent/dialect, then they're conflating accent with race... a peculiar thing to do in a study... it's most often done as a lazy generalisation by people that haven't thought about the subject much.

  13. ragnar

    Is this actually a fair conclusion to draw?

    "CORAAL contains black speakers who use AAVE to various degrees, and hail from Princeville, a city known for its historic African American population, as well as Rochester, New York, and Washington DC.

    VOC is made up of white speakers from Sacramento and Humboldt County, California."

    Wouldn't it have been a better comparison to have a dataset made up of white and black speakers who are all either speaking English or all speaking African American Vernacular English?

    Since wikipedia says AAVE has its own unique grammar, vocabulary and accents, it's unsurprising that an AI dealing with standard English struggles. The AI may be equally good at recognising black speakers who are using conventional English grammar, vocabulary and accents but this study doesn't seem to allow us to draw a conclusion on that front.

  14. FIA Silver badge

    It's otherwise great for white people, or people who sound like them.

    What do 'white people' sound like? Do you mean 'Americans from a paticular geographical area?'

    I'm white, but with a reasonably strong regional (non American) accent and there's quite a lot of things Siri simply can't understand me saying. Ya gets me?

  15. Anonymous Coward
    Anonymous Coward

    <tongue in cheek> I bet those voice recognition systems are ageist, too, and only recognise the vernacular of a particular age range, too!. </tongue in cheek>

    Definitely not a matter of race IMO, but of culture. There's a heck of a lot of English variants and dialects that I can't understand either, but that doesn't make me racist or not interested in accessibility and inclusivity. It just means my limited processing power isn't up to the job of understanding much more than the dataset it was trained on when I was a nipper.

    1. jospanner

      If you didn't think to train your system on the set of data that actually represents your customer base because you just assumed everyone was like you?

      Yeah that's gonna be racism, classism, ageism etc.

    2. John Brown (no body) Silver badge

      Back when I was very young and just starting out at school, we were taught to speak "properly", ie a reasonably standardised English that every English speaker would be likely to understand. But we were also allowed to speak our local accents and use slang words too. We learned to adjust how we speak based on the situation.

      I'm not sure where UK schools currently stand, but there was a phase some years ago where English was seen as "not a problem, so long as the meaning is conveyed", which resulted in at least a generation coming out of school with very poor communication skills other than their local "street" English, let alone barely able to spell.

  16. Nige

    Learn To Spoke Proper England

    The brain has an amazing "universal translator" capability. Whether its due to the amount of training it has - lets face it, we have a lifetime's experience of listening to speech and speaking to others - or if its just an in-built ability when we have to interact with others who speak slower, faster, or with a different tone, inflection, or accent. And they're trying to train these voice recognition systems on a few 1000's of hours worth of recorded dialogue? No chance!

    I always cringe when I'm listening to the Radio and you hear some 'hip & trendy' reporter saying "wiv" and "vis" and "vat" and "sumpink", or people who can't seem to pronounce a 'T' and replace it with a glottal stop. FFS!

    When you're speaking to a wider audience, or maybe someone who isn't a native speaker of whatever your language is, it makes perfect sense to me to try to speak 'properly' and clearly pronounce and enunciate rather than the usual 'vernacular', shortcuts, slang, and lazy pronunciation that all too easily creeps in to common speech.

    1. jospanner

      Re: Learn To Spoke Proper England

      You just seem to be very upset that working class people exist and want to speak in the way that's most natural to us.

      1. John Brown (no body) Silver badge

        Re: Learn To Spoke Proper England

        He specified a "reporter". By definition, they are speaking to a wider audience, not their mates down the pub. They can talk "street", or "working class" if you prefer with their mates and no one will care. But when doing their professional job, which is primarily communication, then they should not be speaking with a local street accent and language.

        After all, it would be plain rude if I wrote this in my native Geordie accent since many here wouldn't be able to fully understand it.

        Gan canny man, Ahm awee hyem noo.

  17. Abdul-Alhazred

    Implicitly, the Turing test is for the AI to pass for a white human, and this is exactly the sort of error a white human would make.

    Turing test passed again.

  18. Anonymous Coward
    Anonymous Coward

    Understands most Indians well.

    I’ve been using the captions on hangout meetings to clarify what many of my off shore colleagues are saying to me. Works a treat on many but not all voices. It’s also not perfect and does go back and correct itself.

  19. BGatez

    been done

  20. jospanner

    Genuinely great watching the white middle class southerners show how they just expect everyone else in the world to bow down to them. The sheer arrogance of people for who society is engineered around throwing a hissy fit when asked to consider that other people exist.

    You designed the system based on your own biases. The problem is that it isn't being used by *you*, it is being used by a wide set of English speaking people. Stop being so self absorbed and understand that different regions and cultural groups have different accents and vocab.

    If that upsets you, tough. Grow up. Other customer facing jobs have to deal with the difficulties of human interaction. You're not special.

  21. Yorick

    Plight of the legal alien

    German living in the US - Siri can't understand me worth shit. "Route me to the Volvo dealer in West Springfield" ... "Calculating route to Audi dealer, is that correct?"

    I've given up. I only type now, my speaking very slowly to or worse, yelling at my phone is not a good look.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like