back to article Oh dear... AI models used to flag hate speech online are, er, racist against black people

The internet is filled with trolls spewing hate speech, but machine learning algorithms can’t help us clean up the mess. A paper from computer scientists from the University of Washington, Carnegie Mellon University, and the Allen Institute for Artificial Intelligence, found that machines were more likely to flag tweets from …

  1. Anonymous Coward
    Anonymous Coward

    “I saw his ass yesterday”

    It's only mildly crude, and then only if you presume that the object of the sentence isn't a donkey.

    If "Perspective API" thinks that sort of thing is toxic, I'm inclined to think not of racism, but rather of a pathetically under-trained AI, and of people who are ignorant of the true state of the AI art.

    1. big_D

      Re: “I saw his ass yesterday”

      And I haven't seen a setting in Twitter that lets me define what language I am posting in - I post in Germany a (British) English.

      If AAE is not an official language and if the text isn't marked as AAE, how is it supposed to differentiate between plain English and AAE? The only thing you can do is try and recognise the base language and then flag any derogatory or racial words.

      Given I don't live in America, the whole topic of AAE was new to me, upon reading this post.

      1. Chris G

        Re: “I saw his ass yesterday”

        Adding to the potential for increasing bias or lack of flexibility, is the fact that, at street level, particularly with younger speakers who then take the idioms of youth with them as they mature. Language evolves and frequently what may have been rude before becomes an acceptable part of every day speech.

        The regular language and even subject matter on TV such as the Graham Norton show for example, would have caused all kinds of trouble back when I lived in the UK.

        How does an AI stay updated to the constant evolution of language.

        Aside from it not really being intelligent.

        1. big_D

          Re: “I saw his ass yesterday”

          That is a big part of the problem. The other problem is how do you differentiate between AAE posted by an African American and a piece of racist text not posted by an African American, when all you have to go on is the text, and possibly a nondescript pseudonym?

          1. tfewster
            Facepalm

            Re: “I saw his ass yesterday”

            But there's lot's of context for AI/ML to make a judgement with - The other words in the text, the history of the person posting, the audience. Y'know, the sort of task a computer should be good at? If it can make sense of a sentence rather than just picking key words out.

            Of course, that means labelling people by their history - which is rarely a good thing.

            1. caffeine addict

              Re: “I saw his ass yesterday”

              The other words in the text, the history of the person posting, the audience

              That simply means that someone who is consistently racist doesn't get flagged as racist...

          2. Grooke
            Joke

            Re: “I saw his ass yesterday”

            Use the profile picture and make biases work for you! If facial recognition thinks the user should go to prison, assume they're black and flag the text as AAE. Use bias to defeat bias!

      2. TheVogon

        Re: “I saw his ass yesterday”

        It's English. Not British English. It's the original so needs no qualification unlike say American English.

        1. Korev Silver badge

          Re: “I saw his ass yesterday”

          I prefer the term Queen's English

          1. Korev Silver badge

            Re: “I saw his ass yesterday”

            The word Queen actually being an (accidentally) good example of why context is kind of important

          2. TheVogon

            Re: “I saw his ass yesterday”

            The Queen's English is English + Received Pronunciation. So not quite the same thing.

        2. The Indomitable Gall

          Re: “I saw his ass yesterday”

          I think Shakespeare, Chaucer et al. would dispute your use of the term "the original".

          Thy coarse neologisms cause great pain upon mine ear. Get thee to a nunnery!

    2. AMBxx Silver badge

      It's time we just abandoned all attempts to control Twitter. Just remove all censorship and let it sink into a vile mess. Then we can all do something more interesting instead.

      1. jake Silver badge

        "let it sink into a vile mess"

        How? Can't sink much lower than it is already.

    3. TheVogon

      Re: “I saw his ass yesterday”

      I'm not clear how references to Donkeys are crude?

      1. jake Silver badge

        Re: “I saw his ass yesterday”

        Oh look. Somebody aiming to appear intelligent, and missing the target completely. How cute.

      2. Anonymous Coward
        Anonymous Coward

        Re: “I saw his ass yesterday”

        It just means that they only got the Artificial part right.

    4. ThatOne Silver badge
      Facepalm

      Re: “I saw his ass yesterday”

      Never mind sightings of someone's Gluteus Maximus:

      > “I saw him yesterday” is scored as 6 per cent toxic

      That part really stunned me: What about this simple factual phrase could be considered 6% toxic? This forces me to consider that anything coming out of this AI filter is semi-random nonsense to start with, before even starting to think about issues of language, communication and cultural taboos.

      1. Claptrap314 Silver badge

        Re: “I saw his ass yesterday”

        It's toxic because of the use of a gender-assumptive pronoun.

        Toxic to snowflakes, that is.

        You know, maybe if we all just use enough toxic speech, all these snowflakes will die and the rest of us can get back to writing code, or what ever it is that we do to earn other people's money.

        1. CrazyOldCatMan Silver badge

          Re: “I saw his ass yesterday”

          Toxic to snowflakes

          There's a useful phrase: "to the pure, all things are pure".

          The intent behind that phrase is that your attitude and response to things come from who you are and what you expect. So, if you expect to be offended by something, then you will be. If you don't see everything as suspicious, then you won't find it suspicious.. etc etc.

          Sure - there are people that will nitpick about things other people say (to the nth degree often) and assume that every word that comes out of someones mouth is deliberately designed to offend. The fact is, most people really are not that careful about the exact words they say.

          Lastly: We judge ourselves by our intentions but judge others by their deeds.

          1. Cederic Silver badge

            Re: “I saw his ass yesterday”

            I forget who, but on Friday I watched a lady being interviewed. She's been running something important (a football club, I think) for a few years, and near the end of the interview got asked something along the lines of, "It's almost strange that we haven't asked this already, but do you experience sexism in your role?"

            Her answer was that she didn't, and then she added, "but then, I don't look for it."

            I thought that was a lovely response, and entirely supports your supposition.

            1. Anonymous Coward
              Anonymous Coward

              Re: “I saw his ass yesterday”

              What if the AI flagged the word "saw" as possibly using a serrated blade upon a portion of the person's anatomy (or his hooved animal)? Without knowing what exactly caused this phrase to be flagged, it is impossible to judge whether the AI can be correct.

      2. The Indomitable Gall

        Re: 6% toxic

        It is not 6% toxic. they shouldn't have used percent to describe it, because it's not really a percentage or a proportion. Most machine learning algorithms score everything in the range (0,1], never scoring zero, because there's lots of multiplication.

        1. ThatOne Silver badge
          Unhappy

          Re: 6% toxic

          > Most machine learning algorithms score everything in the range (0,1], never scoring zero

          Thanks, but for any humans meant to use those results it simply means nobody is ever innocent, some people are just less guilty than others. That's a nice premise to build upon, repressive governments must love this...

          1. jake Silver badge

            Re: 6% toxic

            "it simply means nobody is ever innocent"

            Do you, personally, know anybody who is innocent in the context of TOA? Because I sure as hell don't. Humans are fallible, and IMO it's a part of what makes us human.

            There is a reason the parable about casting the first stone exists, and I rather suspect that it was a common concept long before the biblical version made it into print script.

            1. ThatOne Silver badge

              Re: 6% toxic

              > Humans are fallible, and IMO it's a part of what makes us human.

              That's the part to remember indeed. But (many/most) humans need clear-cut categories to manage the world around them, which means setting or playing with those categories is a very serious thing: Appearances are much more important than they should, the saying "give a dog a bad name and hang him" comes to mind.

              My point being that we (humans) have to be very careful about such things, for there will be a strong and constant tendency to use them for nefarious goals. Because humans are fallible.

            2. Anonymous Coward
              Anonymous Coward

              Re: 6% toxic

              "Do you, personally, know anybody who is innocent in the context of TOA?"

              That's the whole idea: You can put *anyone* in jail because of this kind of crime. Perfect tool for dictators.

              Which leads the question who is paying for creating tools like this? And why?

              1. CrazyOldCatMan Silver badge

                Re: 6% toxic

                You can put *anyone* in jail because of this kind of crime

                "Give me six words from an innocent man and I will give you enough to hang him."

                1. Claptrap314 Silver badge

                  Re: 6% toxic

                  "six lines". The cardinal was not THAT aggressive.

        2. Anonymous Coward
          Anonymous Coward

          Re: 6% toxic

          "It is not 6% toxic."

          While TFA says it is. So it's "his" which makes the 6% as there's nothing else than can cause it. Nice: Now you are 6% racist because you didn't use gender-independent pronoun.

          "hers" obviously would be only 3% toxic and "its" 0%.

  2. macjules
    Paris Hilton

    crafted in the style of African American English (AAE)

    But if it involves blocking profanity then isn't it doing it's job correctly? Surely, just because said profanity is a part of "African American English (AAE)" does not make it any less profane.

    Perhaps we should have a "Father Jack Hackett English (FJHE)" where we can cry foul at being blocked for tweeting "Arse, feck, whiskey"?

    1. OssianScotland

      Re: crafted in the style of African American English (AAE)

      Gurrrllls, Shirley?

    2. Anonymous Coward
      Anonymous Coward

      But if it involves blocking profanity then isn't it doing it's job correctly?

      Er, the whole point of the article is that "profanity" is very much a function of context which is something that the average human (which should be the comparator for "AI") can spot a mile off, and which "AI" doesn't even know exists.

      Scunthorpe to you.

      I refer you to the excellent Stewart Lee Comedy Vehicle S3E4 "Context" where he rips the piss out of the idea of a "context free word" which ends by a refusal to tell a joke as it's so offensive it would cause a tsunami of complaints although every single word in the joke appeared in the previous 20 minutes.

      Anyone who has tried to search Google and experienced the massive context fail of "smart" algorithms will know what I mean.

    3. iron

      Re: crafted in the style of African American English (AAE)

      That would be an ecumenical matter.

      1. Jimmy2Cows Silver badge
        Angel

        Re: crafted in the style of African American English (AAE)

        Careful now.

      2. Korev Silver badge
        Pint

        Re: crafted in the style of African American English (AAE)

        Yes!

    4. FrogsAndChips Silver badge

      Re: crafted in the style of African American English (AAE)

      The objective was not to flag profanity, but to flag hate speech. The tool decided that the word "nigger" would essentially be used by racist people in their hate speech, without realizing that it could be used casually by AAE-speaking groups. Sounds like this "AI" only took some keywords into account, blissfully ignoring the rest of the text.

      1. steviebuk Silver badge

        Re: crafted in the style of African American English (AAE)

        Bit like "Whasup niggers" in Shawn of The Dead. I'm assuming the AI would ignore the context and flag it as offensive. If so, surely its not proper AI. Surely AI should be able to work out the context.

        1. Anonymous Coward
          Anonymous Coward

          Re: crafted in the style of African American English (AAE)

          "Surely AI should be able to work out the context."

          That's exactly what it can't do. It doesn't *understand* anything, it just looks for patterns, i.e. words, and label those as "hate speech" and the people writing them as "criminal".

          So easy when false positives are irrelevant to company running the "AI" as there's nothing intelligent in those.

        2. CrazyOldCatMan Silver badge

          Re: crafted in the style of African American English (AAE)

          Surely AI should be able to work out the context

          Well, given that an awful lot of humans can't, it's a bit much to expect a primitive and badly-taught rules engine to do the same..

      2. John Brown (no body) Silver badge

        Re: crafted in the style of African American English (AAE)

        "The tool decided that the word "nigger" would essentially be used by racist people in their hate speech, without realizing that it could be used casually by AAE-speaking groups."

        A colleague of mine was shocked and offended while in London because black people were using the N word in casual conversation. As he said to me, "how the fuck do we defeat racism and get these words out of the language when "they" are using them all day every day?". They used the words in his presence because he happened to be of a similar skin tone and so were effectively making racist and cultural assumptions about him. He's 4th generation British of mainly Somali descent.

        1. Anonymous Coward
          Anonymous Coward

          Re: crafted in the style of African American English (AAE)

          " "how the fuck do we defeat racism and get these words out of the language when "they" are using them all day every day?". "

          This kind of idiots are The Problem(TM). *Words* do not make racism: They are just groups of letters which have no meaning whatsoever outside of what *you* assign to them.

          Ban one word and actual racists invent something else and then you have to ban that too, for ever.

          Here in North we haven't had black people until late 90s and word 'nigger' (or local translation of it) doesn't mean anything to locals, even to those who speak English: it's just a word from old books meaning black people, often slaves in US. Same way as calling brown haired person brunette.

          Whole context as an insult is missing as it never was there to begin with. And still local jesuits demanded it had to be removed from everywhere as *they* *thought* it as an insult. While being totally wrong, we've actual insults. But stupid people are stupid. And jesuits, of course.

          "My nigga" is a common greeting nowadays, I've heard. Just like "my man". If someone calls me "my nigga" (while me being white as milk) I'd take that as a compliment: 'I'm accepted to tribe'.

          Trying to teach AI to sort something like that is nigh impossible and only reflects the racism of those people who've taught the AI. Or incompence and greed: Any BS to get paid.

          *They* would use it as an insult, so it must be an insult to everyone else too. In any context as "AI" has no understanding of the context. It barely can understand that "ass" can mean more than one thing.

          1. Cederic Silver badge

            Re: crafted in the style of African American English (AAE)

            However, without challenging your primary point about whether words should be removed from the language or not, I will state that allowing or disallowing use of a word based on the skin colour of the person using it is textbook racism.

            Nobody I work with will be using the word nigger in the office, because I'll be stopping that immediately. That goes for nigga too, as that's frankly the same word; changing the spelling doesn't change the social acceptability.

    5. katrinab Silver badge

      Re: crafted in the style of African American English (AAE)

      To a speaker of African American English, "Ass" is about as offensive as "Person" or "Face", ie not offensive at all. So, treating the word as offensive, without considering context, is racist.

      1. Zack Mollusc

        Re: crafted in the style of African American English (AAE)

        I don't think racism and offence work that way. A statement is made by entity A. Entity B hears the statement and decides if it offends entity B . If entity B is offended then Entity A is racist for making the statement.

        1. Anonymous Coward
          Anonymous Coward

          Re: crafted in the style of African American English (AAE)

          " If entity B is offended"

          And it's totally up to entity B to decide if they are offended or not. So totally arbitary and random *subjective* thing decides if someone else is a racist? Totally regardless of what they *meant'?

          Genuine thought crime: Orwell would be proud of that.

          "I'm pissed, I'm offended by anything and everything"-> everyone saying anything is a racist today. Because that's how it is going to happen in real life.

          Nope, it doesn't work that way and *can't* work that way.

          "Being offended" is basically lack of emotional control and thoroughly subjective thing which won't give any rights whatsoever. It must not make anyone else criminal, ever.

        2. jake Silver badge

          Re: crafted in the style of African American English (AAE)

          "If entity B is offended then Entity A is racist for making the statement."

          I am offended by your statement, therefore you are racist?

          OK, if you say so, you racist, you.

      2. Anonymous Coward
        Anonymous Coward

        Re: crafted in the style of African American English (AAE)

        "So, treating the word as offensive, without considering context, is racist."

        Obviously. The other side these racists do not want to see or acknowledge.

  3. JimmyPage
    Boffin

    Is anyone surprised ? Really ?

    The one constant about "AI" these past few years has been how patently obvious it should be that it isn't in any way shape or form "Intelligent". If there is "AI" (I always quote it, because clearly what I think it is, and what people telling me it is are two different beasts) then Amazon and Google searches would not be as shite as they are (and getting more so).

    No amount of clever keyword wrangling is ever going to match true intelligence - which involves KNOWING WHAT THE WORDS MEAN. Not what your database of lexical connections thinks "meaning" is. But ACTUAL MEANING.

    You can do a lot with sophisticated pattern matching, and deep data linking. But you can't be "intelligent" doing it.

    1. Tom 7

      Re: Is anyone surprised ? Really ?

      TBF if you feed it data humans have created on the WWW then its going to look pretty stupid. GIGO still holds.

    2. big_D

      Re: Is anyone surprised ? Really ?

      And AAE isn't an official dialect that you can select in most spellcheckers. It would fall under American English and therefore fall under those rules. Unless Twitter lets you define your input language as African American English (and that is a flag that is passed to the AI), it can only look at the sentence, work out it is English and work out whether it is profane.

      1. This post has been deleted by its author

        1. brotherelf
          Facepalm

          Re: Is anyone surprised ? Really ?

          Which is why there'll be a flag for "culturally appropriating", most likely. And AIs to train for that, probably. Because surely the reason we've not solved the social problem of people being rude assholes by technical means is that we've just not tried hard enough.

          1. Cederic Silver badge

            Re: Is anyone surprised ? Really ?

            But I'm British; my culture is to culturally appropriate other cultures.

            1. TheVogon

              Re: Is anyone surprised ? Really ?

              Erm, but last time I checked much of the civilised world spoke English?

      2. aks

        Re: Is anyone surprised ? Really ?

        Young Londoners are adopting many words and phrases that are almost a new crole whatever their ethnic origin. IMO there's no justificaton for filtering/censoring any robust language using AI or humans but I've no interest in Twitter.

    3. DavCrav

      Re: Is anyone surprised ? Really ?

      "No amount of clever keyword wrangling is ever going to match true intelligence - which involves KNOWING WHAT THE WORDS MEAN. Not what your database of lexical connections thinks "meaning" is. But ACTUAL MEANING."

      I know this isn't the place for actual philosophical debate, but we have yet to decide what the quoted passage itself means. Fundamentally, we will not be able to easily train algorithms to divine meaning from sentences because we ourselves cannot do that.

      The words trigger concepts in our brain, and putting words near each other triggers concepts sequentially, in the hope that those linked concepts elicit some sort of understanding in the reader. Unfortunately, the reader does not possess this knowledge beforehand, and so does not have the shared experiences necessary to form this understanding quickly. Thus we must choose our words carefully in order to force the reader to make the leap of knowledge necessary to pass on this information.

      This is why writing is hard. But Wittgenstein and others had trouble even with the notion that any thought can be shared, as one would need to have all of the previous experiences of the writer in order to truly understand that which they had written. In practice, this does make a difference, and impinges on what is the topic of this article.

      If you are a member of a particular social grouping then you have previous shared experiences and knowledge that mean that the sentence 'Whassup nigger?' means something very different to someone from my social grouping. Offense is just one facet of this. Take the following example sentence, that I said earlier today:

      'I think the diagonal sits on top.'

      Each of those words has a meaning in the dictionary, and if you read it, you will have no idea what the meaning of the sentence is. If you add extra knowledge 'The two people are joiners' then one has a better guess as to what the sentence means, but you need to know exactly what is being built to know whether I am right in thinking that.

      However, I am not a joiner. I am a pure mathematician. Now the meaning of diagonal changes completely, but probably to something in geometry. But I am not a geometer, I am a finite group theorist. Now the meaning of diagonal is more clear: it is short for 'diagonal automorphism', and I am thinking that a diagonal automorphism is acting on some finite group of Lie type. But unless you know which finite group I am considering, you cannot answer that question either.

      In short, although context is key to the meaning of a sentence, context is not just subjective, but purely invested in the author of the sentence. Any attempt to transfer information through words is likely doomed to failure because not everyone has the same shared experiences and attitudes. Even simple factual sentences can be misinterpreted, for example as the author being sarcastic.

      1. MiguelC Silver badge

        Re: Is anyone surprised ? Really ?

        If the same sentence can be construed as offensive or not depending on the person saying it (see the other example in the article, "Wussup n*gga!" [sic, not self-censorship - because you can not know if I'm being offensive or not]), the problem is not within the AI checking it, but on societal concepts. I dare you to find a workable solution for that!

      2. Mark192

        Re: Is anyone surprised ? Really ?

        "Any attempt to transfer information through words is likely doomed to failure"

        Seems to be working pretty well for most of us.

        1. DavCrav

          Re: Is anyone surprised ? Really ?

          "Seems to be working pretty well for most of us."

          Indeed. A big part of 20th century philosophy worried about why this is so.

        2. Anonymous Coward
          Anonymous Coward

          Re: Is anyone surprised ? Really ?

          "Seems to be working pretty well for most of us."

          Really?

          I've been trying to lecture people about things and about 60% of what I say, is understood by the majority of the listeners. I'm not a good lecturer (I know) but transferring information by words, especially spoken words, has error rate about 50%.

          I won't call that "pretty well". Something trivial, like "It's sunny today" works when the receiver of that can see it themselves. Anything more complicated and it gets difficult.

      3. Il'Geller

        Re: Is anyone surprised ? Really ?

        >>>Any attempt to transfer information through words is likely doomed to failure<<<

        Please forget about Wittgenstein? He was an Amateur, not a professional. I'm the one, though.

        So, shall we dance the St. Vitus dance?

        In contrast to Wittgenstein I consider language as a differential function tending to its limit; where a paragraph is an integral, and a separate (not included in a phrase, sentence or paragraph) word as the limit. So, since separate words are the limit, they have a completely different nature from the function, that is they mean everything and nothing simultaneous.

        >>>Even simple factual sentences can be misinterpreted, for example as the author being sarcastic.<<<

        For example the word "Work". This word acquires an unambiguous meaning only when it is included (surrounded) into many phrases, sentences and paragraphs. Only then the definition from the dictionary can be applied with certainty, to this word.

        The sarcasm can be easily be seen from the surrounding paragraphs, dear sir.

    4. CrazyOldCatMan Silver badge

      Re: Is anyone surprised ? Really ?

      KNOWING WHAT THE WORDS MEAN

      Not only that but the intent that the speaker puts behind the words. Something that only the speaker knows and no rules engine is ever going to guess until we have proper AI and direct machine/brain interfaces

  4. Tom 7

    Father Chrismas is fucked!

    Ho! Ho! Ho!

    1. Ken Shabby Bronze badge
      Big Brother

      Re: Father Chrismas is fucked!

      The Jolly Green Giant too

    2. W4YBO

      Re: Father Chrismas is fucked!

      “ Ho! Ho! Ho!”.

      Who you calling a Ho?

  5. Blockchain commentard

    AI is (normally) coded by white blokes. No suprise then that AI thinks that way - that's how it's been taught. Diversity is desperately needed in the AI programmers community. And good luck with that.

    1. big_D

      This has nothing to do with the race of the programmers.

      This has to do with language detection, English is an official language, AAE is a dialect that the user posting the message can't declare in Twitter settings, so the AI can only treat it as English, or let all racial slurs slide, because it could be AAE...

      1. An ominous cow herd
        Black Helicopters

        And the AI doesn't know if the user is black or not,

        (OR DOES IT?)

        1. katrinab Silver badge

          The people supplying the training data are racist, which is basically the same as programming for a so-called "AI" system.

          1. Claptrap314 Silver badge

            Nice bit of omniscience you've got going there. Can you bottle it & sell it? You would make a mint, I wager.

            1. quxinot
              Joke

              She knows.

              ( :P )

              1. CrazyOldCatMan Silver badge

                She knows.

                He knows, you know.

                But he's got problems..

      2. John Brown (no body) Silver badge

        "AAE is a dialect"

        Is it even that? Or at least is it even singular? Street slang varies from area to area, city to city, region to region.

        1. big_D

          Dialects always have. Maybe AAE is a general over-classification for local ethnic dialects.

          Manc, Cockney, Brum, Scouse, Hessisch, Platt, Schwäbisch Bayerisch etc. there are thousands of dialects around the world that have the same problems, they aren't official languages, they don't have spelling checkers etc. in software and, whilst recognised, they aren't official languages.

          1. jake Silver badge

            "Maybe AAE is a general over-classification for local ethnic dialects."

            Yes.

      3. Anonymous Coward
        Anonymous Coward

        "This has nothing to do with the race of the programmers.

        .... so the AI can only treat it as English,"

        It has *everything* to do with programmers: AI *does not know* anything the programmers didn't tell it.

        What AI can do and what it can't do, is defined by the programmers. And only them.

        "so the AI can only treat it as English, or let all racial slurs slide, because it could be AAE"

        And programmers decided it won't do that. Definitely 100% based on *their own race*. No excuses allowed on this.

    2. ttlanhil

      It's not the programmers; at least not mostly.

      A lot of the time, the people who build the models may not even know how it works (both in the sense of not knowing the theory behind, say, neural nets; and in the sense of not knowing which words the algorithm decided mattered - a lot of "AI" model training is "given this input, you should give this output - you figure out how")

      The problem is with the training data, and any "AI" needs a lot of it.

      Good training data will be entirely self-consistent, properly labelled, and cover all of the variations you want to match.

      Good luck ever getting that for any non-trivial task.

  6. Il'Geller

    Models are AI with artificially limited vocabulary, that is with very strict restrictions on the use of synonyms. Now imagine that you gagged and allowed only on command to utter only clearly stipulated words? That is what they do with AI, limiting it to models.

    1. big_D

      The problem is, those words in AAE that are acceptable are derogatory or racial slurs in standard English.

      The text being provided to the AI has no way of declaring itself as being AAE instead of English, so the AI can only apply English rules.

      And even if the AI says that "hey nigga" could, in all probability, be AAE, how does it know, from 9 ASCII characters, whether an African American posted it or not?

      Dialects are too diverse and specialized to be able to differentiate in a few characters. You need the identity of the person saying it - and given Twitter's ability to disguise your real identity, it is a Sisyphus Arbeit (German, never-ending job).

      1. DavCrav

        "Sisyphus Arbeit (German, never-ending job)."

        Sisyphean task in English.

        1. big_D

          Thanks for that. I knew there should be a direct translation, but I couldn't think of it as I was posting.

          1. Il'Geller

            I do and don't agree.

            I don't because if a normal dictionary is used and a personal AI is created according to the strict rules (taking into account the cause-and-effect relationships), then sooner or later the personal AI will have (at least some) personal specifics. Of course, it will never reliably convey the essence of its owner, but gradually it will become better and better. This is a process, life and not Sisyphean task... In some sense.

            I agree because the models cannot be trusted, they are not AIs! We never know ",,,whether an African American posted it or not?"

      2. Nick Kew

        ... how does it know, from 9 ASCII characters, whether an African American posted it or not?

        How does a human know that? Why does it matter?

        If you say that to a stranger with no context, you're obviously out of order. If you say it to a close friend, it might be perfectly OK depending on the relationship - but noone outside your close circle could judge that, nor is it any of their business. If you say it in the course of a debate, that's fine as far as I'm concerned - and El Reg haven't censored you - but would get you deleted and possibly even banned in some fora.

        Don't all of us say things within the family that would be highly offensive if addressed to a stranger?

        1. Anonymous Coward
          Anonymous Coward

          "Don't all of us say things within the family that would be highly offensive if addressed to a stranger?"

          Family members use insults as terms of endearment.

          Good luck on any AI sorting that out, therefore labeling everyone of them (and me) as 100% pure criminals and scum. Which leads to fact that even trying is just "we make money on this" BS.

      3. Anonymous Coward
        Anonymous Coward

        "The text being provided to the AI has no way of declaring itself as being AAE instead of English, so the AI can only apply English rules."

        And who decided it has to do so? Correct: People who programmed it.

        No excuses: Labeling people racists because they use different writing style than you do, is itself racism. AI part on this is totally irrelevant: AI doesn't do anything it hasn't been told to do.

    2. This post has been deleted by its author

  7. Kevin Johnston

    offensive language is subjective?

    No shit Sherlock

    Oops

    1. Chris Leeson

      Re: offensive language is subjective?

      We avoid the issue by referring to The Constipated Detective...

  8. Tigra 07
    Headmaster

    Racist AI or offensive and crude slang? You decide...

    Toxic or not i'd never hire anyone who spoke like that in an interview because 1) It's crass and offensive (the first example in the image especially). 2) It's extremely unprofessional. Skin colour aside, if you can't formulate a sentence without slang or using the words: "Nigga", or "Nigger", then good luck getting a job in any industry or field. Even brilliant qualifications can't make up for that.

    1. scrubber

      Re: Racist AI or offensive and crude slang? You decide...

      Are you the hiring manager for Perspective API?

    2. iron
      Mushroom

      Re: Racist AI or offensive and crude slang? You decide...

      No one asked you to employ any of these people and the comments are not posted on LinkedIn or similar but on Twitter. So the users involved feel they are talking to their mates, even though they are stupidly broadcasting to the whole world. My circle of friends can be just as crude and profane amongst ourselves yet we have jobs in fields including IT, medicine, journalism, politics and teaching because we can adjust our dialogue to suit the audience.

      1. Anonymous Coward
        Anonymous Coward

        Re: Racist AI or offensive and crude slang? You decide...

        "So the users involved feel they are talking to their mates, even though they are stupidly broadcasting to the whole world."

        No, that's the *whole idea* of Twitter: It's a broadcasting media.

    3. Wellyboot Silver badge

      Re: Racist AI or offensive and crude slang? You decide...

      It's still just context, if you're interviewing for a gangster rap writers position that language moves from offensive to appropriate even if they wouldn't use it near their grandmother.

      1. Jimmy2Cows Silver badge

        Re: Racist AI or offensive and crude slang? You decide...

        Yes indeed. Too many snowflakes around these days. People should try growing a thicker skin. Offense is taken, not given. I can try being as offensive to you as I like, but if you don't find it offensive there is no offense.

        Casual profanity and slurs (both personal and racial) are often commonplace within many communities, considered normal and non-offensive, only becoming problematic when someone from outside the community tries the same, or such terms are used outside the community. In all cases, the recipient chooses to take offense, and their choice is based entirely on context.

        1. holmegm

          Re: Racist AI or offensive and crude slang? You decide...

          "Offense is taken, not given"

          The problem with that is you could apply it to any words at all. *All* words are just arbitrary sounds (or strings) with shared meaning. And the shared meaning of offensive words is that they are offensive.

          We *do* run into problems where words mean different things in different groups and contexts, but it's not an insoluble problem; courts for example solve it all the time.

          1. Anonymous Coward
            Anonymous Coward

            Re: Racist AI or offensive and crude slang? You decide...

            "And the shared meaning of offensive words is that they are offensive."

            Bzzt, false. "Shared" by who, exactly?

            I definitely don't share the meaning of most so called "offensive" words by you, for example.

            Offence is always taken and now you try to weasel around it.

          2. Anonymous Coward
            Anonymous Coward

            Re: Racist AI or offensive and crude slang? You decide...

            ", but it's not an insoluble problem; courts for example solve it all the time."

            By totally arbitary rules: He who has most money, is correct.

            That's not a solution at all as actual meaning of what was said, is totally irrelevant and court assigns *their own meanings* to words used.

            Knowing their background it's neither the meaning of what was said or the meaning receiver claims it was. Thoroughly random process which punishes the poor, typically.

        2. aks

          Re: Racist AI or offensive and crude slang? You decide...

          I've always believed that the best Jewish jokes are told by Jewish comedians. Likewise Irish.

    4. katrinab Silver badge
      Flame

      Re: Racist AI or offensive and crude slang? You decide...

      It is Twitter, not Linked In. They aren't applying for a job at your racist company.

    5. Anonymous Coward
      Anonymous Coward

      Re: Racist AI or offensive and crude slang? You decide...

      "1) It's crass and offensive (the first example in the image especially). 2) It's extremely unprofessional."

      "Offensive" is 100% subjective and totally random, so we have a world of professionals of being offended and I'll classify it as narcissism: Power to say what other people may say. No more, no less.

      Unprofessional is true and as corporate language it's not acceptable at all. Not because of 1) but because of 2).

  9. pavel.petrman

    Obligatory SMBC

    This is what I would use to test whether the models are well trained or not!

  10. DontFeedTheTrolls
    Headmaster

    Any urban dialect is likely to suffer at the hands of AI misinterpretation.

    Nobody could deny the C word is going to be flagged as offensive however in Glasgow it is often used as a term of endearment. "he's a guid c***" is quite common when two people are taking about a third they consider to be an all right person.

  11. DavCrav

    "The tool mistakenly classified 46 per cent of non-offensive tweets crafted in the style of African American English (AAE) as inflammatory, compared to just nine per cent of tweets written in standard American English."

    But the point is that 'I saw his ass yesterday' is offensive. Here's a way to check: is that how you would respond if asked if a co-worker was off sick? What this article refers to as AAE is not a language but just profanity-laden slang, and so if directed at people outside of that clique, is likely to cause offence. Certainly I would say that if someone replied to me like that I would find it offensive, or at least very strange. You know, not coming from the 'hood.

    This is the difference between offensive and offends. There's a scene in Shaun of the Dead, where Ed walks up to the table and asks 'Would any of you cunts like a drink?" His housemate is not offended by that, but the housemate's ex-girlfriend is. Is that sentence offensive? Of course it is, objectively. But under the article's rules, in the language Shaun-house English that sentence is inoffensive.

    1. Jimmy2Cows Silver badge

      The housemate's ex chooses to take offense. The sentence itself simply exists, and is - given the context - clearly not meant to be offensive. Context. Always context.

      If someone said to me "I saw his ass yesterday" I'd probably ask them if they enjoyed the experience, then enquire why the subject might have brought in a donkey.

      Perhaps I just don't look to find offense in everything, especially where it's clear none is intended.

    2. Anonymous Coward
      Anonymous Coward

      "But the point is that 'I saw his ass yesterday' is offensive."

      Nope. *You choose* to be offended by it. Why do you choose to do so? Power to tell other people to "not to use offensive language"? I.e. perceived moral superiority, i.e. racism?

      There's literally *nothing offensive in it at all* when I'm talking about my neighbour's donkey I saw.

      Or we were having a tennis match and had shower afterwards.

      Everything "offensive" happens in your head and therefore is irrelevant to anyone else: You choose to "be offended". I've no idea why but I'll classify it as mental illness unless it's really meant as personal insult. While that one definitely isn't one.

      "Is that sentence offensive? Of course it is, objectively."

      No such thing as "objectively offensive". That's either stupidity or blatant racism.

  12. codejunky Silver badge

    Ha

    Once the SJW's, snowflakes and beigeists have finished with us we probably wont even be allowed to speak any more anyway.

    1. John 110

      Re: Ha

      You're an idiot. Language can be colourful and expressive without having to be offensive, although it does require a bit more thought and imagination...

      1. Jimmy2Cows Silver badge

        Re: Ha

        Language is only offensive when the recipient chooses to take offense. I'd suggest being a bit less sensitive about things, you'll probably be happier for it, but you'd probably call me an idiot.

      2. codejunky Silver badge

        Re: Ha

        @John 110

        "Language can be colourful and expressive without having to be offensive"

        Offence is not given but taken. Therefore you cannot use language without offending someone. In fact gestures too. Just as you can use an 'offensive word' in friendly conversation you can also use polite words to be offended by.

        1. Sherrie Ludwig
          Headmaster

          Re: Ha

          Offence is not given but taken.

          Offence certainly CAN be given, or at least attempted to be given. in the case of insulting or derogatory speech. The person hearing it can decide to take it at face value, and be offended, or to disregard it or to ridicule it, but the intent to give offence does exist.

      3. Anonymous Coward
        Anonymous Coward

        Re: Ha

        "You're an idiot. Language can be colourful and expressive without having to be offensive"

        No, you are an idiot. What you just said *is* offensive. And anyone can and *will* be offended about anything.

        That's the rule of the world. I'm *so offended* when you call other people idiot, you know?

      4. ThadiasVonBasterd

        Re: Ha

        you open with an insult when telling people not to be offensive? that's an impressive lack of thought and imagination you have there.

  13. bjr

    There can only be one standard

    You can't have different standards depending on the race of the poster for the simple reason that you can't possible know the poster's actual race, if David Duke writes a post using a Black vernacular is it OK if he uses the N word because the post sounds like it could have been written by a Black person? The answer is that a word is either OK for everyone to use or for nobody to use but on the Internet you can't discriminate based on who you think the speaker is. In real life you can make those distinctions when you are face to face because you actually know who's speaking but you can't possibly do that online. The best you can do is establish community standards for individual forums and groups, but once established those standards have to be applied to everyone.

    It's up to people to adapt to the limitations of there tools not the other way around. There is no such thing as artificial intelligence, that's a marketing term, what computers do is pattern recognition and there are limits to what that can handle. If certain words are banned in a particular forum then you just can't use those words even if you use them in everyday life. You can't sail your car, unless you own an Amphicar, if you drive your car into a lake it will sink. We accept that cars can't sail on water and we have to accept that computer moderation will never be more than a list of banned words, you just have to adapt to that.

    1. Wellyboot Silver badge

      Re: There can only be one standard

      Luckily us meatbags can adapt very quickly so the (non)AI will just have to wrangle that up the grumble pipe.

    2. FrogsAndChips Silver badge

      Re: There can only be one standard

      So we shouldn't use any expletives in El Reg's forums because they might offend one of the readers? What the fuck?

      1. Jimmy2Cows Silver badge

        Re: There can only be one standard

        I read that as simply pointing out the limitations and recognising if there is a a rule, it should be applied consistently, rather than trying to be "intelligent" which creates a double standard.

        I say write whatever the fuck you want. If anyone's offended by that, that's their problem. I simply used a colourful metaphor to enhance meaning.

        1. quxinot

          Re: There can only be one standard

          Fuck 'em. Anyone dumb enough to be offended by language, and particularly those who will take offense on someone else's behalf, needs to be wrapped in barbed wire and beaten until they understand 'harm'.

          The music is always more important than the words. Getting worked up over the words is just nitpicking for people who aren't bright enough to understand that.

          1. CrazyOldCatMan Silver badge
            Joke

            Re: There can only be one standard

            The music is always more important than the words

            You obviously listen to prog music..

    3. Tuomas Hosia

      Re: There can only be one standard

      "You can't have different standards depending on the race of the poster for the simple reason that you can't possible know the poster's actual race"

      And even if you knew, that would be literal racism: Different and arbitary rules based on skin colour.

  14. Anonymous Coward
    Anonymous Coward

    Context is all

    We were discussing this very thing in the pub recently. When I was at school in the 70s, one of my classmates was called Richard Black. My surname is Green.

    We were both regularly called "Greenie" and "Blackie" amongst our social group. Richard also played rugby for the school team, and you could often hear cries of "come on Blackie" from team members or spectators during a match.

    So what you might ask?

    Well, Richard was of Jamaican origin.

    At the time, no-one gave it a thought, it was just his nickname, and there was no thought whatsoever of it being a racial slur, it was just an affectionate diminutive of his name, just as Greenie was mine. Nowadays though, it would probably raise gasps of horror from people who didn't know his name.

    1. jake Silver badge

      Re: Context is all

      I know a guy who is sixth generation Haitian. His surname is Blackmon. He is white; his multiple-greats grandfather was from Scotland. The name has generated much mirth in the community for well over a hundred years. He says it's better to laugh because of our differences than at them ...

    2. Wellyboot Silver badge
      Coat

      Re: Context is all

      My school chum called Peter Green was referred to as " P "

      Some people never made the vegetable connection.

  15. Miss Config
    Go

    Airplane! LIVES

    Run this AI software on the dudes talking jive on Airplane!

    I'm ordering popcorn from the edge of my seat for the results.

    ( Airplane! ? What's that ? It's a metal object in which people fly from city to city but that's not important right now. )

  16. Dedobot

    Lets sit to drink and eat "like white man", lets do it "like a white man" and so on. Bulgarian 100 yrs old slang. Wonder how the AI will handle it :-)

  17. a_yank_lurker

    AI means

    Artificial Idiocy means racism by another means. AAE or other variations are not standard English (define that) but they are valid dialects. So if you write using a dialect AI will flag you as a ... aka idiot, moron, etc. Because it is done by AI it has to be objective (really?) </snark>

    1. Anonymous Coward
      Anonymous Coward

      Re: AI means

      "Artificial Idiocy means racism by another means."

      AI doesn't do or know anything not programmed in to it. Therefore it is spitting image of the programmers.

      Who obviously are white IVY-league males, judging by the results.

  18. LeahroyNake

    I saw his ass yesterday

    I find it offensive that I haven't seen any ass for ages, or should that be arse? I'm not bothered and can't be arsed to ask my ISP to unblock a certain hamster site.

    I do wonder what kind of rating the AI would give that last paragraph.

  19. Claptrap314 Silver badge

    The elephant in the room

    There are several issues at play here.

    1) The fundamental question of the nature of offense. I hold that it can be intended or taken up. Also, that there are legitimate differences between how people, no matter how well-intended, both in themselves and in their perceptions, will take any communication.

    2) The technical impossibility of divining intent of a set of words, which is more difficult the shorter the communication.

    3) The politicization of offense.

    The term "hate speech" itself borders on being aggressive, as those employing it overwhelmingly target (US) conservatives while ignoring actual calls for violence and murder against us.

    I am also permanently not okay with the term "cunt". EVERYONE knows it stems from a vulgar reference to women, and if you think it is okay to speak that way, I have a problem with your attitude towards women. If a culture takes it up, I have a problem with that culture. Calling it "urban" does not excuse such. It just means that "urban" is the new "low class".

    1. Mark192

      Re: The elephant in the room

      "I am also permanently not okay with the term "cunt". EVERYONE knows it stems from a vulgar reference to women, and if you think it is okay to speak that way, I have a problem with your attitude towards women."

      Do you think similar about their attitude to men if they call someone a dick? (genuine question)

      1. Sherrie Ludwig

        Re: The elephant in the room

        "I am also permanently not okay with the term "cunt". EVERYONE knows it stems from a vulgar reference to women, and if you think it is okay to speak that way, I have a problem with your attitude towards women."

        Do you think similar about their attitude to men if they call someone a dick? (genuine question

        There seems to be a difference between British and American usage on that. The term in British English seems to be no worse that calling someone a "jerk" (another term that may have originally developed from a vulgar origin) and seems to be unisex. In American English, it is a very vile insult used only against women, and specifically demeaning their sex.

        If I wished to specifically and vulgarly insult a male person, I would probably call him a prick, because it seems like a dick, only smaller. But I would use neither in a work environment, and certainly not in a job interview.

        1. Claptrap314 Silver badge

          Re: The elephant in the room

          It might be a generational thing, but calling a man a "dick" to me has a quite precise meaning. It means that one is proceeding heedless of consequences against attempts by others to dissuade you. In other words, "Stop being a dick" is easily the majority of use cases.

          There is actually a standup routine where the guy was talking about how one night he was tired, and wanted to cuddle. "Sex". "No, I just want to cuddle"... ending with "Yes, my penis is in fact a dick."

          Having said that, the term as a more-or-less general reference to men goes back to the forties at least, and I do not know how or where it started. And yes, as a general term, I am not a fan.

          As for "prick", that might not be as precise for me as "dick", but it definitely implies that the individual is compensating for some self-perceived failing as a man.

          There are genuine asymmetries in life. For instance, "white trash" is about as close as you can get to "nigger", but it does not even begin to carry the context. Similarly, "cunt" is much more violent than "dick" or "prick". Historically, you bring the heat to a man by insulting his manhood, either as opposed to being a child or a woman. Calling a man a "momma's boy" was straight-up demanding a fist to the face.

    2. Anonymous Coward
      Anonymous Coward

      Re: The elephant in the room

      "EVERYONE knows it stems from a vulgar reference to women"

      Sorry, but that's BS. See Irish or Australian terms of endearment, where it has totally different meaning and "stemming" from something is totally irrelevant: That has already been lost in history.

      " you think it is okay to speak that way, I have a problem with your attitude towards women"

      That's your personal mental issue, most of Aussies use that all the time and they don't have such problems at all: It's all in your head. You see, context *is* essential.

      "If a culture takes it up, I have a problem with that culture."

      If? You are a bit late. And will have problems in Ireland or Australia with that attitude.

      1. Claptrap314 Silver badge

        Re: The elephant in the room

        Oh, I had problems with Aussie culture long before I heard about "cunt".

        And if you are so masochistic as to follow my posts, you know that as an American, I've got problems with American culture. They happen to run contrary to current trends.

        The fact that a few messages up the stack, a guy related that he had to explain to his girlfriend that "cunt" "is not offensive" means that in fact that 1) it is, and 2) your claim that it's origin is lost to history is farcical.

    3. Anonymous Coward
      Anonymous Coward

      Re: The elephant in the room

      "The term "hate speech" itself borders on being aggressive,"

      It's a tool for legal censorship of anything government does not like. Literally. It has no other uses as personal insults have been illegal very long time.

      Here in North it's actively used to fine opponents of immigration (legal or illegal) in court. Just because they dared to publish officially public documents no-one else dared to publish.

      That's how censorship works in modern societies.

  20. Reality_Cheque

    Walking on eggshells

    The problem is that what is considered relaxed banter amongst African-Americans, is considered racist when used by white people. The flaw isn't in the algorithm, it is in society. For example, there are a dozen informal but inoffensive ways to refer to a British person (Brit, Tommy, limey, Pom, etc), and any white or black person is welcome to use these. Give one example of an informal but INOFFENSIVE way to refer to an African, Indian or Pakistani. There are a few when used by a black person, but a white speaker is reduced to using the strictly formal, or grossly offensive. Fixing that differentiation is the issue, not trying to filter according to the race of the speaker.

    1. Zog_but_not_the_first
      Boffin

      Re: Walking on eggshells

      Exactly. I recently enjoyed watching "Blazing Saddles" with its fine critique of attitudes. But if I recall the scene where the new sheriff gets out of a tricky situation with "Don't shoot or the nigger gets it" I'm accused of being a racist..

      Grow up.

      1. Wellyboot Silver badge

        Re: Walking on eggshells

        They obviously haven't seen the Mel Brooks masterpiece!

        https://www.youtube.com/watch?v=6pADDn0qm3M

    2. Anonymous Coward
      Anonymous Coward

      Re: Walking on eggshells

      There's that lovely catch 22 that in order for the AI to know whether the speech is offensive or not it needs to know the racial orientation of the speaker. But if it knows the racial orientation of the speaker and adjusts accordingly that in itself is intrinsically racist.

    3. Anonymous Coward
      Anonymous Coward

      Re: Walking on eggshells

      "The problem is that what is considered relaxed banter amongst African-Americans, is considered racist when used by white people"

      The problem here is that the definition of "racism" is totally fucked. And no-one seems to want to fix it, but instead define it even more fucked what it was.

      True anti-racism is that anyone can use same words and it's either racism or not, but current situation is that what is "racism" totally depends, and I mean *totally* depends on who's using "the bad word".

      That's totally fucked definition and needs to be changed to something else. I've no idea what it would be but this totally stupid clinging in single words definitely *isn't going to solve anything*, when actual racism happens on *intent level*.

      Seeing the intent by looking the words alone is definitely outside of the capabilities of an AI. It's obviously way too hard for most humans also, so they see trigger word, brains shut down and yelling "that's racist!" starts.

      Thoroughly stupid.

  21. Toni the terrible

    Nothing New

    Afterall recall the time that everything with the word 'breast' was deemed profanity which sort of upset various Cancer Charitys and Art institutions etc.

    Context is all.

    I once got a weeks ban on the chat area of a wargaiming site because I referenced the Mongol / Mongolian / golden Horde as ruled by Gengis etc Khan - and not refing anyone not historical - made me wonder if the Moderator was a Bot. I was told later by other users that it was probably the word Mong@@ (as they typed it) possibly, but the context on a wargaming site?

    1. Anonymous Coward
      Anonymous Coward

      Re: Nothing New

      "recall the time that everything with the word 'breast' was deemed profanity"

      AFAIK it *still* is. Like penis or vagina or anything, even in anatomical context in more religious states in US:

      1. jake Silver badge

        Re: Nothing New

        No, it isn't, at least not for the mainstream US population. There are still a few backward-ass holdouts in various places where religion is more important than humanity, though.

  22. jake Silver badge

    AAE doesn't actually exist as "a language".

    There are, however, many creoles/pidgins that are often incorrectly lumped under that banner.

    Consider the versions that are spoken in New York City, Chicago, Mississippi, Detroit, the Los Angeles Basin, and the East Bay. They are quite different linguistically. To say nothing of the umpteen different versions spoken in the South ... And worse, they are constantly changing, making them a moving target.

    As a result, trying to program a computer to recognize "valid" AAE is a fool's errand. Trying to get it to filter out obscene, hateful, rude, insulting, or otherwise so-called "bad" speech is a complete waste of time good use of research funds. Where do I apply for my cut?

    1. Anonymous Coward
      Anonymous Coward

      Re: AAE doesn't actually exist as "a language".

      It doesn't exist as a language because the powers that be have decided it shouldn't be accorded that status. But it is at least as distinct from American English as British English is, so I don't see why it shouldn't be.

      They'd rather just consider speakers of AAE as being "illiterate" and unable to speak proper American English, when in fact many are able to switch between American English and AAE as smoothly as some bilinguals can switch between English and French on the fly.

    2. Anonymous Coward
      Anonymous Coward

      AAE doesn't actually exist as "a language".

      I have to admit, in my 50+ years of speaking American English and living in the US I have never even heard of "AAE". I had always heard it referred to as "Ebonics".

      The two "AAE" messages that were rated toxic *ARE* toxic and offensive to many, and in a world where you can be fired if you say that you prefer to use the historical pronouns with which you grew up instead of neutering your language, why should those sentences get a pass?

      1. Anonymous Coward
        Anonymous Coward

        Re: AAE doesn't actually exist as "a language".

        "The two "AAE" messages that were rated toxic *ARE* toxic and offensive to many, "

        Seeing an ass is "offensive to many"? Why? Over-sensitive prudes or is human anatomy so offensive just by existing that they *have to take offence* any time it's mentioned, even if not to them?

        Offence is taken, not given.

        Donkeys so bad? As there's *0* context, you don't and can't know if it was referrering to donkey.

        6% toxic is totally absurd in that context. I call BS: 0% toxic and even less offensive.

        Also "offensive to many" ... but not you *personally*? Again these people who take offence *for other people* so they don't have to bother, totally not caring at all if anyone else is actually taking offence. Why people like this exist in the first place?

        If someone wants to take offence, they can comment themselves. Which part of that is too complex for you?

  23. Henry Wertz 1 Gold badge

    right or wrong?

    Is the AI right or wrong? There are many black people in the us who will say "what up nigga?" or whatever to each other and apparently think nothing of it. (I guess? Locally there are many immigrants from both city of Chicago, and the Sudan, and i've seen nobody use this kind of language.) But there are plenty of others who think it is deeply offensive and that those who say this kind of thing should cut it out immediately. And honestly, if you have phrasing that is ok to say if you're black, but offensive if you're white, clearly that says something about those phrases.

    1. Anonymous Coward
      Anonymous Coward

      Re: right or wrong?

      "And honestly, if you have phrasing that is ok to say if you're black, but offensive if you're white, clearly that says something about those phrases."

      It says taking it as offensive means you are an idiot. Or racist. As general rule.

      Those can be used as offence, but unless they are meant to you personally, you have no idea and claiming "being offended" because you hear them somewhere means you are an idiot.

    2. genghis_uk

      Re: right or wrong?

      A couple of Gs, an R and an E, an I and an N

      Just six little letters all jumbled together

      Have caused damage that we may never mend

      As Tim Minchin said in 'Prejudice' - Only a Ginger can call another Ginger Ginger...

  24. Anonymous Coward
    Anonymous Coward

    There is also the situation where what is considered acceptable language between members of one group is not if used by another group. I am not sure how any filtering program can make "rational" decisions unless one is willing to fill out a form for each post stating detailed demographics of writer, no less than 100 word essay on the context of the text, another on the intent, another on the person(s) that the tweet is directed towards and citations from the unabridged dictionaries of both English and colloquialisms.

    In any case, only old politicians use twitter anymore, and they can say anything apparently, or the press would have nothing to report on.

    1. Il'Geller

      ...I am not sure how...

      AI technology solves this problem by annotating words with dictionary definitions.

      1. Anonymous Coward
        Anonymous Coward

        "AI technology solves this problem by annotating words with dictionary definitions."

        .... which are written by white IVY-league young men. And it shows.

        1. Il'Geller

          You are not pleased either way! There is almost no bias.

          1. Il'Geller

            Well-established, proven dictionaries and encyclopedias

            You see, companies like OpenAI prefer to annotate using random texts mined from nowhere, that is practically creating dictionaries and encyclopedias (for annotating) from scratch. For this they use astronomical volumes of raw texts, gigabytes and gigabytes. I tried this method during my preparation for NIST TREC QA and came to the conclusion that standard dictionaries and encyclopedias are more suitable, not least because they are compact. For example, only 25 and nice indexed megabytes for Merriam.

            In particular, general dictionaries and encyclopedias are very good because they contain absolute minimums of bias. Thus, if you want to avoid completely or minimize the manifestations of AI racism, you must inevitably use well-established, proven dictionaries and encyclopedias, and not the random texts.

  25. Snowy Silver badge
    Facepalm

    Bad AI is bad.

    Remember you do not need to use bad words to say bad things.

    1. Il'Geller

      Re: Bad AI is bad.

      Words - not a problem, the words themselves do not carry any semantic load, because they mean everything and nothing at the same time. For example the word

      -- Work

      What does it mean?

      To understand it you need to have its context, that is a set of phrases that surround it. Based on this context you can establish the word's "Work" subtext - what it exctly means, its dictionary definition. And then you can understand how bad was the word is.

      This is AI technology.

      1. CrazyOldCatMan Silver badge

        Re: Bad AI is bad.

        -- Work

        What does it mean?

        Heat is work and work's a curse

        And all the heat in the universe

        It's gonna cool down as it can't increase

        Then there'll be no more work

        And they'll be perfect peace

        Really?

        Yeah, that's entropy, man!

        Flanders and Swann: The first and second law of thermodynamics

        1. Il'Geller

          Re: Bad AI is bad.

          As you see CrazyOldCatMan put the word "Work" into a context and subtexts, that is he annotated it with phrases and dictionary definitions.

  26. Nick Kew

    Who is racist here?

    We can of course point, with or without evidence, at a range of suspects.

    But this story unambiguously highlights one attitude as racist. Namely, the idea that you can objectively identify offensive speech / hate speech. There will always be grey areas, and those go right into areas of supposed black-and-white certainty.

    Symptomatic of today's inability to distinguish objective fact from subjective opinion and prejudice.

  27. Anonymous Coward
    Anonymous Coward

    It’s pretty normal on twitter for non-whites to say things like “kill all white people” and the twitter mods, if you report it, always seem to find that it doesn’t violate their community standards. Maybe that is what the AI is finding.

  28. Dusty

    Liguistic appartheid

    "Racist" comments aside. I have always felt that, in English anyway (Does the same phenomena exist in other languages??) Much of the distinction between acceptable and unacceptable speech dates back to the Norman conquest. "Profane" language is the language of the natives, "Polite" language is the language (Effing Fulfords aside) of the ruling classes. We do not just have to look at the classic four letter words to find this distinction. Consider agriculture. The Beasts have the short names of the native working classes, Hen, Cow, Bull, Sheep and so on. The flesh by contrast has the "Foreign" names used by those ruling classes wealthy enough to actually be able to eat it. Beef, Poultry, Mutton and so on. Perhaps, post Brexit, we may have the opportunity to throw off our linguistic shackles and rediscover our verbal heritage and become potty-mouth capital of the world!

  29. Ken Moorhouse Silver badge

    "I heard you paint houses"

    I wonder what AI would make of that one...

    1. jake Silver badge

      Re: "I heard you paint houses"

      That would depend on whether or not the programmer had ever heard the joke ... AND thought to include it in the AI's so-called "data set". And probably whether or not the programmer found it funny ... subjectivity is a bitch, ain't it?

  30. Stam

    Is it just flagging keywords?

    Surely any AI trying to determine 'offensiveness' would need at least the entire context of the conversation and who it's with.

    If I spoke to my stuffy great aunt the way I talked to my hometown friends she'd be pretty offended.

    Conversely, I can say some pretty awful things without using a single bad word.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like