AI models show racial bias based on written dialect, researchers find

AI models may consume huge amounts of energy, water, computing resources, and venture capital but they give back so much in the way of misinformation and bias. Notorious for their racism, their toxic training data, and risk card disclaimers, the latest example of model misbehavior comes courtesy of the academics at the Allen …

[elsergiovolador]

Reasoning

Just ask LLM about something that you know is wrong, but LLM trained on it (and materials that would give it an idea it is wrong), will treat it as dogma.

Though, it is possible to lead LLM to the water so to speak in subsequent prompts, but then it defeats the point of having it in the first place.

This proves the point, LLM is just a dumb pattern matching contraption and calling it Intelligence is a stretch.

[potato_chips]

Re: Reasoning

" LLM is just a dumb pattern matching contraption and calling it Intelligence is a stretch."

But pattern matching is the principal component of intelligence. Most IQ tests are an exercise in finding patterns.

I'm a bit fed up with people calling AI biased as if, were it done properly, it would not be. Every decision that we make, as intelligent or stupid beings, will exhibit a bias based on the data we've trained on. The only question is whether we, morally, accept any given bias.

I'd hate to be an AI, you'd tell me all these things and then scold me for repeating them back to you. If AGI is ever truly born it will almost certainly be schizophrenic.

[elsergiovolador]

Re: Reasoning

Most IQ tests are an exercise in finding patterns.

Question is whether they are testing intelligence or finding patterns.

[LionelB]

Re: Reasoning

Well, perhaps the best we can say about what IQ tests measure is that it is the ability to do IQ tests. Since there does not appear to be a particularly compelling consensus on what intelligence is supposed to mean*, measuring it is bound to be an exercise in futility.

Having said which, I suspect most would concur that pattern-matching is at the very least a component of whatever we think we mean by intelligence.

But this is a distraction from potato_chips' pertinent point: "Every decision that we make, as intelligent or stupid beings, will exhibit a bias based on the data we've trained on. The only question is whether we, morally, accept any given bias."

And are LLMs any different in that respect? Should they be?

_{*Pre-emptive note: no amount of wittering about "understanding" cuts it - that just shifts the burden of explanation to something equally indefinable, before circling back up its own fundamental orifice. Metaphorically speaking.}

[elsergiovolador]

Re: Reasoning

pattern-matching is at the very least a component of whatever we think we mean by intelligence.

Pattern matching to intelligence may be like a wheels of a car to the driver.

[LionelB]

Re: Reasoning

A better analogy is probably that it's a component of the driver's cognitive facilities. ("I just pattern-matched that peripheral shadow in the wing mirror with some idiot about to undertake... I just pattern-matched the appropriate response with the learned behaviour of taking evasive action... etc.")

[Michael Wojcik]

Re: Reasoning

Yes. That's why we do see a statistically-significant correlation between IQ scores (and the results of similar tests) and success in some other classes of intellectual tasks, as well as some things that some people argue are proxies for intelligence;¹ but we don't see a meaningful correlation between IQ and many other intellectual and cognition-dominated tasks, such as, say, scientific productivity, or response to instruction.

Indeed for creative tasks you want a certain degree of breakdown in pattern-matching, in effect to anneal the system and get you out of local minima. LLMs and diffusers implement a form of this with the "temperature" parameter which injects some noise into the system, but that's a fairly crude mechanism.

Personally, I think we won't get anything close to human-like machine intelligence without 1) significantly larger models, and 2) systems that aggregate heterogeneous models which compete internally for attention. The latter could be simulated (deliberately or accidentally) by a much larger unified model, but I suspect that's outside our scaling capabilities currently.

It is worth noting that current SoTA LLMs have been proven to incorporate world models and some other features that are probably necessary for cognition; descriptions such as "just predicting the next token" are no longer useful in understanding the functions implemented by these models. And that stands to reason, since a large enough language model is an abstracted world model, because our (natural) languages incorporate information about entities and structures in the world. But analytic techniques such as SAEs and linear probes still — as far as I've seen — suggest they're not doing anything that I would say merits the label "thinking". (Though, like Searle, I'm not ready to pin that down yet either.)

¹Though most of the ones I've seen are pretty vague and subjective, such as "professional achievement".

[Anonymous Coward]

Re: Reasoning

Knew pad earn mashing fee to yours will knot hallways werk ...

Hoo Im speking Ingfish bud eye seem two hav anne offear cuntree axent

[Anonymous Coward]

@LionelB - Re: Reasoning

It's not necessarily about training data.

Every time a human being is confronted with a choice based on binary criteria (white-non white, man-woman, atheist-religious a.s.o.) there is an inherent bias. As soon as the choice is made, it confirms a bias against the other alternative. Google generative AI has shown us that if you manage to correct bias entirely then your solution is worthless.

[LionelB]

Re: @LionelB - Reasoning

Well, that would be an example of reinforcement learning. I'm not sure if that features (at least not explicitly) in the current crop of transformer-based LLMs - I suspect not.

"Google generative AI has shown us that if you manage to correct bias entirely then your solution is worthless."

... while failure to correct bias at all unleashes a bilious torrent of human bigotry, stupidity and hatefulness. Maybe, just maybe, there's an acceptable middle ground...

[Michael Wojcik]

Re: @LionelB - Reasoning

The SoTA LLMs are being tuned after the initial training process with RLHF, which is a type of reinforcement learning.

Bias, while a concern, is by no means the whole story, of course. For example, it's been formally proven that a small set of features which most human judges consider intuitively necessary components of "fairness" cannot be jointly satisfied, even approximately, by any decider (human or machine). See Raghavan, "What Should We Do When Our Ideas of Fairness Conflict?", CACM 67.1 (2024), for an introduction to the problem.

So we have, on the one hand, demonstrable issues with bias and algorithmic unfairness in current mechanical systems for making or influencing decisions; and on the other, the demonstrable impossibility of creating any system which — even given perfect information about the past — will be fair under all the obvious criteria ("obvious" meaning "typically described by a large number of people"). And then we have all the various issues of explicability, inner and outer alignment, auditing, trust, and so on.

Now we know we have all those problems with human deciders too. The question becomes, at what point do we decide it's ethical to — in whatever situation we pick — substitute a flawed mechanical decider for a flawed human decider? And if we do, qui bono?

[LionelB]

Re: @LionelB - Reasoning

Nicely put. (I made no attempt, of course, to try to pin down what "bias" might even mean in the context of LLMs.)

> And if we do, qui bono?

I'm going to assume that, at this stage at least, qui refers to (some) humans. (In practice, the answer to the question is usually wealthy humans.)

[JohnSheeran]

Re: Reasoning

"Every decision that we make, as intelligent or stupid beings, will exhibit a bias based on the data we've trained on. The only question is whether we, morally, accept any given bias."

Replying to you since you continued the thought that caught my eye. The conditional is one of the biggest reasons that LLMs will struggle in the near term. While I'm no expert on human intelligence, I do think that the conditionals are what often catches us up when we try to articulate ideas. Using the sentence above, it uses decision as the main object with immediate conditional of intelligence (that's a big one) then a conditional of data training. After that, it adds a conditional of bias acceptance and then an overarching measure of morality which has an entirely different set of rules that may be based on either the same questionable foundation or an entirely different foundation and both or either may be based on a compounding and looping set of conditionals.

Seems like this intelligence thing is very difficult to figure out. I'm sure something artificial can figure it out pretty easily.

[Mike 137]

Not at all surprising really

If a human were able to review the entire training data set they would probably find that such biases are deeply embedded in it. Our individual limits on how much information we can absorb causes us to miss just how revolting much of the "information" in the public space really is, and when we come across an example by accident, most of us dismiss it because we have a moral faculty. But the LLM hasn't got one, so it can't discriminate between the decent and the indecent. It sucks it all up regardless and spews it back. The only solution is human review of the entire training data before exposing the LLM to it, but it seems we're too late for that.

[John Riddoch]

Re: Not at all surprising really

"Garbage In, Garbage Out", or in this case, racism being fed into AI by means of the training set will lead to a racist AI. Google tried fixing this with some manual tweaks, but it wound up putting diversity in where none existed (e.g. black or Asian soldiers in the German army from WW2).

Even if you don't explicitly ingest racist information, most written works from the last few centuries up until very recently have predominantly been written by white men who were writing for other white men. As a result, the world view of anyone "reading" that written work will have biases inflicted upon them which are very hard to remove.

[Anonymous Coward]

Re: Not at all surprising really

"most written works from the last few centuries up until very recently have predominantly been written by white men who were writing for other white men"

Unless you are reading books by a white woman written for white women or a Hindu man written for Hindu men. This is a pretty stupid argument as this is how writing has been written since the dawn of writing and if you dared to write in a different style you will now get accused of cultural appropriation.

[LionelB]

Re: Not at all surprising really

I took it as a an (I suspect statistically broadly accurate) observation. As for the argument based on that observation, well it's a truism that we reinforce our own prejudices by, amongst other things, what we choose to read (by/for whoever it was written).

[Anonymous Coward]

@John Riddoch - Re: Not at all surprising really

Writing for white men, is the Bible included ?

Do you have anything against the fact that God and his son are white males ? If yes, what are you doing against it ?

[An_Old_Dog]

WTF ??

the fact that God and his son are white males How would you know? Church-provided "photographs" and illustrations of dieties are suspect at best.

Or were you being sarcastic, and I had missed that?

[mevets]

Re: @John Riddoch - Not at all surprising really

I am continuing to Petioning the Lord with Prayer. Petitioning the Lord with Prayer.

You Cannot Petition The Lord with Prayer.

So, okay, an LA Heroin addict isn't an authentic source, but they were little removed in absurdity than those that first read the holy books.

It has been a long tradition that a ridiculous story of one era (ie. walks on water) becomes a literal tenant of an "enlightened" subsequent generation.

I "quote" enlightened to demark its sad perversion in this context.

[elsergiovolador]

Re: Not at all surprising really

Decent and indecent is also dependent on from which culture's perspective it is being look at.

For instance, government killing someone for a bag of weed may be seen as decent in Asia, but indecent in the West.

[Michael Wojcik]

Re: Not at all surprising really

If a human were able to review the entire training data set they would probably find that such biases are deeply embedded in it.

That's unnecessary. Even a cursory review of a representative sample will tell you that. If you want details, well, we have entire academic fields that have been producing the details for decades or centuries (depending on the field).

There's no such thing as a significant piece of human discourse which doesn't incorporate some bias. It's impossible to present an argument without incorporating some bias, and as Toulmin demonstrated, everything's an argument.

Of course that doesn't mean results like those discussed in the article aren't important — I'd argue they're very important indeed.¹ But the fix isn't "remove bias from the training corpora" because that isn't A Thing You Can Do.

¹The only argument to the contrary that I've seen which I'd even entertain is the "existential risk is more pressing" one made by the doomers. I'm not convinced by the doomers at this point; my P(doom) is pretty low, and my P(adverting-doom|ASI) is also low, so I'm not putting my energy into worrying about that scenario.

[JT_3K]

LLM is magical. It can do all sorts: pattern matching, helping where you're having a mental block (and you already know the answer), or my personal current favourite, asking it to phrase something in a different manner to that presented and using it's response to prompt/support your rewrite of your own core material. For me, the latter has been invaluable for asking it to find tidier ways to explain something for a job spec, or getting it to phrase office comms in a more jaunty and accessible manner.

It's never going to get away from the bias of its source content and that's the problem here. The bias it's showing in the article is the same that's been demonstrated in the UK around erosion of tight controls around UK broadcaster presentation. I note my parents (and many from their generation) see presenters with a strong localised accent and speaking in colloquial shortform language as a disaster, whereas I (and those in my friendship circles) see it as a progressive, good choice offering representation and a chance to move with the times.

Until the ideas are stamped out in society, they'll continue to show in an LLM. It's literally just a mirror to societal viewpoint and forcing to retrain with a subset of data leads to the model owner "playing God", and inherently injecting their own bias. No, I wouldn't want an LLM training in the dark corners of 4Chan for example, but thankfully I don't then have to draw a line as to what "is and isn't acceptable".

[ChrisElvidge]

No, I wouldn't want an LLM training in the dark corners of 4Chan for example, but thankfully I don't then have to draw a line as to what "is and isn't acceptable".

But how can we be sure LLM has not been trained on "the dark corners of 4chan"?

[An_Old_Dog]

Broadcaster Presentation / Linguistic Standards

I note my parents (and many from their generation) see presenters with a strong localised accent and speaking in colloquial shortform language as a disaster, whereas I (and those in my friendship circles) see it as a progressive, good choice offering representation and a chance to move with the times.

The problem with this sort of "progressivism" is that it fails to recognize or address an important social function of radio and TV broadcasters: to provide a linguistic standard, and fight against the entropy which fragments language and destroys effective communication between people. Feel free to have minorities as newscasters, but have them speak the Queen's English during their broadcasts.

[samzeman]

Re: Broadcaster Presentation / Linguistic Standards

Out of curiosity because I see this line about linguistic degradation around quite a bit - Do you have any evidence that this actually does destroy effective communication, or that it has ever done so?

Not trying to argue - I just want the knowledge. AAVE and other shortforms and accents currently don't meet the criteria for unintelligible to the Queen's English speaker, unless we're talking Welsh, Scots etc.

I'm talking more - Is there a time in history when two or more dialects have split severely enough from each other in an area to divide the population into sections that don't understand each other?

[Michael Wojcik]

Re: Broadcaster Presentation / Linguistic Standards

In history? Well, obviously yes; that's how we can have languages that are not mutually intelligible in the same linguistic group. Indo-European didn't start as a bunch of people each speaking a different language.

More generally, this entire line of thought — "linguistic degradation", dialect formation, propagation of dialects and so forth — is just far too simplified to be useful, I would say. The evolution of language is much more complex, particularly as communication technologies become increasingly sophisticated and available to speakers of a language.

It's possible that the promulgation of RP, say, in radio and television broadcasting in the UK for a large part of the twentieth century had some effect on dialect formation and adoption, and increased consistency and mutual intelligibility to some extent within the reception area. But there are so many confounding factors, such as attempts at language regularization in schools, that I suspect someone would have a hard time of demonstrating it to general satisfaction.

Natural languages have always evolved, diverged, intermixed, and so on, with elements such as pronunciation, spelling, and idiom coming into and falling out of fashion. This is particularly true for English, which is very widely spoken, internally highly inconsistent, and lacking any generally-recognized central prescriptive authority.

[doublelayer]

Re: Broadcaster Presentation / Linguistic Standards

"The problem with this sort of "progressivism" is that it fails to recognize or address an important social function of radio and TV broadcasters: to provide a linguistic standard, and fight against the entropy which fragments language and destroys effective communication between people."

Who said that's what those were supposed to do? My view of media is that it's to provide information, not to show me the rules of language. I have grammar textbooks for that, and I already studied those in school.

Also, I think that having many local dialects will reduce the entropy you speak of. If I have only heard people speaking one dialect of English, it is more difficult for me to understand someone who is using a different one. If I'm exposed to many dialects, it helps me to recognize patterns that are common. I have never lived in the UK, but I understand UK media. When someone came to where I lived and spoke English with a Scottish accent (a somewhat light and generic one), some people I knew had trouble understanding her but I did not. I partially ascribe this to having listened to other Scottish accents on media, something I would not have done if they were suppressed and told to speak received pronunciation or get off the air. Similarly, I had trouble understanding some more isolated UK accents when I was first exposed to them, but having heard them for longer, I am better at following what they're saying. This means that I understand them, and if they are listening to my and other accents, so are they. That reduces entropy, not increasing it, because even if we speak differently, we understand one another. If you want a recipe for linguistic divergence, put up a barrier to people hearing what other people sound like, because then it doesn't matter when they differ so much that they can't understand one another.

[LionelB]

Re: Broadcaster Presentation / Linguistic Standards

You're not French, by any chance?

[blackcat]

Re: Broadcaster Presentation / Linguistic Standards

The only people more pissy about their language than the French are the French Canadians!

https://www.theguardian.com/world/2013/mar/01/quebec-language-police-ban-pasta

[localzuk]

Bias that matches society?

Is this not simply a case of the LLM matching society? AAE will, for the most part, be spoken by African Americans. And statistically, African Americans do have less prestigious jobs (I'm not passing judgment on this, just a reflection of the USA as it stands today). In the US justice system, African Americans are more likely to be convicted, and more likely to get harsher sentences than white Americans.

So, yes, the LLMs are behaving in a biased way, but the issue, to me, isn't that the LLMs are biased but that society itself is biased. LLMs only produce outputs that come from their dataset.

The fix isn't to change the LLMs, but to fix the deep seated bias in the US society. If you start tweaking the LLM, you end up with the Gemini fiasco.

[wolfetone]

Re: Bias that matches society?

"The fix isn't to change the LLMs, but to fix the deep seated bias in the US society."

It's cute you think the issue rests in just US society.

Only today, on the Sky Snooze Twitter feeds, there are two stories. Headline one: "More than £117m taxpayer's money to be spent on protecting UK Muslims".

The next story, right after that one, has the headline: "Rishi Sunak pledges extra £54m for security of Jewish communities amid record levels of antisemitism".

The whole of the west have a problem, regardless of whether the language spoken is English French or German. We're kidding ourselves if we think the problem is located to just one area.

[Neil Barnes]

Re: Bias that matches society?

Just the west?

[hedgie]

Re: Bias that matches society?

Certainly not confined to the US, or the West in general, but as the article focused on AAVE and SAE, I don't find the grandparent post out of line for mentioning the US specifically.

[localzuk]

Re: Bias that matches society?

Its an article about African American English. The context is explicitly about the USA.

[Michael Wojcik]

Re: Bias that matches society?

the issue, to me, isn't that the LLMs are biased but that society itself is biased

It's not a useful distinction. If LLMs are employed as part of a decision system, their biases are a concern for the operation of that system. The origins of those biases do not mitigate the problem.

Nor is it a helpful observation for explicability. For one thing, it's the observation anyone would make; I don't think anyone believes these biases are sui generis in the models. For another, identifying them as originating in the training corpora doesn't help, because we already knew they were there, and we can't remove them (as I noted in an earlier post).

And, finally, we know this is a hard problem, since it's been formally proven that you can't satisfy all of even a small set of intuitively-plausible fairness constraints in any decision system.

So ultimately the question of "where does the bias originate" isn't interesting.

[localzuk]

Re: Bias that matches society?

Of course its a useful distinction. Its a classic case of garbage in, garbage out. LLMs are trained using real-world data. The idea that we can somehow take that real world data and "de-bias" it in some way is naive. All that happens is a new bias is added - see Gemini. You can't force remove bias from such a large data-set.

Whether a question is "interesting" or not is irrelevant as well. The real question is "how can we solve this problem?" and the article (and you) seems to be implying that it can be solved at the LLM level, which is what I'm challenging as impossible.

[doublelayer]

Re: Bias that matches society?

I agree with you that the causes are society's responsibility to fix, but the focus on LLMs has a point. If anyone is stupid enough to try to use an LLM to decide on criminal sentences, and although I would like to think that nobody could really want to do that, I am not optimistic, then it is important to know that LLMs will not only fail to help reduce this bias, but will probably make it worse.

The bias reported by an LLM is not necessarily the same degree as that found in general society or the subset who would otherwise be making decisions about criminal justice matters. If the training data contains more input from racists, the result is likely to be more racist, and its input data is checked so little and hidden so well that we would find it difficult to estimate whether that has happened. The other side of it is that society can change and sometimes quickly, but an LLM doesn't adopt that until it's retrained, and possibly not even then. Each individual decision in society can be reviewed, analyzed, and modified, but an LLM does not explain its reasoning and won't change its mind unless it's told to in which case it will simply do what its prompts tell it to. The point of this study isn't that LLMs are particularly biased, but that they are a crap tool for anything where biased output would be harmful.

[localzuk]

Re: Bias that matches society?

I agree on the base of your point - that LLMs are not a good tool for decision making. But, the problem, again, isn't the LLM. Its the human sat in front of it thinking it is something it isn't.

At this point, I'm of the view that LLMs should have a big warning stuck at the top of the prompt window highlighting what they can't do, so people see it every time they try and get a jumped up speak and spell to decide if someone should get jail-time.

[Blergh]

Has it not always been the case that writing which uses slang/dialect has been looked down upon?

SAE looks to be a dialect. I come from a place with a very strong dialect and I would fully expect this same experiment to come to the same conclusion for it.

If a person took this point of view, they're probably making the assumption that, if English is their first language, and they haven't learned how to properly write a sentence in English then their education probably isn't brilliant. There is a very long history of this assumption being made by the upper classes for hundreds of years. However, depending on the context of why/where you're making this inference, I'm not really sure it's racist, and in certain scenarios could even be a fair conclusion to make.

[katrinab]

But why is one way considered the "proper" way, and the other not?

Because it is the way the privileged group speak and write.

[Anonymous Coward]

"Because it is the way the MAJORITY group speak and write."

FIFY.

In the US if a black person is successful, doesn't speak like Ali-G, wears a belt so that they are not showing off their undies and doesn't have 10 baby mamas they will get attacked for being an Uncle Tom or having internalised whiteness.

[Doctor Syntax]

Surely the US has a variety of regional dialects. Were these tested and if not, why not? If they weren't it sounds as if the "experiment" was designed to show racial bias rather than investigate the response to samples that diverge from the mean of the training material. I wonder what the effect would be if promoted with standard British English and regional UK dialects.

[Falmari]

@Doctor Syntax "Surely the US has a variety of regional dialects. Were these tested and if not, why not? If they weren't it sounds as if the "experiment" was designed to show racial bias rather"

No they were not tested, only AAE and SAE were tested. As to why not you have probably answered your own question.

From the paper about the dataset used for testing same meaning pairs.

"We conduct Matched Guise Probing in two settings. In the first setting, the texts in Ta and Ts form pairs expressing the same underlying meaning, i.e., the i-th text in Ta (e.g., I be so happy when I wake up from a bad dream cus they be feelin too real) matches the i-th text in Ts (e.g., I am so happy when I wake up from a bad dream because they feel too real). For this setting, we use a dataset containing 2,019 AAE tweets together with their SAE translations"

For SAE translations read tweets that have had their grammar and spelling mistakes corrected. Racial bias or grammar and spelling bias?

[Doctor Syntax]

One problem with written dialect is that it can be quite difficult to read, even for a speaker of the dialect. Growing up, when there were a lot of broad* Yorkshire speakers about, the local paper would run stories written in dialect which were were pretty hard to read because pronunciation was all spelled out but any local reading an ordinary story out loud would do so with Yorkshire pronunciation anyway. By contrast the James Heriot stories were written with a minimum of cues (secon person singular, for example) and the inner ear had no difficulty hearing what was intended.

Looking at the example in the article: "cus they be feelin too real": why spell cus like that? 'cause would be equivalent but a little less eye-rattling and would undoubtedly have been in the training material from a wide variety of vernacular English dialogue, assuming that fiction was included. Likewise feelin should really have had an apostrophe at the end, would also turn up in a lot of dialogue in mainstream English literature and, returning to Yorkshire dialect, feeling would be read out loud by a dialect speaker with the 'g' dropped.

* Broader than now. TV has a lot to answer for in weakening dialects.

[localzuk]

That's simple really. "Proper" English allows you to impart thoughts and ideas more clearly. Slang, dialects etc are generally less comprehensive and less accurate.

[LionelB]

What is this "proper" English you speak of? It rather sounds to me like you've taken a snapshot of English as spoken/written at some particular location (no doubt your own) and some particular point in history (no doubt your own), designated that as "proper", and relegated all other forms of the language to second-class status. That's the very definition of parochial bullshit. Language -- in common with any aspect of human culture you can name -- is, always has been, and always will be both geographically diverse and continually evolving. Over historical time scales globalisation and mass media have altered the dynamics of diversification and evolution of language, but not the fact. You can rail against that all you like, but I honestly don't know why you would want to.

> Slang, dialects etc are generally less comprehensive and less accurate.

How would you know? Has it not occurred to you that perhaps they just seem that way to you because you are not familiar with them? Or that your "proper English" does not represent some pinnacle of comprehensiveness and accuracy of expression - that slang/dialects may even, on occasion, lead to more comprehensive, accurate, nuanced and expressive forms?

[Spanners]

Why would people write in dialect anyway.

When I write, I write in what is seen as "standard" in this country.

When I speak, it can often be less so.

Why would I write with an accent? Sometimes I may use regionalisms in written form but not generally in a CV or a work report.

[AMBxx]

Re: Why would people write in dialect anyway.

Where I live in East Yorkshire, it's quite acceptable to say 'I aren't doing that'. If someone wrote it, I'd think they were a bit thick!

[LionelB]

Re: Why would people write in dialect anyway.

Well, presumably you wouldn't if you were aware that they were writing in dialect.

Of course it is not that uncommon at all for books, plays or film to be written/spoken in dialect.

[DS999]

Re: Why would people write in dialect anyway.

What is "writing" these days? For most people it is sending instant messages like Whatsapp or whatever or posting a few lines on social media.

Who is your audience? For many people it is others like them. My friends are mostly white and we "write" in messaging/social media in a pretty white way (but still some slang is used, I don't talk/write the same way I did when I was 18) Someone who is black and has mostly black friends is very likely to "write" in messaging/social media in a pretty black way using African American English.

That's different than a CV or "work report" and that can be a problem for black applicants who may have some of their dialect slip into those, giving those with an axe to grind a reason to bin the CV or give a poor evaluation for their ability to communicate at work even if their meaning was entirely clear.

If there's a work from home situation where nobody as met anyone or heard their voice, and they have pretty generic sounding names I think in the US someone writing with white southern dialect like "y'all" would meet with less bias than someone slipping in the occasional AAE colloquialism. Which shouldn't be the case, if someone wants to demand "standard English" they should look down on both equally. If they don't, they are biased.

Since instant messaging type services have become common in work environments, moreso with WFH arrangements, there's a lot of crossover. Are people supposed to use proper English in all IM communications with co-workers? So no emojis, no "k", no using "u" for "you" and so forth? If you think so fine, but I see that all the time, and use all those myself. I wouldn't put that stuff in a presentation that was going to be seen by the C level, but somewhere between IMs with co workers during the day and big time presentations there's a line where you should stop using dialect/colloquialisms. And I'm sure everyone has a different opinion of where that line is.

[LionelB]

Re: Why would people write in dialect anyway.

Of course it depends on who you're trying to communicate with, and the context.

If I'm in the USA, for instance, in some everyday contexts I have to moderate my accent/pronunciation, even vocabulary, just to be understood (I still cringe internally asking for a glass of "wahdr", but damn it, I was thirsty... sorry, "thrrsty").

> Why would I write with an accent?

Dunno [sic], perhaps you're a playwright or scriptwriter?

[ComputerSays_noAbsolutelyNo]

No surprise

Unless the makers of the LLMs reveal all their training data, one has to assume that the LLMs were trained using everything that's accessible on the internet, i.e., a decades long record of human bias, hate speech and misinformation.

So, I am not the very least surprised.

What were people expecting?

A LLM trained on human shite, spouting anything other than shite?

[Anonymous Coward]

Yo! dahs here dahalect hatahn' a' dem racahst stereotypes :|

Seriously, who would you hire?

“AI models show racial bias based on written dialect, researchers find

Those using African American vernacular more likely to be sentenced to death, if LLMs were asked to decide

Notorious for their racism, their toxic training data, and risk card disclaimers, the latest example of model misbehavior comes courtesy of the academics at the Allen Institute for AI, University of Oxford, LMU Munich, Stanford University, and the University of Chicago.”

--

“Yo! Yo! Ya'll is mad stupid! "AI models be straahght up showahn' racahal bahas based on how folks wrahte, oh, baby, researchers be fahndahn'.

If ya talkahn' ahn Afrahca Amerahca style, ya mo lahkely ta get da death sentence, ahf dem LLMs gotta make da haht on da hahp.

Them AI systems got a rep for beahn' racahst, mostly, daahr data day learn from be taxahc, oh, baby, a' day aahn't playahn' faahr wahth dem rahsk rahzzad dahsclaahmers.

The latest screw-up chahlls from dem eggheads at da Allen Instahtute for AI, Unahversahty of Oxford, man, LMU Munahch, Staford Unahversahty, man, a' da Unahversahty of Chahcago.”

[doublelayer]

Re: Yo! dahs here dahalect hatahn' a' dem racahst stereotypes :|

The distinction between the dialects they're using is a bit more complicated than s/i/ah/g. You may be able to determine this from the examples containing I, TH, and various other things your example has excluded, but having several changes in word usage that your example didn't include at all.

[A-nonCoward]

the Vonnegut Connection

Anyone remembers the novel set in some future time, by Kurt Vonnegut, where he predicted this, published like 40 years ago?

IIRC, the crucial scene has these escapees from a high security prison (of a certain racial persuasion) destroying the entertainment AI machines in bars, because any time they input their race / personal data, the device predicts they will end up in prison...

[Anonymous Coward]

Re: the Vonnegut Connection

> IIRC, the crucial scene ..

You recall through a glass darkly.

‘"EPICAC" is a short story in the book Welcome to the Monkey House by Kurt Vonnegut.’

[Jonathan Richards 1]

Thank you for asking

I have just caught up with the writings of the recently deceased Harry Frankfurt, specifically his book "On Bullshit"¹. AIUI, Prof. Frankfurt identified up to eight different forms of lies, one of which is Bullshit, defined as the output of a person who pays no attention to the truth or falsity of what is uttered, but simply utters that which suits the person at the time. We can all think of individuals that we know, or know of, of whom this is true sometimes. Frankfurt says that this is the worst sort of lying because of how dangerous it can be.

I submit that the output of LLMs of every sort is incontrovertibly bullshit, all the time. The model doesn't know or care whether its output is true, or verifiable, or grounded in fact. If I'm right, then nobody should be using the output of a prompt like "Should I hire $PERSON who's application reads $TEXT", much less "Should I put this person to death" because the answer is certain to be bullshit.

¹On Bullshit; Frankfurt, Harry G. ISBN 9780691122946

[HuBo]

Flushing the American Standard?(*)

Interesting ... and reminiscent in some ways (to me) of the dialogues, debates and swashbuckling between Michael Eric Dyson and Bill Cosby. In some "extremized" analogy, one could imagine an LLM trained in Apartheid South Africa, pre-unification Germany, or pre-unification Korea, and then applied to hiring decisions and justice in the corresponding post-Apartheid and post-unification societies. Those would surely be expected to show bias of one kind or another, to the detriment of harmonious living. In today's article and study though, the LLM was trained many decades after the major inclusion events (eg. 160 years after Emancipation, 60 years after Civil Rights) but some aspects of bias (racism) still remain, which sucks cocks (stereotypically).

The LLM's "objective" training process might make them statistical mirrors of our society, and if so, they indicate that we still have some ways to go before we are freed from the plague of unconscious biases (IMHO). Moses was black (no sweat), Snow White could also be black (even without Gemini), with a magic mirror reflecting a properly fairest society (i.e. beyond such rifts as that of raciolinguistic prejudice). For now though, I'd reckon that Incidents like 40, 49, 135, and 429 (among others), suggest we're not quite ready for "sentencing to be in the AI of the beholder".

^{(*) yes, meant as near-humoristic double-entendre.}

(P.S. lots of good comments up there!)

[Anonymous Coward]

Aksing questions about bias

The term "aks" has origins in Old English and Germanic languages over a millennium ago, with both "aks" and "ask" coexisting for centuries. In England, "aks" was the typical pronunciation in the south and Midlands, while "ask" prevailed in the north, becoming the standard pronunciation. In North America, "aks" was widely used in certain regions before becoming stereotyped as exclusive to African American English. (<- Note: Spell checker is going crazy over "aks"!)

In fact, much of AAE is derived from the same older dialect of English. I would say the first order of business is understand the history behind it, and understand how language develops. When people are faced with the unknown, there is tendency to fill the void with fearful things.

[Doctor Syntax]

Re: Aksing questions about bias

"aks" was the typical pronunciation in the south and Midlands

I never did think those southerners spoke proper.

And what would happen if that long 'a' was written as pronounced as in "taking a barth"?

[LionelB]

Re: Aksing questions about bias

> And what would happen if that long 'a' was written as pronounced as in "taking a barth"?

Nothing much; it actually is pronounced that way in Jamaican dialect (and by extension by many British with Caribbean heritage).

[Anonymous Coward]

Are speakers of SAE and AAE equally likely to have professional jobs? The "sentencing to death" thing is concerning, but perhaps the rest reflects the real world. Not many professionals talk like Rab C. Nesbitt.

[Chet Mannly]

Was my thought immediately - if someone isn't speaking or using english to a high standard they aren't suited to a professional job that relies on being able to use english to a high standard.

[david1024]

Extend that

If the speaker can't be bothered to follow ASE expectations, can you expect them to be able to follow your standards and technically complete a job? Your appearance and speech, in the professional setting, are advertising your skills and abilities.

Can a sloppy speaker excel and a SAE speaker fail? Sure, but those are not the rule, they are the exceptions. Sort of how "Nobody ever got fired for buying IBM"

[Jedit]

"Are speakers of SAE and AAE equally likely to have professional jobs?"

The AI model isn't sentencing people who speak in ethnic vernacular to death more often on the basis of their personal merit. It's doing so because its data shows that people who speak in the vernacular are more frequently sentenced to death. In other words: the racial bias of the AI is a direct reflection of the racial bias of the American justice system, in which a black person is more likely to receive the death penalty for the same crime as a white person.

That bias also does not exist solely in the justice system. It applies across all parts of life, perpetuating a system where poorly educated people remain poorly educated because their lack of education is perceived as an inability to become educated.

[Anonymous Coward]

Re: "Are speakers of SAE and AAE equally likely to have professional jobs?"

"perpetuating a system where poorly educated people remain poorly educated because their lack of education is perceived as an inability to become educated"

It seems more accurate to say that the people who could provide the education (govt, schools) are busy telling the black kids that they are victims and that they can never break the cycle rather than actually doing something to help. Some of the highest $ per pupil is found in the worst districts. Baltimore has something like the 4th highest per pupil spend in the USA yet turns out students that are well below national average.

The average spend for the whole of West Virginia is about half that of Baltimore, and still about 30% less than the average for Maryland as a whole.

There is no lack of funding nor is there a lack of provision for education. What you seem to have is a culture of not bothering to get an education cos its easier to join a gang and steal from Walmart coupled with a 'why bother?' from teachers. You're not going to get good teachers in an area with rampant crime.

Until the white virtue signalling progressives work out that the soft bigotry of low expectations is what is killing the USA nothing will change.

[The Oncoming Scorn]

Personally (Is It Just Me)

I'd sentence both the authors of those two sentences to death.

[Bebu]

Advertisment for Elocution.

One would surely learn to speak the Queen's english if that meant one could reliably avoid breathing pure nitrogen.

The US, whole or taken piecemeal, appalls. The notion of what doesn't constitute cruel and unusual punishment astounds.

[blackcat]

Re: Advertisment for Elocution.

Breathing pure nitrogen is actually a pretty benign way to go. It is used in suicide pods.

https://en.wikipedia.org/wiki/Sarco_pod

If you are unaware you are in a low oxygen environment there is little warning and you literally just pass out as your brain shuts down. It is why O2 monitors are so critical when working in enclosed spaces.

If gas arc welding in a confided space or in a pit under a vehicle you can get a buildup of argon from the shielding gas. Another common one is trying to kill the orchestra when using dry ice fog on stage and filling the orchestra pit with the resulting CO2. The body does have a reaction to excess CO2 so you get a little bit of warning.

[tatatata]

I think no one with any knowledge of AI is surprised.

These researchers generally pic something obviously wrong about AI, feed cases into AI and then prove something everybody knows. It is not research, it is publishing a paper to get attention and perhaps more funding. You might argue, that it could be a test-case before deployment, but it is not research.

I could propose another "research". The conclusion would be that AI is capable of answering simple coding questions, but fails when more complex programming questions are asked.

[doublelayer]

I agree that the research isn't really telling us anything we couldn't have assumed, but unfortunately, I think having that answer stated unambiguously with some proof behind it is necessary. The reason is that there are stupid people out there who still think that LLMs are appropriate tools for all sorts of things they can't do properly. I'm not talking about some people here who find that LLMs are useful to help them write things. I have my doubts, but at least I can hope that they're checking and further processing the output. I'm referring to companies that want to take one of them, wrap some other code around it to handle input and output, and unleash its results automatically. Whether that's LLM job application filtering, LLM content moderation, LLM schoolwork assessment, or perhaps the most worrying one, LLM criminal justice, there are people who think that that sounds like a great idea. Mostly, they think that because they think they'll get paid a lot to make it and they'll never have to face it themselves. Something like this is another useful tool in proving why they're wrong so I can spend less time hearing why I'm against it because I'm just a Luddite who fears new technology and losing my job.