back to article Workday sued over its AI job screening tool, candidate claims discrimination

Workday stands accused of building algorithms that have resulted in bias against Black applicants in their 40s, according to a lawsuit. Launched earlier this week in the Northern District Court of California, the case alleges that the HR and payroll SaaS firm "unlawfully offers an algorithm-based applicant screening system …

  1. martinusher Silver badge

    You don't need AI for this

    This is just someone climbing aboard the "band waggon du jour". Resumes are routinely scanned by "artificially unintelligent" software, software notorious for weeding any but the most carefully written resumes that are designed to skirt around this filtering process. Certainly in the technical field we've learned to bypass the resume submission and use either personal contacts or headhunters, just submitting a resume to HR will get it roundfiled.

    My guess is that in my state (CA) many companies would jump at the opportunity to employ a "black, over 40" person, especially if they were doing government work since it helps with the HR box checking. This particular case doesn't look promising as an example since in HR speak he'd be a weak/marginal candidate with likely 'issues' with his work history.

    (Incidentally, although its popular to use the word 'disability' for all sorts of nuisances from a federal disability perspective you have to be pretty much dead to qualify. Being depressed because you can't get a job definitely doesn't qualify.) (Also, only 80 - 100 applications? A week? The EDD isn't quite as brutal as the UK's DWP but they shares quite a bit of DNA.)

    1. JoeCool Silver badge

      Prove it

      Any of it.

      Offer up 1 fact to support any of your claims.

      There is no reason to "trust" that AIs and their corporate users aren't biased, if not arbitrary.

      1. Anonymous Coward
        Anonymous Coward

        Re: Prove it

        "There is no reason to "trust" that AIs and their corporate users aren't biased, if not arbitrary."

        Surely AI is nothing BUT bias. That's entirely how they work. In the absence of any actual ability to reason and think, it's nothing but the bias of what the training set said was good and bad.

      2. NoneSuch Silver badge

        Re: Prove it

        "committed to trustworthy AI"

        AI is like Microsoft. I don't trust either with anything of consequence.

      3. martinusher Silver badge

        Re: Prove it

        >Offer up 1 fact to support any of your claims.

        This kind of BS resume scanning has been going on a whole lot longer than Workday and modern "AI". It actually predates what you could call 'first wave' AI, the production systems of the 1980s.

        Hiring for non government jobs can be a fickle process. I got hired as a software team manager in the late 80s in one of these "we'll hire you at whatever/whenever/however crisis" situations. As the new 'boss' I inherited, amongst other things, an office with a filing cabinet (they're like those icons you see on a computer but with real drawers that contain real bits of paper). Rummaging through the documentary junk pile during an idle moment I found my resume that had been submitted maybe 8 months or more before. It had been roundfiled, of course, except in California you don't do this immediately, state law requires you to file the things and keep them for a year or so (see a HR pro for details).

        There's a tendency for people to think that computers, jobs, HR and what-have-you are modern inventions. The only differences were hard copy (printed on a dot matrix) rather than on-line and laser printer.

    2. zuckzuckgo

      Re: You don't need AI for this

      You seem to be prejudging the situation.

    3. low_resolution_foxxes

      Re: You don't need AI for this

      I dislike these types of legal claims because it smells like 1) the actual cause/complaint of the bias has not been disclosed, 2) it smells like the legal company are shouting 'bias' to shakedown Workday for a payout in a class action lawsuit.

      I've tried reading the court case documents. His claims are woolly do not appear to mention any single factor or specific claim - it merely says that an AI/machine learning model has the "POTENTIAL" to include bias against African Americans, disabled people and "people over 40". I must be honest, I am not aware of any specific reason why an IT company would not want to hire a 41 year... but anyway. It essentially reads: "AI didn't hire me. This must be due to structural racism." and while I admit structural racism exists, you should at least outline how/why it exists.

      It seems to me, on the surface, that it is a fishing exhibition to see if there is any potential inherent AI bias. I could imagine a system perhaps, that could detect significant gaps in a CV, which could be flagged as a potential for someone having less experience in a role, but could also accidentally discriminate against people who have taken a career break or a long round of sick leave.

      I'm not entirely sure how it would discriminate against African Americans or 40+ people. Unless you personally believe that hiring people based on qualifications is racist because certain demographics do not perform as well in college (I don't even want to venture into that quagmire). I doubt it has a button that says "no 42 year olds please". But I could imagine it might have an option for "hiring junior staff for junior roles" because of salary implications. Ultimately, it may just boil down to the fact that the AI does not have a "diversity inclusion process" baked into it, which would possibly be interpreted by some types of people as being racist in itself.

      1. Missing Semicolon Silver badge

        Re: You don't need AI for this

        It needs testing. Submit fake resumes for 38 year old white people, with a similar educational background. If they get hits, then it's discriminating.

        1. Orv Silver badge

          Re: You don't need AI for this

          This is likely just the opening salvo in the litigation -- from here they'll seek to move into the discovery process, where they can obtain statistics about what types of candidates the system accepts or rejects.

        2. spold Silver badge

          Re: You don't need AI for this

          ...just take the same bunch of resumes - change one factor such as ethnicity, age, whether you identify as disabled, or even just the postcode. See if the results are the same.

      2. low_resolution_foxxes

        Re: You don't need AI for this

        I'm also going to take a wild guess here, based purely on guesswork and the nature of the class action lawsuit.

        The individual does not mention the specific cause. Yet claims to be a victim. Being a "victim" would neatly allow him to take part in the class action payout. But harder to tell if it's being driven by the individual, or the legal team seeking to manufacture a lawsuit

  2. Ken Moorhouse Silver badge

    ...trustworthy AI...

    Spot the oxymoron.

  3. trevorde Silver badge

    Alternative headline

    "AI trained to select white, 30s males does not select black man in 40s"

    1. Version 1.0 Silver badge

      Re: Alternative headline

      Clearly the AI was written by a Bag O Wire

  4. Eclectic Man Silver badge

    Little more than "automated pseudoscience"

    Surprised I did not see the link to an article from the Register last October in this one:

    Admittedly this refers to AI analysis of images of applicants' faces rather than CV scans:

    "a paper published in the journal Philosophy and Technology by a pair of researchers at the University of Cambridge, however, demonstrates that the software is little more than "automated pseudoscience". Six computer science undergraduates replicated a commercial model used in industry to examine how AI recruitment software predicts people's personalities using images of their faces. "

    I do recall, years ago reading an article which claimed interviewing was pointless. Just get the names of each candidate with adequate qualifications for the job, put them in a hat and select the desired number at random.

    1. talk_is_cheap

      Re: Little more than "automated pseudoscience"

      You are missing the key step when someone gives you a pile of 50+ CVs and tells you to pick 2 for interviews. You first remove at least half of them by only selecting the lucky ones - divide the pile into 2 without reading any of the CVs and put one pile in the bin. You are now left with the lucky ones.

    2. withQuietEyes

      Re: Little more than "automated pseudoscience"

      I had the same reaction - I'm more surprised that there aren't more (entirely justified) lawsuits against this kind of AI nonsense. They work exactly as designed: make the same selections as the assholes whose decisions you fed it. Perpetuate our biases, but give us the shield of objectivity because "a machine made that decision, so any discrimination is the machine's fault!"

      (Though I do wonder if the lawsuit would make more sense if a few more people banded together, I think this guy on his own might have trouble winning.)

    3. Orv Silver badge

      Re: Little more than "automated pseudoscience"

      Several years back Google assessed their hiring process by checking in on whether candidates that had been highly ranked were more likely to still be working there later. They found their hiring process was no better than chance.

      No one knows how to do hiring.

  5. ChoHag Silver badge

    > We ... mitigate any unintended consequences

    So you're intentionally baking in the racism. Gotcha.

    1. Eclectic Man Silver badge

      intentional vs sub-conscious vs accidental

      "So you're intentionally baking in the racism."

      Well, most bigotry is actually sub-conscious. Having one black friend does not mean you are not racist, having one gay friend does not prevent you being homophobic, liking a woman doesn't prevent you from being a misogynist. And arranging your contacts in alphabetical order does not mean you deliberately discriminate in favour of the A's at the expense of the S's. It 'just happens that way.' The Irish comedian Dave O'Mahoney visited his agent to find out why he wasn't getting much work. While there, the agent took a phone call, and started going through his acts in alphabetical order, at which point 'Dave O'Mahoney' became 'Dave Allen', and his career flourished.*


  6. Anonymous Coward
    Anonymous Coward

    I'm at the point where if I see that the company uses Workday for part of the recruitment process, I don't bother applying.

    1. Anonymous Coward
      Anonymous Coward

      Let's just say that having used Workday and seeing that when you request "1 day" for X purpose ( training, holiday, etc ) that you then have manually type "09:00" in the "Begin Day At" box and "17:00" in the "End Day At" box on the submission form, right next to a message that states, "You must now enter 09:00 as start of day and 17:00 as end of day in each box below where you will be booking for the required days.". So if you have to book a 2 weeks holiday off you have to actually copy "09:00" into 10 boxes and "17:00" in 10 boxes!!! FFS!! It's 2023 not fecking 1983!

      This this little snippet of the cesspit that is Workday design, tells you all you need to to know about how utterly useless the dev team at WorkDay must be if they cannot just simply fill that in as defaults and then ask you to confirm Yes or No.

      1. Why Not?

        Ate their own dog food?

        They probably hired a developer who looked cool.

        Its not like W3C and Microsoft have published any advice.

      2. Anonymous Coward
        Anonymous Coward

        Their cesspit software is so broken that neither my Bachelors or Masters degree are listed on their application UI. Both are fairly common degrees, and you can't enter any free text into the box. So my only choice is to pick two unrelated degrees. What is their fancy Artificial Stupidity algorithm going to make out of that? It's a waste of time to apply!

        Besides, if by chance you end up working there, you will have to deal with this cesspit software on a regular basis.

      3. Orv Silver badge

        We use Kronos, which is almost as bad. For each day I take off I have to select 'Vacation' and then put in a duration of 8 hours, in spite of the fact that I'm salaried, not hourly, and I can't even take partial-day absences.

  7. Nifty

    Funny that, I held one of those Guido Fawkes masks in front of my face for the interview and got the job!

  8. froggreatest

    computer says no

    Workday job application “portals” are quite terrible. Every time I applied it was necessary to fill in all of my details along with the job experience. The futuristic “import my cv” would always trip over in some places and I would need to fix the mistakes. Now if there is AI somewhere, I can only imagine how great it is.

    This chap had to deal with such crazy user experience for 80-100 times.

    1. LybsterRoy Silver badge

      Re: computer says no

      Its a while back now but I worked in recruitment and looked at some of the cv analysis software. Generally speaking it would bo a mediocre job as long as the cv was in the right format. With a different format it would just fall over.

      If its AI trained on "good" cvs and "bad" cvs I'd like to know who decided which was which and how they decided. eg "This cv must be good - he's a director of the company"

    2. Bbuckley

      Re: computer says no

      problem is there is NO SUCH THING as AI. It is a metaphor for Selection Bias. Amazing in this day and age it is not illegal.

    3. Eclectic Man Silver badge

      Re: computer says no

      Umm could this be why older applicants who left the workforce during the Covid-19 measures are still not employed? I mean HMG wants us oldies back being 'economically active'* but we aren't getting the jobs, so maybe it is the hiring process that is at fault?

      *I'm 62 and 'retired'. I resent the idea that I am 'economically inactive' as I still have to pay for things.

    4. Anonymous Coward
      Anonymous Coward

      Re: computer says no

      Booking days off in Workday makes you think that the devs must be working through a timewarp suing some 80x40 text terminal, the interface design in Workday is the absolute pits of shit design! A child of 4 could design a better interface than Workday's!

      1. Michael Wojcik Silver badge

        Re: computer says no

        I'll just note that back when we were using IBM OV/VS (neé PROFS) for this sort of thing, it was more convenient, better performing, and easier to use than Workday. And that was a green-screen application, much of which had been developed using 3270s – though with modern emulators you have a choice of 80x25 or 80x43.

    5. withQuietEyes

      Re: computer says no

      In fairness, aren't basically all application portals terrible? I've yet to encounter one that didn't make me seriously reconsider how badly I wanted to make that application. (And I've never heard back from a single one, which, while a standard part of the application process, is fucking frustrating.)

  9. Someone Else Silver badge

    Oxymoron Alert!

    [Workday] said it was "committed to trustworthy AI" [...]

    Let's assume that this is not the patently obvious oxymoron that, on its face, it is. What constitutes "trustworthy" AI? Is there a certification for such a thing. I mean, we all know that ChatGPT can be "trained" to spew bullshit, racist epithets and so on. And such "training" was done with the best of intentions.1 Such a "commitment" is right up there with, "the safety and security of our customers data is of the upmost importance to us".

    1We've been told time and time again that ChatGPT "training" was supposed to be neutral, done under "strict supervision", yadda, yadda, yadda...

    1. CatWithChainsaw

      Such "training" was done with the best of intentions

      As if we don't know where that road leads!

    2. Michael Wojcik Silver badge

      Re: Oxymoron Alert!

      There are two large research communities working on "trustworthy" AI: one on this problem domain (ethics, bias, etc), and the other on alignment (i.e. not-everybody-dies outcomes). They mostly don't get along, but the point is they're producing a whole big bunch of research on the question "What constitutes 'trustworthy' AI?". There is by no means a consensus, among either group.

      For example, some people have argued at length that you can't even come close to trustworthy AI (or ML or whatever term you want to use) without interpretable models, and I'm quite confident Workday doesn't have an interpretable model. Others think that's the wrong goal, so they wouldn't be satisfied even if Workday did.

      So yes, Workday's statement is rubbish from a theoretical position, as well as in practice.

    3. Michael Wojcik Silver badge

      Re: Oxymoron Alert!

      We've been told time and time again that ChatGPT "training" was supposed to be neutral, done under "strict supervision",

      Have we? Because that's not correct. The GPT training process is unsupervised – it's not feasible to do supervised learning for an LLM. What's supposed to reduce misbehavior for ChatGPT is the RLHF post-training process.

  10. Anonymous Coward
    Anonymous Coward

    Why is he putting his DOB on his resume?

    Putting anything extra that allows bias, unconscious or otherwise on a resume is not a great idea. Especially photos!

    1. LybsterRoy Silver badge

      Re: Why is he putting his DOB on his resume?

      One of the things I suggested people do with their cv was make it easy for the recruiting company to reject. No point in spending time & money going for an interview when you WILL be rejected. Age was something some of my colleagues told people to leave off. Weirdly enough when someone walks through the door with gray hair I tend to guess they're no spring chicken. I also used to recommend don't claim technical skills you don't have, especially when its easy to check.

      1. Spazturtle Silver badge

        Re: Why is he putting his DOB on his resume?

        If the job listing says not to include your DOB like most do these days and you include it then you will get rejected for failing to follow simple instructions, whereas if you had done as told you might have got hired at interview stage. There are a lot more companies who are willing to hire older people than there are who are willing to hire people who can't follow instructions.

      2. werdsmith Silver badge

        Re: Why is he putting his DOB on his resume?

        This is the problem. You can have as many rules as you like telling employers they have to consider everyone. But at the end of the process they decide who they offer the job to. So forcing employers to interview people they don’t intend to recruit is just a waste of time and expense for the employer and the prospect.

        Might as well have under 35 years old as a requirement on the job spec, rather than trying to hide the same requirement with such words as “to join our young dynamic team”.

        If you’ve no intention of employing people 40 years old plus, don’t waste my time and money with your stupid rules.

        1. tiggity Silver badge

          Re: Why is he putting his DOB on his resume?

          A DOB is not needed for "minimum" age estimates

          Given employers typically want dates of academic qualifications, job history* etc...

          e.g. If someone got their degree 20 years ago then a fair guess they are in their 40s or older.

          *Often see last n jobs or last n years

          Last n jobs may seem innocuous, but is not if someone has been in each role for a relatively long time as could easily have a job start date of decades ago.

        2. Missing Semicolon Silver badge

          “to join our young dynamic team”

          Oh, yes. Puke, puke.

    2. withQuietEyes

      Re: Why is he putting his DOB on his resume?

      I don't know about the specific process Workday uses, but I've encountered a few where the DOB was a required field - you couldn't even submit your application otherwise. And for photos, I was told by every adult I know (Switzerland) to put one on my resume, so it might have been something along those lines. (I like the Canadian style way better - you're not even allowed to have one!)

      1. Michael Wojcik Silver badge

        Re: Why is he putting his DOB on his resume?

        Interesting. I never heard anyone recommend putting a photo on a resumé or CV, unless you're sufficiently well-known in your field that it doesn't matter. (This exception is more common in academia, where in smaller fields most established members know one another anyway, from conferences and the like. Most of the senior scholars in my wife's field, at least those working in the US, could pick her out of a crowd.)

  11. Matthew "The Worst Writer on the Internet" Saroff

    This is a Feature, not a Bug

    You see this again and again, ride sharing, where drivers avoid people of color, Facebook allowing microtargeting in employment ads to discriminate, hosts on AirBNB not replying to black would be renters, etc.

    One of the allures of this system for a certain segment of this society is the ability to discriminate without fear of consequences.

    1. CatWithChainsaw

      Re: This is a Feature, not a Bug

      Hiding prejudice behind a computer, a machine that is supposed to deal with numbers and truth, is a way to externalize consequences.

  12. Anonymous Coward
    Anonymous Coward

    Artificial Discrimination

    The computer says, “no”

  13. pompurin

    Apply for any work with the UK civil service and you can't provide any of the following:

    1. Your name

    2. Your university/school

    3. Your DOB

    4. Any other information that may Identify you in other ways

    This is so they can box tick HR initiatives, and waste time interviewing people who realistically never stood a chance.

  14. Anonymous Coward
    Anonymous Coward

    The road goes both ways

    White male here. My first real job interview as a teenager was with the state ecology department. Summer job, picking up trash alongside the highway. Not the best of jobs, but it paid a lot more than bussing tables at a restaurant, and did not require working weekends. I met the hiring manager and he told me I would be great for the job and he would love to hire me.

    Then he told me he was required to hire a "Black female" and could not make me a job offer. So much for all that "White privilege" I had going for me, right?

    1. Anonymous Coward
      Anonymous Coward

      Re: The road goes both ways

      Yes, this definitely proves that you have never enjoyed any unearned privilege. Congratulations.

  15. el rex

    not hired by anyone else in the meantime?

    Hey, even if Workday was indeed biased (it isn't obviously) why anyone else bothered to hire this apparent genius?

    Perhaps his work history is full of grievances against his employer?

    Perhaps he's a full time sjw and less of a productive employee?

    How could he only send his CV to Workday-powered HR, has he not tried with other companies in the meantime? And if he did why even the non AI-Based HR have not hired him?

    This really stinks of easy-money-grab to me, especially considering anything with SecDevOps or Cloud Architect in the CV gets you hired in seconds...

  16. Snowy Silver badge

    ITT Technical Institute, Indiana.

    As far as I can see they shut down in 2016 so that is quite a gap between then and now?

    1. Michael Wojcik Silver badge

      Re: ITT Technical Institute, Indiana.

      I don't follow. I earned my most recent degree in 2012 (21 years after my first two). So what?

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like