Unis turn to webcam-watching AI to invigilate students taking exams. Of course, it struggles with people of color

AI software designed to monitor students via webcam as they take their tests – to detect any attempts at cheating – sometimes fails to identify the students due to their skin color. Products like ExamSoft are being used by colleges and other organizations to make sure students aren’t cheating when they take exams in front of …

[Anonymous Coward]

But does the AI detect that someone one else in the next room has plugged into the screen and keyboard ports on the pc?

As soon as you leave physical invigilated space your exam is compromised completely rather than just somewhat (you've no idea what attempts to cheat are made even in physical invigilated spaces)

[John Jennings]

Actually, they likely can in many cases.

I recently went through the Pearson/Vue system with webcams. Its not just the web-cam, but an app that tunnels into some sort of VM... They can and do detect pretty much anything running on your machine. You have to disable screen sharing apps like teams or dameware - and they detect these proactively - , and require other machines in the vacinity to not only be turned off, but covered up with a blanket.

You have to take multiple pictures with your webcam of your surroundings - detailing much of your surroundings, as well as your ID etc.

A chat window pops up from time to time which needs answering (at least it did before the test, I dont know if it does it during it, it didnt for me)

I believe Pearson/vue have human invigilators watching your video - but I wouldnt be suprised if they did some machine learning as well... The chat windows seemed real enough - but it could have been a touring test!

[Tom 7]

Re: Actually, they likely can in many cases.

Presumably if you run it in VM you can fool them into thinking that your computer is not what it is and share the non-vmed screen with as many as you want.

Also does it work on Linux?

[Neil Barnes]

How will Toyota be able to tell

That their robots aren't cheating?

[AndyS]

Re: How will Toyota be able to tell

Simple, they'll paint them all white.

[Khaptain]

Why use the word racist.

This is an algorithm that has not yet been perfected, how can this person call it racist when it knows nothing about people or colour? For a university student that does not seem like a very intelligent approach, quite the opposite really.

[Luiz Abdala]

Re: Why use the word racist.

That's the funny part. The sample base to train the AI was not large enough.

Hanlon's razor: "Never attribute to malice that which could be adequately explained by stupidity."

[Anonymous Coward]

Re: Why use the word racist.

Was the company or its programmers racist - no.

Was the software racist - yes.

Racism requires intent. Racist does not.

If the software resulted in racially biased results, intent is not an issue. It is racist because it yields different results for different races.

[chivo243]

Re: Why use the word racist.

Poor programming... not racist

[gnasher729]

Re: Why use the word racist.

If you were black and cheated out of your exam because of your skin colour, I’m sure you would agree that it is racist.

Racist or not isn’t necessarily a matter of intent, but if it happens due to incompetence or thoughtlessness it’s just as bad.

[Cederic]

Re: Why use the word racist.

I would report a technical issue and invite an alternate approach - e.g. a human invigilator.

If nothing else the person sat at the table doesn't know the underlying reason for the issue. Maybe it's not skin colour, it could just struggle with beards.

(I do actually agree it was likely to be skin colour, not least because I'm a photographer that finds it much harder to capture the features of some people than others purely because of camera sensor contrast constraints. That doesn't make it racist.)

[katrinab]

Re: Why use the word racist.

There is direct racism, where you believe that people of a particular race are inferior to people of a different race, and want to actively harm them in some way.

There is indirect racism, where you treat everyone the same, but in a way that adversely affects some people because of their race.

People with darker skin were unable to take exams because a facial recognition algorithm couldn't recognise their faces. That is indirect racism.

[John Jennings]

Re: Why use the word racist.

Not necessarily - cameras are simply not good enough to pick out the lower contrasts you often find in people of colour. Or, there are bright spots from reflections which breaks the array of facial modelling... Ye canne break the laws of physics of a camera - that does not make it racist - especially if the camera is only capable of 1-2 megapixels. Its what is technically possible at this time. What is racist (perhaps) is not offering an alternative verification if the system is known to be weak with somone

[fajensen]

Re: Why use the word racist.

Wouldn't IR illumination work, technically at least?

[katrinab]

Re: Why use the word racist.

Because it discriminates against people with dark skin. Whether they intentionally set out to do this, or just didn't bother to test on different skin colours, it doesn't matter. People with dark skin are adversely impacted, therefore it is racist. Racism doesn't require intent.

[DavCrav]

Re: Why use the word racist.

"Racism doesn't require intent."

So light is racist because it doesn't reflect as much off a black guy's face?

[gnasher729]

Re: Why use the word racist.

You are racist for coming up with all kinds of stupid excuses and straw men.

[DavCrav]

Re: Why use the word racist.

"You are racist for coming up with all kinds of stupid excuses and straw men."

You mean stupid excuses like contrast? And anyway, the guy seemed to think an algorithm, which isn't alive, is racist, because racism doesn't require either intent or life signs. Since light treats white and black people differently, I want to know whether it is racist.

[Dan 55]

Re: Why use the word racist.

Imagine if you knocked up a website without accessibility which stopped blind or disabled people from using it. This is discrimination. You might never have thought to do add accessibility to the website but it was your job to do so and you didn't do it.

Now imagine you're training an algorithm and choose a dataset which wasn't large or varied enough for it to do the job required, and produces suboptimal or wrong results for non-white people. It was your job to ensure this didn't happen but you didn't. This is also at the very least discrimination.

Light is obviously not racist, you bleeding fool.

[DavCrav]

Re: Why use the word racist.

"Light is obviously not racist, you bleeding fool."

Yes, it's called reductio ad absurdam.

Of course light isn't racist. But light has differing effects with regards white and black people, and in particular makes it much harder to make out black faces in dark conditions. If humans (who are very good at pattern recognition) have more trouble in low-light conditions with black faces than white ones, why it is clear that a computer should be able to treat both equally well in higher-light conditions?

The point is that racism is meant to be where decisions taken by people have a deleterious consequence for one or more racial group. After many, many examples of algorithms having difficulty with black faces, perhaps it's time to start thinking that, maybe, algorithms have a harder time distinguishing black faces because it is an objectively harder problem?

If that is the case, then suddenly it looks unseemly to throw words like 'racist' around to describe the perhaps completely blameless programmer (because, as you have stated, non-people cannot be racist, so if the algorithm is racist that means the people who programmed it must be).

Repeating the 'these computer programmers are racist' mantra/libel is starting to become tedious, when the evidence is stacking up that it is just more difficult. Of course, evidence is not a proof, but it should be fairly easy to check. Use a million-strong dataset of white faces and a million-strong dataset of black faces. Then evaluate the performance of the algorithm on a new selection. I've not heard of such an experiment being conducted, but it would go a long way to settling this argument.

And, here's the interesting question: what happens if it turns out that these algorithms will always be worse at distinguishing black faces? You know, like humans are.

[nichomach]

Re: Why use the word racist.

And that's why reductio ad absurdum is treated as a logical fallacy. Scenario: You are developing a technology to invigilate exams. This depends upon facial recognition. The people being invigilated may be black or white, but you only test on white people. Your product discriminates against black people because it can't recognise them. You didn't *intend* to adversely affect them, but you should damn well have tested your product to make sure that it didn't, and if you were unable to ensure that it didn't, you should not have certified that product as ready for use. That is casual racism; you do not intend to be racist, but you are, because you either do not consider the issue, or ignore it.

[stiine]

Re: Why use the word racist.

"The people being invigilated may be black or white..."

This is about as wrong as a statement can get without stating that true = false.

Humans can be, and are, nearly every shade of brown/pink/yellow from white to black, regardless of their race.

Also, someone said that developer was racist for not choosing a valid image training dataset. That's a very unhealthy assumption considering that several of these datasets have recently been pulled because of their contents, or lack their of.

[Dan 55]

Re: Why use the word racist.

After many, many examples of algorithms having difficulty with black faces, perhaps it's time to start thinking that, maybe, algorithms have a harder time distinguishing black faces because it is an objectively harder problem?

No, it's time to start choosing better datasets.

And, here's the interesting question: what happens if it turns out that these algorithms will always be worse at distinguishing black faces? You know, like humans are.

Fuck me, we have to start from the very basics:

Cross-race effect

[JetSetJim]

Re: Why use the word racist.

> perhaps it's time to start thinking that, maybe, algorithms have a harder time distinguishing black faces because it is an objectively harder problem?

Perhaps it is, but then perhaps people should also stop releasing products that claim to do something, but don't work so well for the people of colour that make up large portions of society in general. Perhaps these algorithms are not yet ready for the prime time and need further work?

[codejunky]

Re: Why use the word racist.

@Dan 55

"Imagine if you knocked up a website without accessibility which stopped blind or disabled people from using it. This is discrimination. You might never have thought to do add accessibility to the website but it was your job to do so and you didn't do it."

No it isnt. That is not discrimination at all full stop no way. Discrimination would be to purposefully stop someone from using the website, say intentionally making it impossible for disabled people to use with the intent of stopping them.

If you accidentally nudge someone or back into someone that is called an accident. A mistake. If you thump someone that is assault. You might hurt someone with both actions but only one of them is malevolent, intent matters.

I look forward to the 'god is racist' for making people different etc (I am not a believer in such a being but I expect some idjit to come out with it).

[Jimmy2Cows]

Re: Why use the word racist.

As has already been mentioned, discrimination does not require intent.

Failing to add accessiblity features to a website resulting in an exclusionary experience for visually impaired etc. It's this exclusion which makes the website discriminatory, whether or not that was the authors' intent.

Same for AI failing to recognise black people more than white people. That (probably) wasn't the authors' intent, but when it disproportionally affects black people, that is discrimination. In this case racial discrimination. Does need to be planned. It's enough for it simply to occur.

Given how often this situation crops up whenever AI is used for facial recognition, in any context, ExamSoft were naive at best to consider it a viable, non-discriminatory solution.

[codejunky]

Re: Why use the word racist.

@Jimmy2Cows

"As has already been mentioned, discrimination does not require intent."

Dan 55 substituted the word discrimination for racism, so in the context of racism it does require intent to be racist. Racism is an intentional act. You are right about discrimination not requiring intent.

'Failing to add accessiblity features to a website resulting in an exclusionary experience for visually impaired etc.'

True just as running is exclusionary to people unable to run. Or a more useful observation, increased melanin content results in reduced vitamin D. Every day normality that people are different and a standardised system cannot apply to everybody, so we try to make them fit as many as possible.

"Given how often this situation crops up whenever AI is used for facial recognition, in any context, ExamSoft were naive at best to consider it a viable, non-discriminatory solution."

Given how often this situation crops up you would have thought someone would have tried to make it work for coloured people. Yet all of these algorithms seem to have the same inherent problems. As someone else pointed out, he had a beard. I wonder how it would cope with some of the fashionable moustaches.

[nichomach]

Re: Why use the word racist.

Discrimination on the basis of race IS racism. There isn't any extra bar to leap to get there. This is just casual unthinking racism.

[Roland6]

Re: Why use the word racist.

>This is just casual unthinking racism.

Aka Unconscious bias.

[Dan 55]

Re: Why use the word racist.

I did not substitute, as the word "racism" is so loaded and it's difficult to have a rational debate about it, I tried to explain in a way that shows that as discrimination does not require intent (because that argument is generally accepted) neither does discrimination concerning race (aka racism) require intent. It's known as "casual racism", look it up.

As someone else pointed out, he had a beard. I wonder how it would cope with some of the fashionable moustaches.

Facial recognition algorithms are now being trained with masks to recognise people while wearing masks. This itself should give you a hint that solving problems with facial recognition algorithms and race is not impossible and the problem is not down to race itself, so we can't just shrug our shoulders and say "oh, it's because they're black, this is impossible to solve and they should just put up with their life being shittier because of it" or "he has a beard" or whatever. The algorithm must be trained with more images of black people's faces, more images of bearded faces, and more black people's faces with beards so that it reflects real life.

[codejunky]

Re: Why use the word racist.

@Dan 55

"I did not substitute, as the word "racism" is so loaded and it's difficult to have a rational debate about it"

My mistake, so you were discussing something different (discrimination) where I assumed your meaning to be racism. My mistake.

"I tried to explain in a way that shows that as discrimination does not require intent (because that argument is generally accepted) neither does discrimination concerning race (aka racism) require intent"

This is why it sounds like you used substitution because racism does require intent. Racism is feelings, prejudice the very things that require intent (on the actor or the receiver). Discrimination is the norm that people are different. Racism is to believe in superiority due to difference.

"casual racism"

Which still requires the belief in superiority over another race. Without that it doesnt exist.

"This itself should give you a hint that solving problems with facial recognition algorithms and race is not impossible and the problem is not down to race itself"

Awesome, so do it. If it is so easy then go ahead and show all these boffins who have tried and failed how to do it. Remember this must work with varying quality of camera with highly varying surroundings. how good are these mask trained AI's with colour? If its good enough maybe half the face needs cutting off during training.

"The algorithm must be trained with more images of black people's faces, more images of bearded faces, and more black people's faces with beards so that it reflects real life."

And you will still have people fail. It is a statistical probability model so some people will fail the detection.

The other possibility is that nobody does the exam because of covid restrictions and so screw the lot? Or is it best for the majority to test with this technology and those it cant work with have to do it a different way? Covid wont wait for more AI testing, its here. And the education system needs solutions, even imperfect ones.

[Dan 55]

Re: Why use the word racist.

My mistake, so you were discussing something different (discrimination) where I assumed your meaning to be racism. My mistake.

From the Cambridge dictionary:

discrimination

treating a person or particular group of people differently, especially in a worse way from the way in which you treat other people, because of their skin colour, sex, sexuality, etc.:

It is indeed your mistake.

Discrimination is the norm that people are different. Racism is to believe in superiority due to difference.

So, er, racism is a kind of discrimination, then? Yes. Are we there yet?

If it is so easy then go ahead and show all these boffins who have tried and failed how to do it. Remember this must work with varying quality of camera with highly varying surroundings. how good are these mask trained AI's with colour? If its good enough maybe half the face needs cutting off during training.

They've tried and failed to do it because they've shoved in a dataset which doesn't have enough examples, not checked the results enough, and put it into production.

The other possibility is that nobody does the exam because of covid restrictions and so screw the lot? Or is it best for the majority to test with this technology and those it cant work with have to do it a different way? Covid wont wait for more AI testing, its here. And the education system needs solutions, even imperfect ones.

That penalises people because they are black. Well done you. You do understand where this leads, don't you? Actually you're probably blissfully unaware, and I'm not willing to argue about it for the rest of the day. As always, the George Bernard Shaw quote applies.

[uccsoundman]

Re: Why use the word racist.

Or you could do like a company I worked for. Beards messed up the security cameras, so they implemented a new rule; everybody had to shave off their beard. Don't like to be clean shaven? Find a job elsewhere. This was really hard on the men who (due to medical issues) could not shave their beards. They had a hard time finding new work. This also begged the question of the men who wear beards for religious reasons. This was never an issue for us. The company was in the Bible belt and very few men of those religions lived there.

[Roland6]

Re: Why use the word racist.

>No it isnt. That is not discrimination at all full stop no way.

In the UK it is - such a website would not comply with the requirements of the Equality Act 2010 (which replaced the Disability Discrimination Act 1995).

Any one designing and/or developing webpages and/or websites in the UK would be expected know that their work must take account of this Act. So "I forgot" isn't a defense.

[Mike 137]

Re: Why use the word racist.

"Imagine if you knocked up a website without accessibility which stopped blind [...] people from using it."

We don't have to imagine. Huge numbers of web sites do fail in this respect.

[codejunky]

Re: Why use the word racist.

@DavCrav

"because racism doesn't require either intent or life signs"

This is one of the best tricks of making everything racist- intent doesnt matter. Intent would indicate malice but since that is mostly not an issue bar extremists and imported prejudice it is difficult to argue racism all the time.

Is it a broken AI system, yes. A statistical model system making mistakes is going to happen, and certain features may cause it issues (I wonder how it works for disfigured people?). That is a normal response to this situation. 'RACIST!' is the response if we strip away thinking and ignore intent.

Note- it would then potentially be racist to stop people failing the statistical model incorrectly from taking the test although probably under different observation.

[jospanner]

Re: Why use the word racist.

And who says that Britain has a race problem?

[codejunky]

Re: Why use the word racist.

@jospanner

"And who says that Britain has a race problem?"

Oddly enough people who dont get out much it seems. The UK is one of the least racist countries in Europe.

[Anonymous Coward]

Re: Why use the word racist.

Home secretary's 'dangerous' rhetoric 'putting lawyers at risk'

‘I’m so scared’: They came to the UK seeking safety, only to be targeted by the far right

Moria camp: Germany, France and other EU countries to take 400 child refugees

[codejunky]

Re: Why use the word racist.

@AC

Bar the sources being the Guardian and Independent, even if we accept them at face value that still does not refute my comment. Even a 2019 EU study found the UK second least racist after Malta-

https://fra.europa.eu/sites/default/files/fra_uploads/fra-2019-being-black-in-the-eu-summary_en.pdf

[Khaptain]

Re: Why use the word racist.

@katrinab is using the Flamebait icon so I would presume that the comment is actually stabbing at the ridiculous usage of the word "rascist" which has completely lost it's original meaning.

It's as though is has become counterproductive as it no longer defines a definitive situation, and with no definitive situation we can longer take any positive action as we no longer no what we are trying to resolve.

It now appears to having become defined as "someone that does not does not agree with my ideology, regardless of how ridiculous the concepts of my ideology".. and I do not see how that can help those that actually suffer from racism as it was previously defined.

[jospanner]

Re: Why use the word racist.

Discrimination does not require intent.

No amount of Daily Heil reading changes this fact.

[codejunky]

Re: Why use the word racist.

@jospanner

"Discrimination does not require intent."

But racism does

[jospanner]

Re: Why use the word racist.

No, it does not. Just because it hurts your feelings doesn't make it incorrect.

Racism is the discrimination based on race. Intent comes nowhere near this definition. Whether you have a bias towards doing shitty things to black people is the important thing, not how much you intended to do it.

[codejunky]

Re: Why use the word racist.

@jospanner

"Racism is the discrimination based on race"

racism

/ˈreɪsɪz(ə)m/

prejudice, discrimination, or antagonism directed against a person or people on the basis of their membership of a particular racial or ethnic group, typically one that is a minority or marginalized.

the belief that different races possess distinct characteristics, abilities, or qualities, especially so as to distinguish them as inferior or superior to one another.

Quick search yields the answer. Directed against. Inferiority/superiority. This requires intent. Yes the software can discriminate on race, but without the developed intention to exclude on race it would seem a mistake. But since AI in general seems to suffer when analysing high melanin content it is more probable to be a physical difficulty instead of an intended one.

Discrimination is about recognising differences. Racism is a bad part of discrimination but is not the same thing. I have mentioned before that melanin content has an effect on people medically, particularly reduced vitamin D. Noting the difference is discriminatory because the needs may be different, but we wouldnt call that racism as there is no derogatory aspect (we aint thinking one is superior to the other).

[gnasher729]

Re: Why use the word racist.

Isn’t it strange that white people are not affected? I’m sure the algorithm could be calibrated so it affects only white people, not black people. So you would be fine with that, right?

[AndyS]

Re: Why use the word racist.

Nobody is fine with it, stop being obtuse. If the software was developed in SA, and didn't recognise white students, it would be exactly the same problem.

The word "racist" carries overtones of decisions and intent. This is simply a faulty bit of software - it is clearly not working well, and clearly not acceptable. You could make an argument that there is a problem within the software development community that repeatedly leads to problems like that - you could even argue that this is a symptom of systemic racism within that community.

But to say the software is "racist" is, well, kind of silly. Although, that said, everyone knows exactly what he meant, so, big deal? The issue isn't his choice of words, but the software causing him problems.

[gnasher729]

Re: Why use the word racist.

So you are stating now that you can imagine software developed in South Africa to disadvantage white people, and software created in the USA or Western Europe to disadvantage black people? In other words it _is_ racism?

[herman]

Re: Why use the word racist.

In the USA, everything is tied to racism. Religion, clothing, food, hair styles, shampoo, even diseases like Covid19 - are all racist in the USA.

[AndyS]

Re: Why use the word racist.

Well to be fair, you did elect a racist twit. Who has incited race riots, stripped oversight of police violence, put brown children in cages in the desert, refused to address Covid19 appropriately...

So yeah, it's a bit of a shit-show. Turns out the country is, in fact, still full of racists.

[codejunky]

Re: Why use the word racist.

@AndyS

"Well to be fair, you did elect a racist twit"

Helped broker peace between Israel and UAE, hired people regardless of colour, pictured with various people of colour. I aint seeing it.

"Who has incited race riots"

When? Even denounced a group he apparently didnt know much about but were known as racists (proud boys) while Biden refuses to condemn BLM, Antifa etc who are at least half the problem in race riots.

"put brown children in cages in the desert"

Continued the border policies as far as I understood it. Obama sending additional support to the southern border.

"refused to address Covid19 appropriately"

Appropriately being a questionable action. Currently Sweden is performing well economically with the same or better Covid results as more locked down countries. Locked down countries continuing to lock down because the virus spreads more when people are not locked down. Also the man is accused of being some tyrannical dictator yet his opponents want him to enforce a lockdown he has no power to do. A president who has made moves that piss off other countries in a determined effort to get a vaccine available to Americans ASAP.

This morning I was reading how Biden is complaining about the economy under Trump yet praised the economy under Obama. Both after massive economic events (crash/covid) and yet the economy is recovering better under Trump. Of course Biden has to complain politically but factually Trump has done well (even with his lack of political polish)

[jospanner]

Re: Why use the word racist.

White people find accusations of racism more abhorrent than actual racism, a case study.

[James 47]

Re: Why use the word racist.

It's impossible to prove you're not a racist. You may as well accuse people of finding young children sexually attractive. Also, by labelling everything under the sun a racist it really dilutes the seriousness of actual racism.

[Ken Hagan]

Re: Why use the word racist.

Because it discriminates on the basis of skin colour?

I have no problem with the language here. I'm not abou to attribute malice to a computer but if you need a simple word to describe the problem then racist works for me.

[Empire of the Pussycat]

Re: Why use the word racist.

Perhaps it's racist because in spite of similar and well reported problems, of which they must have been aware, they...

apparently failed to train it well enough to avoid the problem

apparently failed to test it well enough to detect the problem

definitely went ahead and used it in a context where the problem could have a negative on the subject

[Prst. V.Jeltz]

Re: Why use the word racist.

well , i suspect he was using the term to hammer home the end result of the experience,

but anyway

how can this person call it racist when it knows nothing about people or colour?

refusing to accept the person in front of you is actually a person because they are a different colour sounds pretty racist to me.

Like our white conquistador ancestors exploring new lands saying things like.

"Oh look old chap , that fuzzy wuzzy looks just like us , except he's clearly not white coloured , and therefore inferior, so its ok to bundle him in a sack and sell him as a slave"

[Robert Grant]

Re: Why use the word racist.

This is an algorithm that has not yet been perfected, how can this person call it racist when it knows nothing about people or colour? For a university student that does not seem like a very intelligent approach, quite the opposite really.

Look, I agree. These articles are generally extremely simplistic, to the point of misidentifying even the most basic problem.

Which is: if this software struggles with darker facial colours, then it shouldn't be in use. It's not the software (all software tries to do things; sometimes with greater or lesser success), it's the universities choosing to use it inappropriately.

[jospanner]

Re: Why use the word racist.

"This is an algorithm that has not yet been perfected, how can this person call it racist when it knows nothing about people or colour? "

A question that answers itself.

[Roland6]

Re: Why use the word racist.

>This is an algorithm that has not yet been perfected

Yes, and from the ExamSoft spokespersons comment: “the vast majority of those who have attempted to complete a mock exam have successfully done so.” it would seem they were totally blind to and dismissive of the bias in their solftware. That is unless the vast majority of candidates includes a substantial number of non-white candidates; but then if that was the case the spokesperson would have used different words....

[Flywheel]

Re: Why use the word racist.

Not yet been perfected? Er, this has been going on since the 1950s

And to quote the article to make a point,

"At the time, in the '50s, the people who were buying cameras were mostly Caucasian people," she says. "And so I guess they didn't see the need for the market to expand to a broader range of skin tones."

[Roland6]

Re: Why use the word racist.

Not yet been perfected?

I wonder if the problem is actually at a lower level ie. I suspect ExamSoft are using a third-party facial recognition engine, as it seems the UK Home Office are experiencing similar problems:

UK passport photo checker shows bias against dark-skinned women

Aside: What I like about this specific article is that at the bottom they give clear details of the method used to create their test data set.

[herman]

IR cameras

Why use only visible light for face recognition?

A computer vision system can use any kind of light. Infrared light doesn't show melanin.

[DavCrav]

Re: IR cameras

"A computer vision system can use any kind of light. Infrared light doesn't show melanin."

Do you have an IR camera on your laptop?

[Ken Shabby]

Re: IR cameras

Yes, Thinkpad P52

[Khaptain]

Re: IR cameras

"A computer vision system can use any kind of light. Infrared light doesn't show melanin"

Algorithm's do not see melanin regardless of the light that is being used..... They see varying values which are can be analysed in order to determine features and attributes. If the values are confused, inconsistent or poorly defined then the algorithm will have great difficulty in making a determination..

The guy in the image also has a beard which adds difficulty as it is almost impossible to define the structure or outline of the face due to it being partly hidden....

I am not defending anyone other than the programmers who have an extremely, extremely difficult problem to resolve.

AI and ML need huge amounts of good input and huge amounts of human intelligence in order to be useful... It is still an "extremely" immature technology, immature in the sense that it is still growing and has a lot to learn..

[Pascal Monett]

"racist technology"

It's not the tech that is racist, it's the idiots who built it who utterly failed to take into account that white people aren't the only people on the planet.

Given how long it's been since we've been reading about these issues, I find it utterly incomprehensible that the problem still exists. Hello ? Train your damn "AI" with pictures of brown people. Can't be that hard, now can it ?

[katrinab]

Re: "racist technology"

Indeed, as far as I can see, the only company that has managed to do that is Apple with their FaceID.

[gnasher729]

Re: "racist technology"

Downvotes because? You don’t like the idea that FaceId works just fine for black people?

[Anonymous Coward]

Re: "racist technology"

"Downvotes because? You don’t like the idea that FaceId works just fine for black people?"

Oh really .... maybe not quite for everyone, even these white skinned people

https://www.newsweek.com/iphone-x-racist-apple-refunds-device-cant-tell-chinese-people-apart-woman-751263

[AndyMulhearn]

Re: "racist technology"

The difficulty with Face ID is that it doesn’t depend only on the kind of limited camera fitted to most laptops, including Apple kit. It uses an additional “TrueDepth” camera working in tandem with the standard camera. And allegedly Face ID won’t come to laptops any time soon - possibly until Apple wind back the ultra thin screen surround fetish - because the camera is currently too big to fit anything other that an iPhone or iPad screen bezel.

So yeah it can be done just not on commodity kit, which doesn’t negate the poor performance of the setup mentioned in the story in any way.

[Anonymous Coward]

Do not use visible light

The cams needs to use IR light, ultrasound, or other invisible waves that does not care about skin color.

I do not understand why racists are calling brown people black. Just go to Africa, skin bleach is a HUGE industry, they do not want to be black, they pay to be brown.

Or maybe the problem is, that people of african descend are afraid to be compared with people from the iddle east, often described as brown ? And they do not want to be compared to africans.

[RichardBarrell]

Needless use of AI to fuck-up a non-problem

When I was a young warthog (when heeee was a young warthoooooooog) they used to have us all sit in a large room and have a human being "watch" us (mostly by the expedient method known as "reading the newspaper").

I can't think of any reason bar stupidity or greed why you shouldn't be able to just pay a human to do the same invigilation job that the AI is failing at. Put 30 cam feeds side by side on the invigilator's screen, have them sit down in front of a wall of faces while they sit and read the newspaper. Job done. You don't even need the invigilators to be able to see all cameras at any given time; if they're flicking through cameras at random, and the exam-takers know this, and the exam-takers don't know whether or not they're being watched at any given moment, then that's just as scary as being constantly watched. Just ask Michael Foucault.

Or... better method... say "fuck it" to exams and grade coursework instead, which is arguably a better approach anyway.

[Mike 137]

"things like glances to something off-screen"

I often look up at the ceiling when I'm thinking, and sometimes I just look around to rest my eyes, particularly if I have to stare at a screen for any length of time. So that makes me an exam cheat?

Until some sensible metrics of what exclusively constitutes cheating are thought up, tools like these will discriminate unfairly on many more grounds than skin colour. People who design such systems should consider the real problem before coming up with the rules.

[mark l 2]

How does such poorly written software ever get released? Do software companies not test their software anymore, or do they just take the Microsoft approach to testing, release it and then recall or patch it when problems start?