back to article Unis turn to webcam-watching AI to invigilate students taking exams. Of course, it struggles with people of color

AI software designed to monitor students via webcam as they take their tests – to detect any attempts at cheating – sometimes fails to identify the students due to their skin color. Products like ExamSoft are being used by colleges and other organizations to make sure students aren’t cheating when they take exams in front of …

  1. Anonymous Coward
    Anonymous Coward

    But does the AI detect that someone one else in the next room has plugged into the screen and keyboard ports on the pc?

    As soon as you leave physical invigilated space your exam is compromised completely rather than just somewhat (you've no idea what attempts to cheat are made even in physical invigilated spaces)

    1. John Jennings Silver badge

      Actually, they likely can in many cases.

      I recently went through the Pearson/Vue system with webcams. Its not just the web-cam, but an app that tunnels into some sort of VM... They can and do detect pretty much anything running on your machine. You have to disable screen sharing apps like teams or dameware - and they detect these proactively - , and require other machines in the vacinity to not only be turned off, but covered up with a blanket.

      You have to take multiple pictures with your webcam of your surroundings - detailing much of your surroundings, as well as your ID etc.

      A chat window pops up from time to time which needs answering (at least it did before the test, I dont know if it does it during it, it didnt for me)

      I believe Pearson/vue have human invigilators watching your video - but I wouldnt be suprised if they did some machine learning as well... The chat windows seemed real enough - but it could have been a touring test!

      1. Tom 7 Silver badge

        Re: Actually, they likely can in many cases.

        Presumably if you run it in VM you can fool them into thinking that your computer is not what it is and share the non-vmed screen with as many as you want.

        Also does it work on Linux?

  2. Neil Barnes Silver badge
    Terminator

    How will Toyota be able to tell

    That their robots aren't cheating?

    1. AndyS

      Re: How will Toyota be able to tell

      Simple, they'll paint them all white.

  3. Khaptain Silver badge

    Why use the word racist.

    This is an algorithm that has not yet been perfected, how can this person call it racist when it knows nothing about people or colour? For a university student that does not seem like a very intelligent approach, quite the opposite really.

    1. Luiz Abdala

      Re: Why use the word racist.

      That's the funny part. The sample base to train the AI was not large enough.

      Hanlon's razor: "Never attribute to malice that which could be adequately explained by stupidity."

      1. HildyJ Silver badge
        FAIL

        Re: Why use the word racist.

        Was the company or its programmers racist - no.

        Was the software racist - yes.

        Racism requires intent. Racist does not.

        If the software resulted in racially biased results, intent is not an issue. It is racist because it yields different results for different races.

    2. chivo243 Silver badge

      Re: Why use the word racist.

      Poor programming... not racist

      1. gnasher729 Silver badge

        Re: Why use the word racist.

        If you were black and cheated out of your exam because of your skin colour, I’m sure you would agree that it is racist.

        Racist or not isn’t necessarily a matter of intent, but if it happens due to incompetence or thoughtlessness it’s just as bad.

        1. Cederic Silver badge

          Re: Why use the word racist.

          I would report a technical issue and invite an alternate approach - e.g. a human invigilator.

          If nothing else the person sat at the table doesn't know the underlying reason for the issue. Maybe it's not skin colour, it could just struggle with beards.

          (I do actually agree it was likely to be skin colour, not least because I'm a photographer that finds it much harder to capture the features of some people than others purely because of camera sensor contrast constraints. That doesn't make it racist.)

      2. katrinab Silver badge
        Megaphone

        Re: Why use the word racist.

        There is direct racism, where you believe that people of a particular race are inferior to people of a different race, and want to actively harm them in some way.

        There is indirect racism, where you treat everyone the same, but in a way that adversely affects some people because of their race.

        People with darker skin were unable to take exams because a facial recognition algorithm couldn't recognise their faces. That is indirect racism.

        1. John Jennings Silver badge

          Re: Why use the word racist.

          Not necessarily - cameras are simply not good enough to pick out the lower contrasts you often find in people of colour. Or, there are bright spots from reflections which breaks the array of facial modelling... Ye canne break the laws of physics of a camera - that does not make it racist - especially if the camera is only capable of 1-2 megapixels. Its what is technically possible at this time. What is racist (perhaps) is not offering an alternative verification if the system is known to be weak with somone

          1. fajensen Silver badge

            Re: Why use the word racist.

            Wouldn't IR illumination work, technically at least?

    3. katrinab Silver badge
      Flame

      Re: Why use the word racist.

      Because it discriminates against people with dark skin. Whether they intentionally set out to do this, or just didn't bother to test on different skin colours, it doesn't matter. People with dark skin are adversely impacted, therefore it is racist. Racism doesn't require intent.

      1. DavCrav Silver badge

        Re: Why use the word racist.

        "Racism doesn't require intent."

        So light is racist because it doesn't reflect as much off a black guy's face?

        1. gnasher729 Silver badge

          Re: Why use the word racist.

          You are racist for coming up with all kinds of stupid excuses and straw men.

          1. DavCrav Silver badge

            Re: Why use the word racist.

            "You are racist for coming up with all kinds of stupid excuses and straw men."

            You mean stupid excuses like contrast? And anyway, the guy seemed to think an algorithm, which isn't alive, is racist, because racism doesn't require either intent or life signs. Since light treats white and black people differently, I want to know whether it is racist.

            1. Dan 55 Silver badge
              Boffin

              Re: Why use the word racist.

              Imagine if you knocked up a website without accessibility which stopped blind or disabled people from using it. This is discrimination. You might never have thought to do add accessibility to the website but it was your job to do so and you didn't do it.

              Now imagine you're training an algorithm and choose a dataset which wasn't large or varied enough for it to do the job required, and produces suboptimal or wrong results for non-white people. It was your job to ensure this didn't happen but you didn't. This is also at the very least discrimination.

              Light is obviously not racist, you bleeding fool.

              1. DavCrav Silver badge

                Re: Why use the word racist.

                "Light is obviously not racist, you bleeding fool."

                Yes, it's called reductio ad absurdam.

                Of course light isn't racist. But light has differing effects with regards white and black people, and in particular makes it much harder to make out black faces in dark conditions. If humans (who are very good at pattern recognition) have more trouble in low-light conditions with black faces than white ones, why it is clear that a computer should be able to treat both equally well in higher-light conditions?

                The point is that racism is meant to be where decisions taken by people have a deleterious consequence for one or more racial group. After many, many examples of algorithms having difficulty with black faces, perhaps it's time to start thinking that, maybe, algorithms have a harder time distinguishing black faces because it is an objectively harder problem?

                If that is the case, then suddenly it looks unseemly to throw words like 'racist' around to describe the perhaps completely blameless programmer (because, as you have stated, non-people cannot be racist, so if the algorithm is racist that means the people who programmed it must be).

                Repeating the 'these computer programmers are racist' mantra/libel is starting to become tedious, when the evidence is stacking up that it is just more difficult. Of course, evidence is not a proof, but it should be fairly easy to check. Use a million-strong dataset of white faces and a million-strong dataset of black faces. Then evaluate the performance of the algorithm on a new selection. I've not heard of such an experiment being conducted, but it would go a long way to settling this argument.

                And, here's the interesting question: what happens if it turns out that these algorithms will always be worse at distinguishing black faces? You know, like humans are.

                1. nichomach

                  Re: Why use the word racist.

                  And that's why reductio ad absurdum is treated as a logical fallacy. Scenario: You are developing a technology to invigilate exams. This depends upon facial recognition. The people being invigilated may be black or white, but you only test on white people. Your product discriminates against black people because it can't recognise them. You didn't *intend* to adversely affect them, but you should damn well have tested your product to make sure that it didn't, and if you were unable to ensure that it didn't, you should not have certified that product as ready for use. That is casual racism; you do not intend to be racist, but you are, because you either do not consider the issue, or ignore it.

                  1. stiine Silver badge
                    Facepalm

                    Re: Why use the word racist.

                    "The people being invigilated may be black or white..."

                    This is about as wrong as a statement can get without stating that true = false.

                    Humans can be, and are, nearly every shade of brown/pink/yellow from white to black, regardless of their race.

                    Also, someone said that developer was racist for not choosing a valid image training dataset. That's a very unhealthy assumption considering that several of these datasets have recently been pulled because of their contents, or lack their of.

                2. Dan 55 Silver badge

                  Re: Why use the word racist.

                  After many, many examples of algorithms having difficulty with black faces, perhaps it's time to start thinking that, maybe, algorithms have a harder time distinguishing black faces because it is an objectively harder problem?

                  No, it's time to start choosing better datasets.

                  And, here's the interesting question: what happens if it turns out that these algorithms will always be worse at distinguishing black faces? You know, like humans are.

                  Fuck me, we have to start from the very basics:

                  Cross-race effect

                3. JetSetJim Silver badge

                  Re: Why use the word racist.

                  > perhaps it's time to start thinking that, maybe, algorithms have a harder time distinguishing black faces because it is an objectively harder problem?

                  Perhaps it is, but then perhaps people should also stop releasing products that claim to do something, but don't work so well for the people of colour that make up large portions of society in general. Perhaps these algorithms are not yet ready for the prime time and need further work?

              2. codejunky Silver badge

                Re: Why use the word racist.

                @Dan 55

                "Imagine if you knocked up a website without accessibility which stopped blind or disabled people from using it. This is discrimination. You might never have thought to do add accessibility to the website but it was your job to do so and you didn't do it."

                No it isnt. That is not discrimination at all full stop no way. Discrimination would be to purposefully stop someone from using the website, say intentionally making it impossible for disabled people to use with the intent of stopping them.

                If you accidentally nudge someone or back into someone that is called an accident. A mistake. If you thump someone that is assault. You might hurt someone with both actions but only one of them is malevolent, intent matters.

                I look forward to the 'god is racist' for making people different etc (I am not a believer in such a being but I expect some idjit to come out with it).

                1. Jimmy2Cows Silver badge

                  Re: Why use the word racist.

                  As has already been mentioned, discrimination does not require intent.

                  Failing to add accessiblity features to a website resulting in an exclusionary experience for visually impaired etc. It's this exclusion which makes the website discriminatory, whether or not that was the authors' intent.

                  Same for AI failing to recognise black people more than white people. That (probably) wasn't the authors' intent, but when it disproportionally affects black people, that is discrimination. In this case racial discrimination. Does need to be planned. It's enough for it simply to occur.

                  Given how often this situation crops up whenever AI is used for facial recognition, in any context, ExamSoft were naive at best to consider it a viable, non-discriminatory solution.

                  1. codejunky Silver badge

                    Re: Why use the word racist.

                    @Jimmy2Cows

                    "As has already been mentioned, discrimination does not require intent."

                    Dan 55 substituted the word discrimination for racism, so in the context of racism it does require intent to be racist. Racism is an intentional act. You are right about discrimination not requiring intent.

                    'Failing to add accessiblity features to a website resulting in an exclusionary experience for visually impaired etc.'

                    True just as running is exclusionary to people unable to run. Or a more useful observation, increased melanin content results in reduced vitamin D. Every day normality that people are different and a standardised system cannot apply to everybody, so we try to make them fit as many as possible.

                    "Given how often this situation crops up whenever AI is used for facial recognition, in any context, ExamSoft were naive at best to consider it a viable, non-discriminatory solution."

                    Given how often this situation crops up you would have thought someone would have tried to make it work for coloured people. Yet all of these algorithms seem to have the same inherent problems. As someone else pointed out, he had a beard. I wonder how it would cope with some of the fashionable moustaches.

                    1. nichomach

                      Re: Why use the word racist.

                      Discrimination on the basis of race IS racism. There isn't any extra bar to leap to get there. This is just casual unthinking racism.

                      1. This post has been deleted by a moderator

                        1. jospanner

                          Re: Why use the word racist.

                          Yeah engaging with this is pointless, this is just a sad little nerd who doesn't like the idea that the world doesn't revolve around him.

                          Structural racism is a thing, i'm sorry that you're too much of an intellectual coward to confront this.

                      2. Roland6 Silver badge

                        Re: Why use the word racist.

                        >This is just casual unthinking racism.

                        Aka Unconscious bias.

                    2. Dan 55 Silver badge
                      Stop

                      Re: Why use the word racist.

                      I did not substitute, as the word "racism" is so loaded and it's difficult to have a rational debate about it, I tried to explain in a way that shows that as discrimination does not require intent (because that argument is generally accepted) neither does discrimination concerning race (aka racism) require intent. It's known as "casual racism", look it up.

                      As someone else pointed out, he had a beard. I wonder how it would cope with some of the fashionable moustaches.

                      Facial recognition algorithms are now being trained with masks to recognise people while wearing masks. This itself should give you a hint that solving problems with facial recognition algorithms and race is not impossible and the problem is not down to race itself, so we can't just shrug our shoulders and say "oh, it's because they're black, this is impossible to solve and they should just put up with their life being shittier because of it" or "he has a beard" or whatever. The algorithm must be trained with more images of black people's faces, more images of bearded faces, and more black people's faces with beards so that it reflects real life.

                      1. codejunky Silver badge

                        Re: Why use the word racist.

                        @Dan 55

                        "I did not substitute, as the word "racism" is so loaded and it's difficult to have a rational debate about it"

                        My mistake, so you were discussing something different (discrimination) where I assumed your meaning to be racism. My mistake.

                        "I tried to explain in a way that shows that as discrimination does not require intent (because that argument is generally accepted) neither does discrimination concerning race (aka racism) require intent"

                        This is why it sounds like you used substitution because racism does require intent. Racism is feelings, prejudice the very things that require intent (on the actor or the receiver). Discrimination is the norm that people are different. Racism is to believe in superiority due to difference.

                        "casual racism"

                        Which still requires the belief in superiority over another race. Without that it doesnt exist.

                        "This itself should give you a hint that solving problems with facial recognition algorithms and race is not impossible and the problem is not down to race itself"

                        Awesome, so do it. If it is so easy then go ahead and show all these boffins who have tried and failed how to do it. Remember this must work with varying quality of camera with highly varying surroundings. how good are these mask trained AI's with colour? If its good enough maybe half the face needs cutting off during training.

                        "The algorithm must be trained with more images of black people's faces, more images of bearded faces, and more black people's faces with beards so that it reflects real life."

                        And you will still have people fail. It is a statistical probability model so some people will fail the detection.

                        The other possibility is that nobody does the exam because of covid restrictions and so screw the lot? Or is it best for the majority to test with this technology and those it cant work with have to do it a different way? Covid wont wait for more AI testing, its here. And the education system needs solutions, even imperfect ones.

                        1. Dan 55 Silver badge

                          Re: Why use the word racist.

                          My mistake, so you were discussing something different (discrimination) where I assumed your meaning to be racism. My mistake.

                          From the Cambridge dictionary:

                          discrimination

                          treating a person or particular group of people differently, especially in a worse way from the way in which you treat other people, because of their skin colour, sex, sexuality, etc.:

                          It is indeed your mistake.

                          Discrimination is the norm that people are different. Racism is to believe in superiority due to difference.

                          So, er, racism is a kind of discrimination, then? Yes. Are we there yet?

                          If it is so easy then go ahead and show all these boffins who have tried and failed how to do it. Remember this must work with varying quality of camera with highly varying surroundings. how good are these mask trained AI's with colour? If its good enough maybe half the face needs cutting off during training.

                          They've tried and failed to do it because they've shoved in a dataset which doesn't have enough examples, not checked the results enough, and put it into production.

                          The other possibility is that nobody does the exam because of covid restrictions and so screw the lot? Or is it best for the majority to test with this technology and those it cant work with have to do it a different way? Covid wont wait for more AI testing, its here. And the education system needs solutions, even imperfect ones.

                          That penalises people because they are black. Well done you. You do understand where this leads, don't you? Actually you're probably blissfully unaware, and I'm not willing to argue about it for the rest of the day. As always, the George Bernard Shaw quote applies.

                          1. This post has been deleted by a moderator

                            1. jospanner

                              Re: Why use the word racist.

                              For those following along at home, this is textbook alt-reich nonsense: "Never play defensive".

                              This person has literally been confronted with the idea that this software doesn't work properly due to faulty data, and therefore discriminates based on race. He instantly deflects and makes it the parent post's problem, and then deflects against with more bad faith questions.

                              Such people are not swayed by facts - the basic idea that "discrimination requires no intent" has been beaten to death at this point and is quite self evident with even a moment's thought - the purpose is to waste your time and create a reaction.

                              This is the behaviour of a nasty little psychopath and nothing more.

                              1. codejunky Silver badge

                                Re: Why use the word racist.

                                @jospanner

                                "He instantly deflects and makes it the parent post's problem, and then deflects against with more bad faith questions."

                                Eh? The word described to this situation was racism. And Dan 55 used the word discrimination to which I pointed out the difference. That isnt deflecting, that is pointing out that racism is a part of discrimination (bad one) but discrimination itself is more than just racism. Dan 55 confused the discussion because he thought they were the same thing causing Jimmy2Cows to correctly correct me about discrimination because Dan 55 substituted discrimination for racism (which I was apparently correct to assume).

                                "the basic idea that "discrimination requires no intent""

                                Actually that is the point Jimmy2Cows made because Dan 55 substituted the word discrimination for racism. Basically Dan 55 was wrong and I corrected him. That you repeat that here shows you didnt read my post before a moderator removed it.

                                1. Dan 55 Silver badge

                                  Re: Why use the word racist.

                                  The moderador has corrected you three times so far, but don't let that deter you, do carry on.

                                  1. codejunky Silver badge

                                    Re: Why use the word racist.

                                    @Dan 55

                                    "The moderador has corrected you three times so far"

                                    No they havnt. So far they have removed 3 of my comments but at no point has a moderator said why. Considering the topic and the rioters/thugs using racial issues currently I expect they are erring on the side of caution.

                                    So far you have confused the discussion because now people are correcting the definition of discrimination because you substituted it for racism.

                                2. Man inna barrel Bronze badge

                                  Re: Why use the word racist.

                                  The algorithm has flaws, and does not perform well with dark coloured skins. That is not racism. It may just be a technical problem to be fixed. Perhaps the flaws were caused by training the algorithm on an inadequate data set. That is carelessness, but not racism.

                                  The algorithm was presumably tested, before deployment. Somebody evidently did not test for performance on faces with coloured skin. That is beyond carelessness. Perhaps the flaws were discovered, but thought "acceptable". That is a judgement, not an omission, which makes it racist.

                                  1. codejunky Silver badge

                                    Re: Why use the word racist.

                                    @Man inna barrel

                                    It might even be possible that the software was tested with coloured skin and worked under the 'ideal' test situations. If this is to work on personal machines then the hardware will be different as will the surroundings. Such would be impossible to test for all possibilities.

                                    I dont think we have enough information to call racism but some people have such a low bar to call something racist that they consider your first paragraph to constitute racism.

                                    As you probably read, for some the bar is so low that discrimination at all is considered racism.

                      2. uccsoundman

                        Re: Why use the word racist.

                        Or you could do like a company I worked for. Beards messed up the security cameras, so they implemented a new rule; everybody had to shave off their beard. Don't like to be clean shaven? Find a job elsewhere. This was really hard on the men who (due to medical issues) could not shave their beards. They had a hard time finding new work. This also begged the question of the men who wear beards for religious reasons. This was never an issue for us. The company was in the Bible belt and very few men of those religions lived there.

                2. Roland6 Silver badge

                  Re: Why use the word racist.

                  >No it isnt. That is not discrimination at all full stop no way.

                  In the UK it is - such a website would not comply with the requirements of the Equality Act 2010 (which replaced the Disability Discrimination Act 1995).

                  Any one designing and/or developing webpages and/or websites in the UK would be expected know that their work must take account of this Act. So "I forgot" isn't a defense.

              3. Mike 137 Silver badge

                Re: Why use the word racist.

                "Imagine if you knocked up a website without accessibility which stopped blind [...] people from using it."

                We don't have to imagine. Huge numbers of web sites do fail in this respect.

            2. codejunky Silver badge

              Re: Why use the word racist.

              @DavCrav

              "because racism doesn't require either intent or life signs"

              This is one of the best tricks of making everything racist- intent doesnt matter. Intent would indicate malice but since that is mostly not an issue bar extremists and imported prejudice it is difficult to argue racism all the time.

              Is it a broken AI system, yes. A statistical model system making mistakes is going to happen, and certain features may cause it issues (I wonder how it works for disfigured people?). That is a normal response to this situation. 'RACIST!' is the response if we strip away thinking and ignore intent.

              Note- it would then potentially be racist to stop people failing the statistical model incorrectly from taking the test although probably under different observation.

              1. jospanner

                Re: Why use the word racist.

                And who says that Britain has a race problem?

                1. codejunky Silver badge

                  Re: Why use the word racist.

                  @jospanner

                  "And who says that Britain has a race problem?"

                  Oddly enough people who dont get out much it seems. The UK is one of the least racist countries in Europe.

                  1. Anonymous Coward
                    1. codejunky Silver badge

                      Re: Why use the word racist.

                      @AC

                      Bar the sources being the Guardian and Independent, even if we accept them at face value that still does not refute my comment. Even a 2019 EU study found the UK second least racist after Malta-

                      https://fra.europa.eu/sites/default/files/fra_uploads/fra-2019-being-black-in-the-eu-summary_en.pdf

      2. Khaptain Silver badge

        Re: Why use the word racist.

        @katrinab is using the Flamebait icon so I would presume that the comment is actually stabbing at the ridiculous usage of the word "rascist" which has completely lost it's original meaning.

        It's as though is has become counterproductive as it no longer defines a definitive situation, and with no definitive situation we can longer take any positive action as we no longer no what we are trying to resolve.

        It now appears to having become defined as "someone that does not does not agree with my ideology, regardless of how ridiculous the concepts of my ideology".. and I do not see how that can help those that actually suffer from racism as it was previously defined.

        1. jospanner

          Re: Why use the word racist.

          Discrimination does not require intent.

          No amount of Daily Heil reading changes this fact.

          1. codejunky Silver badge

            Re: Why use the word racist.

            @jospanner

            "Discrimination does not require intent."

            But racism does

            1. jospanner

              Re: Why use the word racist.

              No, it does not. Just because it hurts your feelings doesn't make it incorrect.

              Racism is the discrimination based on race. Intent comes nowhere near this definition. Whether you have a bias towards doing shitty things to black people is the important thing, not how much you intended to do it.

              1. codejunky Silver badge

                Re: Why use the word racist.

                @jospanner

                "Racism is the discrimination based on race"

                racism

                /ˈreɪsɪz(ə)m/

                prejudice, discrimination, or antagonism directed against a person or people on the basis of their membership of a particular racial or ethnic group, typically one that is a minority or marginalized.

                the belief that different races possess distinct characteristics, abilities, or qualities, especially so as to distinguish them as inferior or superior to one another.

                Quick search yields the answer. Directed against. Inferiority/superiority. This requires intent. Yes the software can discriminate on race, but without the developed intention to exclude on race it would seem a mistake. But since AI in general seems to suffer when analysing high melanin content it is more probable to be a physical difficulty instead of an intended one.

                Discrimination is about recognising differences. Racism is a bad part of discrimination but is not the same thing. I have mentioned before that melanin content has an effect on people medically, particularly reduced vitamin D. Noting the difference is discriminatory because the needs may be different, but we wouldnt call that racism as there is no derogatory aspect (we aint thinking one is superior to the other).

    4. gnasher729 Silver badge

      Re: Why use the word racist.

      Isn’t it strange that white people are not affected? I’m sure the algorithm could be calibrated so it affects only white people, not black people. So you would be fine with that, right?

      1. AndyS

        Re: Why use the word racist.

        Nobody is fine with it, stop being obtuse. If the software was developed in SA, and didn't recognise white students, it would be exactly the same problem.

        The word "racist" carries overtones of decisions and intent. This is simply a faulty bit of software - it is clearly not working well, and clearly not acceptable. You could make an argument that there is a problem within the software development community that repeatedly leads to problems like that - you could even argue that this is a symptom of systemic racism within that community.

        But to say the software is "racist" is, well, kind of silly. Although, that said, everyone knows exactly what he meant, so, big deal? The issue isn't his choice of words, but the software causing him problems.

        1. gnasher729 Silver badge

          Re: Why use the word racist.

          So you are stating now that you can imagine software developed in South Africa to disadvantage white people, and software created in the USA or Western Europe to disadvantage black people? In other words it _is_ racism?

    5. herman Silver badge

      Re: Why use the word racist.

      In the USA, everything is tied to racism. Religion, clothing, food, hair styles, shampoo, even diseases like Covid19 - are all racist in the USA.

      1. AndyS

        Re: Why use the word racist.

        Well to be fair, you did elect a racist twit. Who has incited race riots, stripped oversight of police violence, put brown children in cages in the desert, refused to address Covid19 appropriately...

        So yeah, it's a bit of a shit-show. Turns out the country is, in fact, still full of racists.

        1. codejunky Silver badge

          Re: Why use the word racist.

          @AndyS

          "Well to be fair, you did elect a racist twit"

          Helped broker peace between Israel and UAE, hired people regardless of colour, pictured with various people of colour. I aint seeing it.

          "Who has incited race riots"

          When? Even denounced a group he apparently didnt know much about but were known as racists (proud boys) while Biden refuses to condemn BLM, Antifa etc who are at least half the problem in race riots.

          "put brown children in cages in the desert"

          Continued the border policies as far as I understood it. Obama sending additional support to the southern border.

          "refused to address Covid19 appropriately"

          Appropriately being a questionable action. Currently Sweden is performing well economically with the same or better Covid results as more locked down countries. Locked down countries continuing to lock down because the virus spreads more when people are not locked down. Also the man is accused of being some tyrannical dictator yet his opponents want him to enforce a lockdown he has no power to do. A president who has made moves that piss off other countries in a determined effort to get a vaccine available to Americans ASAP.

          This morning I was reading how Biden is complaining about the economy under Trump yet praised the economy under Obama. Both after massive economic events (crash/covid) and yet the economy is recovering better under Trump. Of course Biden has to complain politically but factually Trump has done well (even with his lack of political polish)

          1. jospanner

            Re: Why use the word racist.

            White people find accusations of racism more abhorrent than actual racism, a case study.

            1. James 47

              Re: Why use the word racist.

              It's impossible to prove you're not a racist. You may as well accuse people of finding young children sexually attractive. Also, by labelling everything under the sun a racist it really dilutes the seriousness of actual racism.

    6. Ken Hagan Gold badge

      Re: Why use the word racist.

      Because it discriminates on the basis of skin colour?

      I have no problem with the language here. I'm not abou to attribute malice to a computer but if you need a simple word to describe the problem then racist works for me.

    7. Empire of the Pussycat

      Re: Why use the word racist.

      Perhaps it's racist because in spite of similar and well reported problems, of which they must have been aware, they...

      apparently failed to train it well enough to avoid the problem

      apparently failed to test it well enough to detect the problem

      definitely went ahead and used it in a context where the problem could have a negative on the subject

    8. Prst. V.Jeltz Silver badge

      Re: Why use the word racist.

      well , i suspect he was using the term to hammer home the end result of the experience,

      but anyway

      how can this person call it racist when it knows nothing about people or colour?

      refusing to accept the person in front of you is actually a person because they are a different colour sounds pretty racist to me.

      Like our white conquistador ancestors exploring new lands saying things like.

      "Oh look old chap , that fuzzy wuzzy looks just like us , except he's clearly not white coloured , and therefore inferior, so its ok to bundle him in a sack and sell him as a slave"

      1. This post has been deleted by a moderator

    9. Robert Grant Silver badge

      Re: Why use the word racist.

      This is an algorithm that has not yet been perfected, how can this person call it racist when it knows nothing about people or colour? For a university student that does not seem like a very intelligent approach, quite the opposite really.

      Look, I agree. These articles are generally extremely simplistic, to the point of misidentifying even the most basic problem.

      Which is: if this software struggles with darker facial colours, then it shouldn't be in use. It's not the software (all software tries to do things; sometimes with greater or lesser success), it's the universities choosing to use it inappropriately.

    10. jospanner

      Re: Why use the word racist.

      "This is an algorithm that has not yet been perfected, how can this person call it racist when it knows nothing about people or colour? "

      A question that answers itself.

    11. Roland6 Silver badge

      Re: Why use the word racist.

      >This is an algorithm that has not yet been perfected

      Yes, and from the ExamSoft spokespersons comment: “the vast majority of those who have attempted to complete a mock exam have successfully done so.” it would seem they were totally blind to and dismissive of the bias in their solftware. That is unless the vast majority of candidates includes a substantial number of non-white candidates; but then if that was the case the spokesperson would have used different words....

    12. Flywheel Silver badge
      FAIL

      Re: Why use the word racist.

      Not yet been perfected? Er, this has been going on since the 1950s

      And to quote the article to make a point,

      "At the time, in the '50s, the people who were buying cameras were mostly Caucasian people," she says. "And so I guess they didn't see the need for the market to expand to a broader range of skin tones."

      1. Roland6 Silver badge

        Re: Why use the word racist.

        Not yet been perfected?

        I wonder if the problem is actually at a lower level ie. I suspect ExamSoft are using a third-party facial recognition engine, as it seems the UK Home Office are experiencing similar problems:

        UK passport photo checker shows bias against dark-skinned women

        Aside: What I like about this specific article is that at the bottom they give clear details of the method used to create their test data set.

  4. herman Silver badge

    IR cameras

    Why use only visible light for face recognition?

    A computer vision system can use any kind of light. Infrared light doesn't show melanin.

    1. DavCrav Silver badge

      Re: IR cameras

      "A computer vision system can use any kind of light. Infrared light doesn't show melanin."

      Do you have an IR camera on your laptop?

      1. Ken Shabby

        Re: IR cameras

        Yes, Thinkpad P52

    2. Khaptain Silver badge

      Re: IR cameras

      "A computer vision system can use any kind of light. Infrared light doesn't show melanin"

      Algorithm's do not see melanin regardless of the light that is being used..... They see varying values which are can be analysed in order to determine features and attributes. If the values are confused, inconsistent or poorly defined then the algorithm will have great difficulty in making a determination..

      The guy in the image also has a beard which adds difficulty as it is almost impossible to define the structure or outline of the face due to it being partly hidden....

      I am not defending anyone other than the programmers who have an extremely, extremely difficult problem to resolve.

      AI and ML need huge amounts of good input and huge amounts of human intelligence in order to be useful... It is still an "extremely" immature technology, immature in the sense that it is still growing and has a lot to learn..

  5. Pascal Monett Silver badge
    Flame

    "racist technology"

    It's not the tech that is racist, it's the idiots who built it who utterly failed to take into account that white people aren't the only people on the planet.

    Given how long it's been since we've been reading about these issues, I find it utterly incomprehensible that the problem still exists. Hello ? Train your damn "AI" with pictures of brown people. Can't be that hard, now can it ?

    1. katrinab Silver badge
      Unhappy

      Re: "racist technology"

      Indeed, as far as I can see, the only company that has managed to do that is Apple with their FaceID.

      1. gnasher729 Silver badge

        Re: "racist technology"

        Downvotes because? You don’t like the idea that FaceId works just fine for black people?

        1. Anonymous Coward
          Anonymous Coward

          Re: "racist technology"

          "Downvotes because? You don’t like the idea that FaceId works just fine for black people?"

          Oh really .... maybe not quite for everyone, even these white skinned people

          https://www.newsweek.com/iphone-x-racist-apple-refunds-device-cant-tell-chinese-people-apart-woman-751263

      2. AndyMulhearn

        Re: "racist technology"

        The difficulty with Face ID is that it doesn’t depend only on the kind of limited camera fitted to most laptops, including Apple kit. It uses an additional “TrueDepth” camera working in tandem with the standard camera. And allegedly Face ID won’t come to laptops any time soon - possibly until Apple wind back the ultra thin screen surround fetish - because the camera is currently too big to fit anything other that an iPhone or iPad screen bezel.

        So yeah it can be done just not on commodity kit, which doesn’t negate the poor performance of the setup mentioned in the story in any way.

  6. Anonymous Coward
    Anonymous Coward

    Do not use visible light

    The cams needs to use IR light, ultrasound, or other invisible waves that does not care about skin color.

    I do not understand why racists are calling brown people black. Just go to Africa, skin bleach is a HUGE industry, they do not want to be black, they pay to be brown.

    Or maybe the problem is, that people of african descend are afraid to be compared with people from the iddle east, often described as brown ? And they do not want to be compared to africans.

  7. RichardBarrell

    Needless use of AI to fuck-up a non-problem

    When I was a young warthog (when heeee was a young warthoooooooog) they used to have us all sit in a large room and have a human being "watch" us (mostly by the expedient method known as "reading the newspaper").

    I can't think of any reason bar stupidity or greed why you shouldn't be able to just pay a human to do the same invigilation job that the AI is failing at. Put 30 cam feeds side by side on the invigilator's screen, have them sit down in front of a wall of faces while they sit and read the newspaper. Job done. You don't even need the invigilators to be able to see all cameras at any given time; if they're flicking through cameras at random, and the exam-takers know this, and the exam-takers don't know whether or not they're being watched at any given moment, then that's just as scary as being constantly watched. Just ask Michael Foucault.

    Or... better method... say "fuck it" to exams and grade coursework instead, which is arguably a better approach anyway.

  8. Mike 137 Silver badge

    "things like glances to something off-screen"

    I often look up at the ceiling when I'm thinking, and sometimes I just look around to rest my eyes, particularly if I have to stare at a screen for any length of time. So that makes me an exam cheat?

    Until some sensible metrics of what exclusively constitutes cheating are thought up, tools like these will discriminate unfairly on many more grounds than skin colour. People who design such systems should consider the real problem before coming up with the rules.

  9. mark l 2 Silver badge

    How does such poorly written software ever get released? Do software companies not test their software anymore, or do they just take the Microsoft approach to testing, release it and then recall or patch it when problems start?

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2021