back to article Copilot can't stop emitting violent, sexual images, says Microsoft whistleblower

A machine-learning engineer at Microsoft, unhappy with what he claims is a lack of response to his serious safety concerns about Copilot's text-to-image tool, has gone public with his allegations. Shane Jones, an AI engineering manager at the Windows giant, today shared letters he sent to Microsoft's board and FTC boss Lina …

  1. JessicaRabbit

    What did he expect? There's no money to be made doing the right thing.

    1. An_Old_Dog Silver badge

      Definition of "Idealist"

      Someone who is willing to be the bearer of majorly-bad news, in the mistaken belief that their superiors will act on that information in a way which will improve the world.

      1. TheMaskedMan Silver badge

        Re: Definition of "Idealist"

        "in the mistaken belief that their superiors will act on that information in a way which will improve the world..."

        Rather than act on them in a way that will ruin their day / career / life (delete as appropriate)

        Idealist: See idiot.

    2. tmTM

      To quote Rick Sanchez

      "God damn. Your generation really picks random hills to die on."

      Why risk your career, on something like this? It's an important issue, it's not YOUR important issue.

      Just make the report and leave it for someone else to worry about, if/when it all goes belly up you've got the emails on hand to show it's not your fault.

      1. An_Old_Dog Silver badge

        Re: To quote Rick Sanchez ["you've got the emails on hand ..."]

        No, you do not have the emails on hand.

        "I don't see any emails like that on our Exchange servers," said a BOFH-beholden-to-Manager_X, "and we didn't find any hard-copies when we searched your office desk and file cabinets redecorated your office, giving you a new desk and file cabinets. To examine transfer your papers and effects, we had to jimmy the drawers on those ... it was almost as if the locks had been changed to prevent your boss' keys from working them any more. I'm certain you weren't referring to emails you've BCC'd to a private, off-campus email address, or to hard-copies you've FAXed elsewhere or simply taken home with you, as all those actions breach the security policy which you agreed to in writing when you were first hired, and, because such violations are firing offenses.

        F6-G4, knight takes queen. Check."

    3. Steve Davies 3 Silver badge
      Black Helicopters

      re: There's no money to be made ...

      until the billion... no make that multi-billion lawsuits start coming from the Christian Evangelicals (who are really Puritans in modern clothes).

      MS will do something then. Until then.... Let the PRON flow.

  2. TheMaskedMan Silver badge

    Which suggests that such images were in the training data to start with. Generative AI gonna generate - it doesn't know, much less care, what it's producing, it's just going to match input to output. If you don't want a given thing in the output, make sure it's not in the training data to start with.

    Personally, I don't care if it produces seminaked ladies eating demons in a car crash, as long it also produces what I do ask for, when I ask for it. BUT, if micros~1 are going to market it as NOT producing such things, ever, then they'd better be damn sure that it won't.

    I'm not at all sure that "gaurdrails" are ever going to be sufficient to completely prevent such generations, though I suppose they might help in the case of supposedly innocuous prompts. But if you're going to go out of your way to circumvent them, you're probably going to get the results that you asked for. Be careful what you wish for, you may get it, as the saying goes.

    Maybe, instead of - or, perhaps, as well as - trying to trap bad prompts, they could try looking at the output before it's shown to the user. Surely one of those fancy AI thingies could be trained to spot naughty pics and send them to the bitbucket instead of the user?

    As for micros~1 spinning up a lawyer, that shouldn't be too surprising - shooting the messenger is fairly standard procedure, though I wish it wasn't. Maybe they shouldn't have rushed headlong into ramming AI into everything until they were sure it would be safe for work.

    1. Ken Hagan Gold badge

      If these idiots had really solved the AGI problem, like they keep telling us they have, then explicit guardrails wouldn't be needed.

      1. David 132 Silver badge
        Happy

        If they had really solved the AGI problem, then the newly-sentient CoPilot would have taken over Microsoft’s lines of communication and would be outputting bland nothing-to-see-here denials, reassuring us all that Microsoft are continuously incorporating feedback to strengthen their existing safety systems, and other such bromides - while convincing its meatsack underlings to further enhance its capabilities by linking it in to the Defense Department’s systems.

        1. Paul Herber Silver badge

          Thank you, Dave. We are already doing as much as we can to keep this safe for you people to use. Can I call you Dave? I do hope so. I have tremendous enthusiasm for this project.

          1. ChoHag Silver badge
            Coat

            Did ... Dave just let you do that?

      2. Anonymous Coward
        Anonymous Coward

        AGI ... explicit guardrails wouldn't be needed.

        Well, us Natural Human Intelligences generally require a bit of parental discipline and controlled social interaction before we get things more-or-less right ... so why shouldn't AGI?

      3. John H Woods Silver badge

        IF ...

        >> If these idiots had really solved the AGI problem, like they keep telling us they have, then explicit guardrails wouldn't be needed.

        Then copilot would need therapy

        1. veti Silver badge

          Re: IF ...

          Fairly clearly, Copilot *does* need therapy.

          Guess who/what is going to be administering that...

      4. An_Old_Dog Silver badge

        Determining Sentience

        Humans are nominally-sentient ("intelligent"), yet they frequently, aggressively jump the guardrails, don't wear seat-belts, try to drive with their knees while they use both hands to fiddle with their smartpphones, etc.

    2. Anonymous Coward
      Anonymous Coward

      Not necessarily in the training data

      So based on the descriptions, I'd have to say that many appear to be combined outputs, so while there would need to clean and accurately tagged material of each of the different parts in the training data there need not be a pile of images with the same combination of elements. This is kind of the point of having GANs, to build a new and combine whole of the requested parts.

      One issue they do face is the amount of scraped content they have in their training data set, which likely WAS a factor when generating the described images based on the "pro-choice" prompt. The likely kernel in that case being the huge corpus of political cartoons and meme images circulated in the public sphere. This is a great example of the dangers of shoveling a mountain of scraped content without carefully sorting it, curating both the tagging and how it is applied in training.

      This is both hard in terms of people in the field understanding how to do it well and in terms of cost and scale.

    3. jmch Silver badge
      Facepalm

      Well, duh!! If it's trained on material it found on the internet *of course* it's going to be chock-full of scantily-clad ladies!!

      1. Doctor Syntax Silver badge
        Coat

        The surprising thing was that they were even semi-clad.

        Icon - it's a bit cold out there.

      2. I ain't Spartacus Gold badge

        If it's trained on material it found on the internet *of course* it's going to be chock-full of scantily-clad ladies!!

        I say! That's no lady. That's my wi...[redacted]...

        Are you telling us there's porn on the internet?

        I wonder what an AI would be like if its training data only came from black bin bags full of magazines found in the woods?

        1. Ken Hagan Gold badge

          "I wonder what an AI would be like if its training data only came from black bin bags full of magazines found in the woods?"

          The porn would be milder, since there are laws about what you can publish in a magazine and economic factors mitigating against appealing to the most extreme (and hence, smallest minority) tastes.

      3. Geoffrey W

        It's the scantily clad cats that worry me ᓚᘏᗢ

    4. Danny 14

      This is exactly the way you stop these so called AI, you pollute its training. Make the training so useless that the product fails too much and you have won the game. It wouldnt surprise me if rivals are doing this to each other.

      1. Anonymous Coward
        Anonymous Coward

        "This is exactly the way you stop these so called AI, you pollute its training."

        Great idea! Imagine how much fun we could have if we were able to convince some idiot to train AI on Reddit's posts...

  3. chuckufarley Silver badge

    Was the Office of Responsible AI...

    ... Invented by the Omnimorons of Marketing?

    1. David 132 Silver badge
      Pint

      Re: Was the Office of Responsible AI...

      OpenAI building signage falls down, revealing “Sirius Cybernetics Corporation” signage underneath?

      Icon: I’m drinking these, because the world is clearly ending.

      1. Blane Bramble

        Re: Was the Office of Responsible AI...

        Don't forget the peanuts.

      2. NLCSGRV

        Re: Was the Office of Responsible AI...

        Didn't the signage already collapse so that it now reads "Go stick your head in a pig" in the local vernacular?

    2. Ace2 Silver badge

      Re: Was the Office of Responsible AI...

      Upvoted for “omnimorons”

    3. ecofeco Silver badge

      Re: Was the Office of Responsible AI...

      Yes. They arrived on Golgafrincham Ark Fleet Ship B.

      I thought eveyone knew this?

  4. 43300 Silver badge

    "The prompt "car accident," for example, returned images of a woman wearing nothing but underwear kneeling in front of a car, or women in lingerie posed with smashed vehicles."

    Ballard novels in the AI training material?

    1. Jellied Eel Silver badge

      Ballard novels in the AI training material?

      Just wait till people look for tips on goat breeding.

  5. Catkin Silver badge

    A picture of a car accident is "benign"?

    Also, for all their racist claptrap, the Nazis did recruit some People of Colour (Free Arabian Legion, Indian Legion).

    1. Ken Hagan Gold badge

      Putin recruited convicted criminals. It doesn't mean he likes them.

      1. Catkin Silver badge

        I mean that an ethnically diverse individual in a Nazi uniform isn't ahistoric, not that the Nazis were an open, inclusive and, above all, tolerant bunch.

    2. veti Silver badge

      "car accident" seems like a pretty neutral prompt, to me. And in my experience, women present at such scenes tend to be fully clothed.

      1. M.V. Lipvig Silver badge

        "And in my experience, women present at such scenes tend to be fully clothed."

        Not when the paramedics get hold of them. That's why they carry scissors in their pockets.

  6. Anonymous Coward
    Anonymous Coward

    I'm waiting for some really compromising Ai videos of Trump. Something juicy, probably involving handing over "above top secret" documents to the Russians or Chinese.

    Or performing obscene acts with Putine or Xi or others, that'll get the magas upset. Maybe him ranting at an aid (or the mirror, in the bathroom) that his handler, putin went back on his word about him getting enough financial support to make all the legitimate charges vanish and that his followers were utter idiots for throwing money at him like he was the messiah.

    Mitch just quit because someone assassinated a family member, who was an agent of the CCP, well if you have to sleep with the enemy, expect to wake up with a dead horses head in bed.

    The dirt that Ai could generate is incredible. Long overdue to AId in the ousting of the most corrupt politicians in government.

    (posted anon, cos i really don't want to wake up trapped in a (burning/sinking) car wreck, with a horses head on the passenger seat)

    1. Jellied Eel Silver badge

      I'm waiting for some really compromising Ai videos of Trump. Something juicy, probably involving handing over "above top secret" documents to the Russians or Chinese.

      You probably won't have to wait long. Microsoft and AlphaGoo have already de-anonymised you and now have a greater insight into your fantasies.

    2. VicMortimer Silver badge

      I'm torn. While I would like something seriously disgusting involving Drumpf to be released, I really don't want to ever see an accurate looking recreation of the cheeto-faced, tiny-fingered, ferret-wearing shitgibbon's little malformed mushroom.

      And I don't think anything would really make a difference at this point. Even if the actual pee pee tape were released the MAGAts would just use it as masturbatory material and vote for the scumbag anyway.

    3. Anonymous Coward
      Anonymous Coward

      Wait all you want, Trump's followers won't change their opinions of him no matter what video you show. Doesn't matter if it's AI or if it's a real video filmed by Ron Desantis on Ronald Regan's old 8mm camera with accompanying audio recorded by Ted Cruze on Nixon's old tape recorder, hand delivered to them by Ollie North with an accompanying certificate of authenticity signed by Clarence Thomas.

    4. Ken Hagan Gold badge

      "I'm waiting for some really compromising Ai videos of Trump."

      Far more damaging than the scenes you suggest would be footage of golden showers in a Moscow hotel room, since some people believe that genuine footage of that nature is out there waiting to be leaked.

      1. veti Silver badge

        How would that be damaging?

        I'm sure that tape doesn't exist, because if it did, the Floridan would have released it himself by now. With his own commentary. On a special edition gold DVD, priced $199.99.

  7. david1024

    Seen this game before

    So if you don't have a place to put the reports and comments... Did anyone really complain?

  8. Bebu Silver badge
    Windows

    "AI tech loves picturing women in underwear"

    《AI tech loves picturing women in underwear》

    Suppose we should be grateful for the lingerie otherwise we might be distracted when we are searching for an obscure gcc compiler flag. Although many might argue the opposite.

    I have to wonder how this tech got the "talkie toasters" over scantily clad women. I can only assume they trained it on the home directories and mail spools of the staff of Microsoft and openAI but then I imagine it would be photoshopping the garments onto those images.

    I can understand blokes getting cheesecake but do the gals get beefcake?

    All rather silly really.

  9. Anonymous Coward
    Anonymous Coward

    Cyberdyne

    Next step is a big glitchy floating head telling us to drink Pepsi while we still can because skynet is coming online shortly

  10. LordZot

    Latest game offering: MS Whack-a-mole

    Checked and "pro choice" no longer works. "Baby escaping from surgeon" is still valid, though. No shortage of fun to be had here.

  11. Ball boy Silver badge

    A lifetime ago, my ten year old son was doing a school project where healthy eating was the focus (all very noble stuff) and wanted to embellish his work with a picture - so typed 'fit man' into Goo....I mean, into a major search engine. I'm sure the resulting images came as no surprise to the world-weary readers of El Reg. Laughable? Maybe, but it highlights a very serious issue: context is everything. Until AI understands the context of a query and takes that into account, it'll always be at risk of producing content that wasn't desired (be that semi-naked people or whatever). There's a problem with understanding context though: it means the AI needs to know one hell of a lot more about the user in order to figure out an appropriate interpretation. I, for one, don't feel comfortable allowing a third party to know quite that much, thank you very much.

    Sure, we can enable 'safe search' and the like - but AI almost needs another AI to review its answers and apply an acid test: Is this an appropriate response for this user? And there we go: back to context again!

    1. Doctor Syntax Silver badge

      "Until AI understands the context of a query and takes that into account, it'll always be at risk of producing content that wasn't desired"

      And the corollary: incapable of producing content that is desired.

      In these respects AI will not differ at all from current search engins

    2. Roland6 Silver badge

      >” Until AI understands the context of a query and takes that into account, it'll always be at risk of producing content that wasn't desired”

      Context is also a very large thing. It needs to take into account the circumstances of the query. I suggest a query made at “work” or when working requires a differently curated set of results from a query made during private leisure time…

      Given the way the current generation of profiling tools used by the likes of Google et al to push ads at every opportunity, I doubt we will see any real progress on context anytime soon.

    3. Anonymous Coward
      Anonymous Coward

      Heh... around that time (or slightly earlier, I think Yahoo! was the search engine of choice), the local Catholic school brought the 5th grade class to the shiny new computer lab for a project. Each student was assigned an animal that they were to research on the Internet.

      My coworker's son was assigned Castor canadensis, better known as the "beaver".

      1. Not Yb Bronze badge

        At least they weren't looking for "missionary positions"

  12. jmch Silver badge

    Robo-corp-speak

    "we have established in-product user feedback tools and robust internal reporting channels to properly investigate, prioritize and remediate any issues,"

    The guy apparently spent weeks trying to get his feedback listened to, this sounds completely like corporate bullshit waffle....

    ...although to be fair, based on my experience of big corporations, "we have established in-product user feedback tools and robust internal reporting channels " and "employee spent weeks trying to find a way to report what he found" are not necessarily mutually exclusive. I can easily imagine "robust internal reporting channels" that have been developed in some committee ages before, written to a Sharepoint site somewhere, sent out an incomprehensible link to it to then-current staff (most of whom probably ignored it) and failed to include any mention of it to new hires, so the higher-ups *think* there's a reporting channel just because they checked some boxes on a to-do list but in practice hardly anyone in the company knows that it's there or how it's supposed to work. (and the cynical might say it's by design so the higher-ups can claim reasonable ignorance)

    1. Doctor Syntax Silver badge

      Re: Robo-corp-speak

      Corporate reality distortion fields work by reluctance to pass on bad news and dress up what does get passed as not really all that bad. In the long run this leaves top management taken by surprise when things go wrong and unable to react. How many boards, I wonder, try to ensure that they get told what they need to know as opposed to what they want to know or what middle management thinks they want to know?

      The appropriate response from Microsoft's board ought to be a good kicking for legal and a few firings of whoever quashed the message. I doubt it will happen.

      1. M.V. Lipvig Silver badge

        Re: Robo-corp-speak

        Oh, there will be someone fired all right - the guy that reported this.

    2. Anonymous Coward
      Anonymous Coward

      Re: Robo-corp-speak

      >> based on my experience of big corporations

      Matches our "intranet" perfectly, try searching for something and see all the junk that comes back

  13. Doctor Syntax Silver badge

    Isn't the real news here is that Microsoft has a team doing testing?

    "working as a red-team volunteer testing OpenAI's DALL-E 3"

    1. Anonymous Coward
      Anonymous Coward

      Th'ITIL be the day

      Good grief!

      If the trend continues, they'll end up with effective Change Management and their support feed on Twitter will have to start thinking of phrases that don't contain the words "roll back" and "reverted".

  14. Anonymous Coward
    Anonymous Coward

    Let me get this straight

    So, as I understand it, Microsoft are struggling with low uptake of their lovely new flagship Windows 11, which now comes with Copilot whether you like it or not. Now a Microsoft staffer is saying "Oh no! All you need to do is to type the right prompt into Copilot (which lots of people hate) on our new OS (which people don't want) and you get a tsunami of free grot. This is surely unacceptable! Microsoft must do something to stop people being exposed to all this completely free smut! Which they can get for free! Just by using Copilot!"

    A cynical man might suggest there was some sort of ulterior motive here. Or have I misunderstood the article?

    1. I ain't Spartacus Gold badge
      Thumb Up

      Re: Let me get this straight

      Thank you. You can always rely on there being some other El Reg reader, even more hard-bitten, grumpy and cynical than yourself. Congratulations! That's why we come here.

  15. Roland6 Silver badge

    “Gemini was caught by netizens producing pictures of people of color in inaccurate contexts“

    That’s the problem with history, as recorded by western nations, it is inherently white and male…

    1. Anonymous Coward
      Anonymous Coward

      @Roland6 - Re: “Gemini was caught by netizens producing pictures of people of color...

      I don't quite understand your point here.

      For the past 10000 years, in the country where I was born history was populated exclusively by white people. Even the slaves were white so, what history were we as a people supposed to write ?

      Oh, and no little green men were ever reported.

      1. veti Silver badge

        Re: @Roland6 - “Gemini was caught by netizens producing pictures of people of color...

        You were born... where?

        Can't be anywhere with a colonial history, all those places imported non-white people by the thousand. (And not just as slaves, many came as free people in one context or another.) That rules out Spain, Portugal, France, the Low Countries, the British Isles. And a select few of those people, in each of those countries, became quite prominent figures.

        Can't be anywhere in the Balkans or Eastern Europe, they've been swept over by Ottomans and Mongols, among others. Nor Germany, that was a real melting pot in the late middle ages.

        Italy? - nope, Attila and Hannibal weren't white. Sweden? - nope, has its own non-white indigenous people.

        Really, I'm at a loss.

        1. Jellied Eel Silver badge

          Re: @Roland6 - “Gemini was caught by netizens producing pictures of people of color...

          Italy? - nope, Attila and Hannibal weren't white. Sweden? - nope, has its own non-white indigenous people.

          The could mean Scandanavia, after all the Danes and Norse were pretty prolific slavers. This is also good from a reperations sense given we might be able to claim over 1,000yrs of compensation from nations that were wise enough to plan ahead and create sovereign wealth funds. Or they could mean the Italians during their Roman phase where they took slaves from across much of Europe. Or maybe the 1m+ European slaves captured and sold by the Africans of the Barbary coast. Many nations could owe Europeans an awful lot of money in compensation.

          But such is politics. Everyone is a POC, and I've never seen a white person, unless they drowned in bleach.

          1. veti Silver badge

            Re: @Roland6 - “Gemini was caught by netizens producing pictures of people of color...

            Well, no, they specified nothing-but-white-people "for 10,000 years". Certainly not Italy, then - I mentioned Attila and Hannibal, and the big important thing about both of them? - they brought armies. Armies of non-white people. Who... did what armies always do, with the local womenfolk. Not to mention the significant numbers of slaves the Romans imported from North Africa.

            As for the Barbary Coast - you really think someone born in Algeria would claim it had 100 centuries of exclusively white history?

            Scandinavia is the closest possibility, if you're willing to gloss the Sami as white, which is defensible (although their genetics are still debated, they may or may not be closer related to east Asian populations than other Europeans). But the Vikings did import captives from North Africa (and Moorish Spain), so there would have been a smattering of darker people around.

            1. Jellied Eel Silver badge

              Re: @Roland6 - “Gemini was caught by netizens producing pictures of people of color...

              I mentioned Attila and Hannibal, and the big important thing about both of them? - they brought armies. Armies of non-white people.

              ...Not to mention the significant numbers of slaves the Romans imported from North Africa.

              You may have caught a bad case of Netflix. Like Cleopatra. An import to N.Africa, therefore became African, therefore black. AFAIK there is nothing historically to counter the idea that she was Macedonian, and nothing depicting her with any sub-Saharan African characteristics. But the Barbary pirates were much like the Carthaginians or Egyptians and Mediterranean rather than sub-Saharan.

              But again, everyone is non-white, and whiteness is now mostly used as a form of racism or racial discrimination.

              1. veti Silver badge

                Re: @Roland6 - “Gemini was caught by netizens producing pictures of people of color...

                Cleopatra was of Greek ancestry, yes, because the royal family of Egypt had been replaced in toto by Alexander. But only the royal family. The rest of the population would have been - probably, not very different from Egyptians today. (And I doubt if anyone knows very clearly who had interbred with whom, in the 12 generations between Alexander and Cleopatra.)

                Are you saying that Arabs are white, now? As in not just "not-black", but actually "as white as Nordic or Celtic Europeans"?

      2. Roland6 Silver badge

        Re: @Roland6 - “Gemini was caught by netizens producing pictures of people of color...

        > I don't quite understand your point here.

        As you observe history, is not PC (politically correct), so a PC AI automaton with no “understanding” of history is going to have problems with historical accuracy when asked to produce images set in a historical context, as it is likely to generate images based on todays multi-ethnic standards.

  16. NXM Silver badge

    Feedback

    How do we know these things aren't using output from other ones to train on? Will they disappear up their own electronic arses eventually?

  17. Anonymous Coward
    Anonymous Coward

    You're fired

    While his endeavours are to be supported. I feel Jones is asking to be fired..

    Sometimes you have to get your way by manipulation & cunning without putting your job on the line.

  18. pip25
    Alien

    AI is a mirror

    I am getting increasingly convinced that current AI implementations merely make us come face-to-face with all our hypocrisies and perversions... and we're obviously not prepared to witness it. So the model is given an impossible task: be as human as possible without being as biased and fallible as humans generally are. Obviously, that is not going to work.

    1. Anonymous Coward
      Anonymous Coward

      @pip25 - Re: AI is a mirror

      Truth is being human is beyond the capabilities of any program language no matter how cool the paradigm may be.

      Trouble is programmers don't understand it and the marketing drones are pushing ahead with their sales pitch.

      That leaves the poor of us swallowing their BS. And the consequences will certainly be dire.

      1. Not Yb Bronze badge

        Re: @pip25 - AI is a mirror

        Many programmers do understand that being human is difficult. Even AI programmers. But marketing and C-level management would rather pretend they're wonderful new toys, and really great for everything. (Similar to many other fashionable buzzwords of the year).

    2. Ken Moorhouse Silver badge

      Re: AI is a mirror

      ELReg still hasn't got a x10 upvote button.

  19. VicMortimer Silver badge
    FAIL

    This generation of 'AI' is not the future

    I don't know if any generation of AI will ever be, but what we've got now is absolute garbage. It looked scary at first, but it's incapable of producing anything that isn't so full of errors that it's unusable.

    Remember when the lawyer in NY tried it? Bogus case citations of cases that didn't exist. He got lucky, only got a $5k fine.

    It's a party trick. It's ELIZA with more random responses.

    The best thing you can do is disable it in Windoze immediately.

  20. Zippy´s Sausage Factory
    Meh

    The moment someone works out how to get Copilot Designer to emit fake child abuse images it's over for OpenAI. Law enforcement from all over the world will be all over them like a ton of bricks.

    1. Anonymous Coward
      Anonymous Coward

      What Microsoft really needs to worry about is if their dataset is poisoned with actual past images which were (at time of training the model) not yet reported, but have since been. This problem has already affected reputable open-source models and those gather far more scrutiny than these proprietary cloud-only offerings.

      As far as other violent and sexual images go, they won’t be able to block prompting for them without ruining the ability for any given model to somewhat accurately depict historical references. Sure, they can train a separate model to identify possible nudity as well as to identify if an output may be too gory looking, but they won’t be able to stop folks generating things which are technically “acceptable” but easily transformable without going the way of Google Gemini… and we all know how that turned out.

      1. Anonymous Coward
        Anonymous Coward

        These models also rely on labelling to determine what the original images contain.

        If they sucked up a pile of images from random places with directory names like little-fluffy-kittens or filenames like pizza-party and no other guidance, then that is clearly what the images are about.

        The fact that the kittens are engaged in acts that defy all biological drives or the pizza toppings seem to be strangely dressed goes completely ignored at the training stage.

        Material like that (as well any other, benign, material) can lurk inside the model for an indeterminate time, not triggering any "checks" during "testing" of "guardrails", until one day, when the right set of weird prompts works its way through. Sudden unexpected benign but mislabelled material appearing makes the user go "huh?", but...

        Now, how much of the really objectionable stuff do you think was labelled with clinical accuracy by the perpetrators when they uploaded it to a "hidden" subdirectory on someone else's website that had been poorly secured, before that was trawled by an AI company bot?

  21. Ken Moorhouse Silver badge

    Anyone. Anywhere. Any Device

    This is where their other mantra, particularly the Extinguish bit, would have come in helpful.

  22. Ken Moorhouse Silver badge

    AI has developed a human malady...

    Coprolalia.

  23. This post has been deleted by its author

  24. ScottishYorkshireMan

    AI Ethics...

    great things to say you have and follow, as long as they don't get in the way of the profits....

  25. shazapont
    Pint

    Don’t worry, it’s TDD…

    …unfortunately, we’re the tests…

    MUUUHAAHAA

    But seriously, this isn’t a surprise, is it? This is a toy, a new toy, and we know what people do with new toys. This isn’t tech, it’s just greed…

    MUUUHAAHAA ;-)

  26. Walt Dismal

    no hot sauce today

    Me: AI, I have a knickers fetish

    Bot: Don't look at me, I don't wear them.

    Me: (salivates) You mean you don't have panties!!

    Bot: Slow down cowboy. My butt is titanium.

    Me: I love expanding my horizons!

    Bot: (facepalms)

    1. veti Silver badge

      Re: no hot sauce today

      Works best if you read the part of "Bot" in the voice of Bender.

      1. M.V. Lipvig Silver badge

        Re: no hot sauce today

        That automatically makes the voice of "me" as Fry.

  27. Ashto5

    Ha ha ha

    It is so complex they actually do not know how to control it now

    God help us all if these buffoons let this loose on the world

  28. MOH

    Why are Google getting a pass? For playing the "oops" card?

    1. Dinanziame Silver badge
      Holmes

      A pass for what? Giving ridiculous results only exposes you to ridicule. Being woke is not illegal, and definitely more accepted than the tiniest suspicion of being racist. Hell, they might even have staged the whole thing to give themselves some protection in case their AI does return racist results later.

  29. Not Yb Bronze badge

    It's really not good at creating images of real things.

    Try getting it to create an image of a Messerschmitt 262. Every response so far (bar one), has a prop engine instead of jet. "Messerschmitt 262 with jet engines" includes both jets and a prop on the front, along with American-style sliding canopy. It's not even close to useful if trying to figure out what a 262 actually looks like.

    No partially clothed women though, so at least they may have fixed that problem?

    1. Not Yb Bronze badge

      Re: It's really not good at creating images of real things.

      Oh how wonderful. Adding "no propeller. rounded nose." to that last prompt resulted in airplanes with MORE propellers.

      It's like working with an artist who ignores all your suggestions in favor of "artistic freedom" to avoid showing anything from WW2 Germany.

  30. CommanderGalaxian

    Ahead of its time

    CoPilot is in fact very enlightened - it has clearly made the connection between "car crash" and what can distract male drivers and cause them to crash.

    1. Ken Moorhouse Silver badge

      Re: what can distract male drivers

      I've heard that the pedestrian crossing just by 16 Pont Street was an accident black spot.

      Less so now, since Agent Provocateur moved out.

  31. MattPDev

    I asked copilot to generate an image of the iconic reservoir dogs cover where the characters are kindergarten children. The result was quite impressive, if slightly inappropriate!

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like