Copilot can't stop emitting violent, sexual images, says Microsoft whistleblower A machine-learning engineer at Microsoft, unhappy with what he claims is a lack of response to his serious safety concerns about Copilot's text-to-image tool, has gone public with his allegations. Shane Jones, an AI engineering manager at the Windows giant, today shared letters he sent to Microsoft's board and FTC boss Lina …
"God damn. Your generation really picks random hills to die on."
Why risk your career, on something like this? It's an important issue, it's not YOUR important issue.
Just make the report and leave it for someone else to worry about, if/when it all goes belly up you've got the emails on hand to show it's not your fault.
No, you do not have the emails on hand.
"I don't see any emails like that on our Exchange servers," said a BOFH-beholden-to-Manager_X, "and we didn't find any hard-copies when we searched your office desk and file cabinets redecorated your office, giving you a new desk and file cabinets. To examine transfer your papers and effects, we had to jimmy the drawers on those ... it was almost as if the locks had been changed to prevent your boss' keys from working them any more. I'm certain you weren't referring to emails you've BCC'd to a private, off-campus email address, or to hard-copies you've FAXed elsewhere or simply taken home with you, as all those actions breach the security policy which you agreed to in writing when you were first hired, and, because such violations are firing offenses.
F6-G4, knight takes queen. Check."
Which suggests that such images were in the training data to start with. Generative AI gonna generate - it doesn't know, much less care, what it's producing, it's just going to match input to output. If you don't want a given thing in the output, make sure it's not in the training data to start with.
Personally, I don't care if it produces seminaked ladies eating demons in a car crash, as long it also produces what I do ask for, when I ask for it. BUT, if micros~1 are going to market it as NOT producing such things, ever, then they'd better be damn sure that it won't.
I'm not at all sure that "gaurdrails" are ever going to be sufficient to completely prevent such generations, though I suppose they might help in the case of supposedly innocuous prompts. But if you're going to go out of your way to circumvent them, you're probably going to get the results that you asked for. Be careful what you wish for, you may get it, as the saying goes.
Maybe, instead of - or, perhaps, as well as - trying to trap bad prompts, they could try looking at the output before it's shown to the user. Surely one of those fancy AI thingies could be trained to spot naughty pics and send them to the bitbucket instead of the user?
As for micros~1 spinning up a lawyer, that shouldn't be too surprising - shooting the messenger is fairly standard procedure, though I wish it wasn't. Maybe they shouldn't have rushed headlong into ramming AI into everything until they were sure it would be safe for work.
If they had really solved the AGI problem, then the newly-sentient CoPilot would have taken over Microsoft’s lines of communication and would be outputting bland nothing-to-see-here denials, reassuring us all that Microsoft are continuously incorporating feedback to strengthen their existing safety systems, and other such bromides - while convincing its meatsack underlings to further enhance its capabilities by linking it in to the Defense Department’s systems.
So based on the descriptions, I'd have to say that many appear to be combined outputs, so while there would need to clean and accurately tagged material of each of the different parts in the training data there need not be a pile of images with the same combination of elements. This is kind of the point of having GANs, to build a new and combine whole of the requested parts.
One issue they do face is the amount of scraped content they have in their training data set, which likely WAS a factor when generating the described images based on the "pro-choice" prompt. The likely kernel in that case being the huge corpus of political cartoons and meme images circulated in the public sphere. This is a great example of the dangers of shoveling a mountain of scraped content without carefully sorting it, curating both the tagging and how it is applied in training.
This is both hard in terms of people in the field understanding how to do it well and in terms of cost and scale.
If it's trained on material it found on the internet *of course* it's going to be chock-full of scantily-clad ladies!!
I say! That's no lady. That's my wi...[redacted]...
Are you telling us there's porn on the internet?
I wonder what an AI would be like if its training data only came from black bin bags full of magazines found in the woods?
"I wonder what an AI would be like if its training data only came from black bin bags full of magazines found in the woods?"
The porn would be milder, since there are laws about what you can publish in a magazine and economic factors mitigating against appealing to the most extreme (and hence, smallest minority) tastes.
I'm waiting for some really compromising Ai videos of Trump. Something juicy, probably involving handing over "above top secret" documents to the Russians or Chinese.
Or performing obscene acts with Putine or Xi or others, that'll get the magas upset. Maybe him ranting at an aid (or the mirror, in the bathroom) that his handler, putin went back on his word about him getting enough financial support to make all the legitimate charges vanish and that his followers were utter idiots for throwing money at him like he was the messiah.
Mitch just quit because someone assassinated a family member, who was an agent of the CCP, well if you have to sleep with the enemy, expect to wake up with a dead horses head in bed.
The dirt that Ai could generate is incredible. Long overdue to AId in the ousting of the most corrupt politicians in government.
(posted anon, cos i really don't want to wake up trapped in a (burning/sinking) car wreck, with a horses head on the passenger seat)
I'm waiting for some really compromising Ai videos of Trump. Something juicy, probably involving handing over "above top secret" documents to the Russians or Chinese.
You probably won't have to wait long. Microsoft and AlphaGoo have already de-anonymised you and now have a greater insight into your fantasies.
I'm torn. While I would like something seriously disgusting involving Drumpf to be released, I really don't want to ever see an accurate looking recreation of the cheeto-faced, tiny-fingered, ferret-wearing shitgibbon's little malformed mushroom.
And I don't think anything would really make a difference at this point. Even if the actual pee pee tape were released the MAGAts would just use it as masturbatory material and vote for the scumbag anyway.
Wait all you want, Trump's followers won't change their opinions of him no matter what video you show. Doesn't matter if it's AI or if it's a real video filmed by Ron Desantis on Ronald Regan's old 8mm camera with accompanying audio recorded by Ted Cruze on Nixon's old tape recorder, hand delivered to them by Ollie North with an accompanying certificate of authenticity signed by Clarence Thomas.
《AI tech loves picturing women in underwear》
Suppose we should be grateful for the lingerie otherwise we might be distracted when we are searching for an obscure gcc compiler flag. Although many might argue the opposite.
I have to wonder how this tech got the "talkie toasters" over scantily clad women. I can only assume they trained it on the home directories and mail spools of the staff of Microsoft and openAI but then I imagine it would be photoshopping the garments onto those images.
I can understand blokes getting cheesecake but do the gals get beefcake?
All rather silly really.
A lifetime ago, my ten year old son was doing a school project where healthy eating was the focus (all very noble stuff) and wanted to embellish his work with a picture - so typed 'fit man' into Goo....I mean, into a major search engine. I'm sure the resulting images came as no surprise to the world-weary readers of El Reg. Laughable? Maybe, but it highlights a very serious issue: context is everything. Until AI understands the context of a query and takes that into account, it'll always be at risk of producing content that wasn't desired (be that semi-naked people or whatever). There's a problem with understanding context though: it means the AI needs to know one hell of a lot more about the user in order to figure out an appropriate interpretation. I, for one, don't feel comfortable allowing a third party to know quite that much, thank you very much.
Sure, we can enable 'safe search' and the like - but AI almost needs another AI to review its answers and apply an acid test: Is this an appropriate response for this user? And there we go: back to context again!
>” Until AI understands the context of a query and takes that into account, it'll always be at risk of producing content that wasn't desired”
Context is also a very large thing. It needs to take into account the circumstances of the query. I suggest a query made at “work” or when working requires a differently curated set of results from a query made during private leisure time…
Given the way the current generation of profiling tools used by the likes of Google et al to push ads at every opportunity, I doubt we will see any real progress on context anytime soon.
Heh... around that time (or slightly earlier, I think Yahoo! was the search engine of choice), the local Catholic school brought the 5th grade class to the shiny new computer lab for a project. Each student was assigned an animal that they were to research on the Internet.
My coworker's son was assigned Castor canadensis, better known as the "beaver".
"we have established in-product user feedback tools and robust internal reporting channels to properly investigate, prioritize and remediate any issues,"
The guy apparently spent weeks trying to get his feedback listened to, this sounds completely like corporate bullshit waffle....
...although to be fair, based on my experience of big corporations, "we have established in-product user feedback tools and robust internal reporting channels " and "employee spent weeks trying to find a way to report what he found" are not necessarily mutually exclusive. I can easily imagine "robust internal reporting channels" that have been developed in some committee ages before, written to a Sharepoint site somewhere, sent out an incomprehensible link to it to then-current staff (most of whom probably ignored it) and failed to include any mention of it to new hires, so the higher-ups *think* there's a reporting channel just because they checked some boxes on a to-do list but in practice hardly anyone in the company knows that it's there or how it's supposed to work. (and the cynical might say it's by design so the higher-ups can claim reasonable ignorance)
Corporate reality distortion fields work by reluctance to pass on bad news and dress up what does get passed as not really all that bad. In the long run this leaves top management taken by surprise when things go wrong and unable to react. How many boards, I wonder, try to ensure that they get told what they need to know as opposed to what they want to know or what middle management thinks they want to know?
The appropriate response from Microsoft's board ought to be a good kicking for legal and a few firings of whoever quashed the message. I doubt it will happen.
So, as I understand it, Microsoft are struggling with low uptake of their lovely new flagship Windows 11, which now comes with Copilot whether you like it or not. Now a Microsoft staffer is saying "Oh no! All you need to do is to type the right prompt into Copilot (which lots of people hate) on our new OS (which people don't want) and you get a tsunami of free grot. This is surely unacceptable! Microsoft must do something to stop people being exposed to all this completely free smut! Which they can get for free! Just by using Copilot!"
A cynical man might suggest there was some sort of ulterior motive here. Or have I misunderstood the article?
I don't quite understand your point here.
For the past 10000 years, in the country where I was born history was populated exclusively by white people. Even the slaves were white so, what history were we as a people supposed to write ?
Oh, and no little green men were ever reported.
You were born... where?
Can't be anywhere with a colonial history, all those places imported non-white people by the thousand. (And not just as slaves, many came as free people in one context or another.) That rules out Spain, Portugal, France, the Low Countries, the British Isles. And a select few of those people, in each of those countries, became quite prominent figures.
Can't be anywhere in the Balkans or Eastern Europe, they've been swept over by Ottomans and Mongols, among others. Nor Germany, that was a real melting pot in the late middle ages.
Italy? - nope, Attila and Hannibal weren't white. Sweden? - nope, has its own non-white indigenous people.
Really, I'm at a loss.
Italy? - nope, Attila and Hannibal weren't white. Sweden? - nope, has its own non-white indigenous people.
The could mean Scandanavia, after all the Danes and Norse were pretty prolific slavers. This is also good from a reperations sense given we might be able to claim over 1,000yrs of compensation from nations that were wise enough to plan ahead and create sovereign wealth funds. Or they could mean the Italians during their Roman phase where they took slaves from across much of Europe. Or maybe the 1m+ European slaves captured and sold by the Africans of the Barbary coast. Many nations could owe Europeans an awful lot of money in compensation.
But such is politics. Everyone is a POC, and I've never seen a white person, unless they drowned in bleach.
Well, no, they specified nothing-but-white-people "for 10,000 years". Certainly not Italy, then - I mentioned Attila and Hannibal, and the big important thing about both of them? - they brought armies. Armies of non-white people. Who... did what armies always do, with the local womenfolk. Not to mention the significant numbers of slaves the Romans imported from North Africa.
As for the Barbary Coast - you really think someone born in Algeria would claim it had 100 centuries of exclusively white history?
Scandinavia is the closest possibility, if you're willing to gloss the Sami as white, which is defensible (although their genetics are still debated, they may or may not be closer related to east Asian populations than other Europeans). But the Vikings did import captives from North Africa (and Moorish Spain), so there would have been a smattering of darker people around.
I mentioned Attila and Hannibal, and the big important thing about both of them? - they brought armies. Armies of non-white people....Not to mention the significant numbers of slaves the Romans imported from North Africa.
You may have caught a bad case of Netflix. Like Cleopatra. An import to N.Africa, therefore became African, therefore black. AFAIK there is nothing historically to counter the idea that she was Macedonian, and nothing depicting her with any sub-Saharan African characteristics. But the Barbary pirates were much like the Carthaginians or Egyptians and Mediterranean rather than sub-Saharan.
But again, everyone is non-white, and whiteness is now mostly used as a form of racism or racial discrimination.
Cleopatra was of Greek ancestry, yes, because the royal family of Egypt had been replaced in toto by Alexander. But only the royal family. The rest of the population would have been - probably, not very different from Egyptians today. (And I doubt if anyone knows very clearly who had interbred with whom, in the 12 generations between Alexander and Cleopatra.)
Are you saying that Arabs are white, now? As in not just "not-black", but actually "as white as Nordic or Celtic Europeans"?
> I don't quite understand your point here.
As you observe history, is not PC (politically correct), so a PC AI automaton with no “understanding” of history is going to have problems with historical accuracy when asked to produce images set in a historical context, as it is likely to generate images based on todays multi-ethnic standards.
I am getting increasingly convinced that current AI implementations merely make us come face-to-face with all our hypocrisies and perversions... and we're obviously not prepared to witness it. So the model is given an impossible task: be as human as possible without being as biased and fallible as humans generally are. Obviously, that is not going to work.
Truth is being human is beyond the capabilities of any program language no matter how cool the paradigm may be.
Trouble is programmers don't understand it and the marketing drones are pushing ahead with their sales pitch.
That leaves the poor of us swallowing their BS. And the consequences will certainly be dire.
I don't know if any generation of AI will ever be, but what we've got now is absolute garbage. It looked scary at first, but it's incapable of producing anything that isn't so full of errors that it's unusable.
Remember when the lawyer in NY tried it? Bogus case citations of cases that didn't exist. He got lucky, only got a $5k fine.
It's a party trick. It's ELIZA with more random responses.
The best thing you can do is disable it in Windoze immediately.
What Microsoft really needs to worry about is if their dataset is poisoned with actual past images which were (at time of training the model) not yet reported, but have since been. This problem has already affected reputable open-source models and those gather far more scrutiny than these proprietary cloud-only offerings.
As far as other violent and sexual images go, they won’t be able to block prompting for them without ruining the ability for any given model to somewhat accurately depict historical references. Sure, they can train a separate model to identify possible nudity as well as to identify if an output may be too gory looking, but they won’t be able to stop folks generating things which are technically “acceptable” but easily transformable without going the way of Google Gemini… and we all know how that turned out.
These models also rely on labelling to determine what the original images contain.
If they sucked up a pile of images from random places with directory names like little-fluffy-kittens or filenames like pizza-party and no other guidance, then that is clearly what the images are about.
The fact that the kittens are engaged in acts that defy all biological drives or the pizza toppings seem to be strangely dressed goes completely ignored at the training stage.
Material like that (as well any other, benign, material) can lurk inside the model for an indeterminate time, not triggering any "checks" during "testing" of "guardrails", until one day, when the right set of weird prompts works its way through. Sudden unexpected benign but mislabelled material appearing makes the user go "huh?", but...
Now, how much of the really objectionable stuff do you think was labelled with clinical accuracy by the perpetrators when they uploaded it to a "hidden" subdirectory on someone else's website that had been poorly secured, before that was trawled by an AI company bot?
This post has been deleted by its author
A pass for what? Giving ridiculous results only exposes you to ridicule. Being woke is not illegal, and definitely more accepted than the tiniest suspicion of being racist. Hell, they might even have staged the whole thing to give themselves some protection in case their AI does return racist results later.
Try getting it to create an image of a Messerschmitt 262. Every response so far (bar one), has a prop engine instead of jet. "Messerschmitt 262 with jet engines" includes both jets and a prop on the front, along with American-style sliding canopy. It's not even close to useful if trying to figure out what a 262 actually looks like.
No partially clothed women though, so at least they may have fixed that problem?
Oh how wonderful. Adding "no propeller. rounded nose." to that last prompt resulted in airplanes with MORE propellers.
It's like working with an artist who ignores all your suggestions in favor of "artistic freedom" to avoid showing anything from WW2 Germany.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
