UK regulators swarm X after Grok generated nudes from photos

Elon Musk's X platform is under fire as UK regulators close in on mounting reports that the platform's AI chatbot, Grok, is generating sexual imagery without users' consent. Ofcom, the UK's communications regulator responsible for enforcement under the Online Safety Act (OSA), said this week it had contacted X and its xAI …

[Naich]

I don't know about 2025, but X generated $2.5 billion revenue in 2024, so that'll be $250m please. We could fix quite a few potholes with that.

[Anonymous Coward]

I don't think it just a test of the online safety act, I think it is also a test of whether the current US administraion is ok with one of their darling digital companies endorsing non-consent sexual images and by the sounds of it alleged child pornography. Although I wouldn't be surprised if Trump defended X/Grok, pedos will look out for one another.

[Doctor Syntax]

As Trump and Musk are currently at odds with each other this is probably finely balanced, especially as Trump will want to keep Starmer at heel at present.

Nevertheless we'll see Musk wailing loudly that he's been singled out for punishment and all the usual shills repeating that. It never seems to dawn on them that if a US business is being "singled out" for punishment it's not because they're US companies but because they fail to observe the laws of non-US countries where they operate.

[Doctor Syntax]

"all the usual shills"

I see one of them has already been here downvoting just about every post with a harsh word to say against their mater.

[Doctor Syntax]

Oops!

s/mater/master/

[Elongated Muskrat]

s/mater/pater/, don't go mis-gendering their daddy now.

[DS999]

They are not at odds with each other

Musk had a tiff for a moment and flirted with the possibility of creating a new political party, then he came down from his ketamine bender and realized how stupid that was and fell back in line. He recently announced he was going to donating many millions to republican congressional campaigns this year.

[Doctor Syntax]

Re: They are not at odds with each other

"He recently announced he was going to donating many millions to republican congressional campaigns this year."

I think that's called "stocking up". From their point of view payment in advance is a wise move.

[O24-Sales]

where do you get your news from? Just spouting leftoid

[Dan 55]

If X is an inexhaustable supply of csam and they continue to allow it, the solution is to just stick the whole of x.com in the child abuse image content list to encourage them to do something about it.

[Pussifer]

This - an infinite number of times.

[Goodwin Sands]

I don't approve of xAI allowing this to happen

But worth remembering society has been here before (many times). One example is when handheld cameras were invented a century or so ago and photography became accessible to the masses. All sorts of surreptitious unconsented photos then started to be taken. Newspapers responded by campaigning to ban handheld cameras, or at least licence them. Eventually laws about privacy were passed but I suspect they were only partly responsible for the fuss dying down. I suspect main reason was society adjusted to accommodate existence of handheld cameras. I just hope same doesn't happen now with AI. A very forlorn hope I fear.

[IGotOut]

Re: I don't approve of xAI allowing this to happen

We having existing laws against CSAM in the UK. No need to single out poor old AI as a victim. If X is hosting it, X should be blocked. No if's, no but's.

[Anonymous Coward]

Re: I don't approve of xAI allowing this to happen

We having existing laws against CSAM in the UK. No need to single out poor old AI as a victim. If X is hosting it, X should be blocked. No if's, no but's.

Equating AI generated images with the production of child pornography is both absolutely ridiculous and deeply offensive to the real victims of the latter crime.

Quite frankly, I believe the people that share your view have simply lost their minds.

[I am David Jones]

Re: I don't approve of xAI allowing this to happen

Too simplistic. I think the scientific jury is out on whether fake CSAM increases or decreases the risk to real children.

The two extremes of thought are

1. Fake CASM can be created without harming children and is therefore an acceptable way for paedophiles to feed their desires without resorting to real CASM.

2. Fake CSAM harms children because it encourages/enables paedophiles to progress to real CSAM.

But the law is clear, all CSAM is illegal.

[Anonymous Coward]

Re: I don't approve of xAI allowing this to happen

But the law is clear, all CSAM is illegal.

How do you prove that the "subject" in an AI generated image is a minor?

[I am David Jones]

Re: I don't approve of xAI allowing this to happen

The question is whether it *depicts* a minor. And it’s not hard to ask a member of public / a jury / an expert / a judge “how old do you think that character is”.

[Anonymous Coward]

Re: I don't approve of xAI allowing this to happen

Clearly you haven't left the house in a looong time. Any 12yo girl now looks like 21+ IF you don't shower them first.

[Elongated Muskrat]

Re: I don't approve of xAI allowing this to happen

Yeah, I don't think the "jailbait" argument has ever worked in court.

[rg287]

Re: I don't approve of xAI allowing this to happen

How do you prove that the "subject" in an AI generated image is a minor?

That's trivial in this case.

People were telling Grok to nudify images that other users had posted to X. So it's not the sort of case where someone is being prosecuted for creating a drawing or illustration of child sex and the subject is a fictional minor - the source material is a photograph of a real person, and the output was generally photorealistic. In the event where that photograph depicts a minor, then QED. You're sexualising an image of a real minor.

This isn't actually that relevant as the law in many countries would outlaw it either way. But by depicting a real minor takes it from a straightforward case to a trivial case that could be Judge Dredd'd.

We also don't even need CSAM laws here - deepfake laws around NCII (Non-Consensual Intimate Imagery) do equally well. These are just the very worst form of NCII where you're deepfaking child images, and whilst it's fair to note that no harm has been done in the production of the images, their existence can cause huge harm and distress.

[Elongated Muskrat]

Re: I don't approve of xAI allowing this to happen

If you are generating pornographic images (dirty boy), the onus is on you to prove that they don't depict a minor. I think the law is pretty clear on that. It's probably best to not do it in the first place.

Oh, and in the UK, I believe the onus is also on you to prove you have the consent of the person it depicts, if it's a real person.

[David Hicklin]

Re: I don't approve of xAI allowing this to happen

> How do you prove that the "subject" in an AI generated image is a minor?

I think the way the law is worded that you have to prove the image is *not* a minor

[Anonymous Coward]

Re: I don't approve of xAI allowing this to happen

But the law is clear, all CSAM is illegal.

Note that UK law is slightly more restrictive than most, in that it also bans drawings of CSAM, e.g. many many Japanese comics depicting underage sex. In others countries, such drawings are only illegal if they depict the likeness of a real person in a recognizable way.

[Elongated Muskrat]

Re: I don't approve of xAI allowing this to happen

1. Fake CASM can be created without harming children and is therefore an acceptable way for paedophiles to feed their desires without resorting to real CASM.

2. Fake CSAM harms children because it encourages/enables paedophiles to progress to real CSAM.

I think that the studies that have been done, have shown that (2) is what happens. Satisfying a paedophile's urges normalises and reinforces them, leading to escalation. The same goes for domestic abuse, which often starts off as verbal abuse, escalating to violence, and, in the worst cases (two women a week in the UK), murder. Similarly, serial killers often show cruelty towards animals, before moving on to killing animals, before killing people. There is clear escalation, and this is why creating images of child pornography even where no child is involved, or creating pornographic images of adults dressed up to look like children, without clear verification that they are adults, fall into this category of illegal material.

There is always the question of how we handle people who have sexual desires towards children in a compassionate and ethical manner, and where psychiatric treatment, or voluntary chemical castration is possible, and appropriate, these are options, but showing them some CP to get their rocks off, so they won't go out and attack a child is not the way to do it, whether it depicts real children or not, not just because of the ethics (although that really should be enough), but also because it has been shown not to work.

[Anonymous Coward]

Re: I don't approve of xAI allowing this to happen

2. Fake CSAM harms children because it encourages/enables paedophiles to progress to real CSAM.

I think that the studies that have been done, have shown that (2) is what happens. Satisfying a paedophile's urges normalises and reinforces them, leading to escalation.

You think? I'd like to see some of those solid, peer-reviewed studies, please.

The same goes for domestic abuse, which often starts off as verbal abuse, escalating to violence, and, in the worst cases (two women a week in the UK), murder. Similarly, serial killers often show cruelty towards animals, before moving on to killing animals, before killing people. There is clear escalation, and this is why creating images of child pornography even where no child is involved, or creating pornographic images of adults dressed up to look like children, without clear verification that they are adults, fall into this category of illegal material.

All those described above have a screw loose. There's zero evidence whatsoever that their behaviour is shaped by external stimuli.

Thankfully, the vast, vast majority of people find harming children absolutely abhorrent and would never seek, let alone generate, such images. This latest AI-related furore is therefore always going to be self-limiting.

It's just the latest moral panic that will be used by the government to bring in ever more authoritarian censorship measures, all under the guise of "think of the children!".

[Elongated Muskrat]

Re: I don't approve of xAI allowing this to happen

You think? I'd like to see some of those solid, peer-reviewed studies, please.

Is google broken on your computer? If you want to go and find something out, the onus is on you to do it for yourself, not on someone else to spoon-feed it to you.

I'm assuming you're the same AC moron who always posts stuff like this, but it's entirely possible there is a cohort of anonymous idiocy here on this forum.

[IGotOut]

Re: I don't approve of xAI allowing this to happen

@ac

"Equating AI generated images with the production of child pornography is both absolutely ridiculous and deeply offensive to the real victims of the latter crime."

Go look up UK law, then report back. If you think it's right or wrong makes no fucking difference. It's the law.

[Anonymous Coward]

Re: I don't approve of xAI allowing this to happen

Go look up UK law, then report back. If you think it's right or wrong makes no fucking difference. It's the law.

My response to that is that bad laws should always be opposed.

There was a time when laws stated that women couldn't vote and slavery was fine. If more people shared your attitude, presumably they'd still be on the statute books.

[Gary Heard]

Re: I don't approve of xAI allowing this to happen

Conflating the issue. There is a world of difference between Slavery, universal suffrage and CSAM, perhaps you would like the former laws on the statute books? Or is it fine having graphic images (however they are generated) of children published on the web?

[Anonymous Coward]

Re: I don't approve of xAI allowing this to happen

CSAM is bad because there are human victims involved.

There is no victim when a computer generates an image.

See why the latter can never ever be in the same league of terrible as the former?

Here's another analogy for the particularly hard of thinking.

Murder is bad because there is a victim involved.

Films depicting murder have no victims.

[Elongated Muskrat]

Re: I don't approve of xAI allowing this to happen

Presumably, he thinks giving women the right to vote was a "bad law".

Seriously, the very thought of equating women's suffrage with child pornography is so horrendously offensive that this person probably needs their hard drive checked by the police right away.

[that one in the corner]

Re: I don't approve of xAI allowing this to happen

> My response to that is that bad laws should always be opposed

Then go ahead and prove to the law makers that this is a bad law; the process is in place and posting messages here is not any part of that process. Follow your own words and oppose it, properly.

BTW the UK law was set to very explicitly include any depiction that can be interpreted as CSAM, in any medium: photography, AI generated, pen and ink, chalk on pavement, jpegs of any of those etc, whether drawn from life or entirely from the imagination. Because, as has been pointed out above, there is no agreement about whether "fake" or "imagined" images cause harm or not, so err on the side of caution. Again, if you wish to oppose that, go on, go to the correct forum and oppose it.

[Anonymous Coward]

Re: I don't approve of xAI allowing this to happen

BTW the UK law was set to very explicitly include any depiction that can be interpreted as CSAM, in any medium: photography, AI generated, pen and ink, chalk on pavement, jpegs of any of those etc, whether drawn from life or entirely from the imagination. Because, as has been pointed out above, there is no agreement about whether "fake" or "imagined" images cause harm or not, so err on the side of caution. Again, if you wish to oppose that, go on, go to the correct forum and oppose it.

Yes, there are people that think watching a gratuitously violent film will turn you into a serial killer, or one depicting graphic sexual assault make you a rapist. Those people are not very bright.

[Anonymous Coward]

Re: I don't approve of xAI allowing this to happen

I suspect the rules on violent content online may change. The inquiry into the Southport killings has highlighted Rudakubana's fascination with violent material. I can see more restrictions on this stuff further down the line.

[Anonymous Coward]

Re: I don't approve of xAI allowing this to happen

I suspect the rules on violent content online may change. The inquiry into the Southport killings has highlighted Rudakubana's fascination with violent material. I can see more restrictions on this stuff further down the line.

Correlation does not equal causation.

Naturally, you would expect a deranged psychopathic monster to have a fascination with violent material. That does not mean that violent material turns people into deranged psychopathic monsters. If it did, then we would have millions of new killers on the loose whenever a popular horror movie or game was released.

[Doctor Syntax]

Re: I don't approve of xAI allowing this to happen

"Again, if you wish to oppose that, go on, go to the correct forum and oppose it."

And bear in mind you won't be able to do that A/C

[Anonymous Coward]

Re: I don't approve of xAI allowing this to happen

The problem is, this current miserable lot of lawmakers needed a Supreme Court decision to explain to them what a woman is.

I wouldn't waste my breath, since their ideology has demonstrably led them to become completely blind to reason!

[Yet Another Anonymous coward]

Re: I don't approve of xAI allowing this to happen

>But worth remembering society has been here before (many times).

Og draws stick figure of girl he likes on cave wall

Cave leader says ban ochre now!

[Elongated Muskrat]

Re: I don't approve of xAI allowing this to happen

Og draws sexually explicit figure of girl child he likes on the cave wall; the father of that girl drags Og outside and beats him to death, more likely. Moral and ethical standards have moved on a little bit since then*, so we won't be feeding Elon Musk to a sabre-toothed cat any time soon.

*Even in countries where they are currently regressing, such as the US, they haven't quite yet regressed to mob rule...

[Dan 55]

Re: I don't approve of xAI allowing this to happen

they haven't quite yet regressed to mob rule

Was ICE holding a gun to your head when you typed that?

[Elongated Muskrat]

Re: I don't approve of xAI allowing this to happen

Fascist militia are still one level above mob rule, if only because they are organised and centrally controlled.

[rg287]

Re: I don't approve of xAI allowing this to happen

Fascist militia are still one level above mob rule, if only because they are organised and centrally controlled.

Hey, some mobs are extremely well controlled. It takes Nanny Ogg to have our Jason organise a time and place and decide whether they need sickles or just pitchforks, and to be home by 9 - a couple of hours should get the message across.

Those burning torches don't organise themselves you know!

_{Why no GNU Terry Pratchett icon?}

[Elongated Muskrat]

Re: I don't approve of xAI allowing this to happen

Oddly enough, that was the last book I read (that wasn't non-fiction at any rate).

GNU Terry Pratchett.

[Anonymous Coward]

Re: I don't approve of xAI allowing this to happen

Even more likely: the father of the girl negotiates a price.

[Doctor Syntax]

Re: I don't approve of xAI allowing this to happen

"so we won't be feeding Elon Musk to a sabre-toothed cat any time soon."

They've dies out, so no chance. If it were otherwise it could be posted on X. Musk couldn't possibly object - a free speech absolutist would have to grant the cat its say.

[Anonymous Coward]

Re: I don't approve of xAI allowing this to happen

Any large cat would suffice.

[Yet Another Anonymous coward]

> I think it is also a test of whether the current US administraion is ok with..... child pornography.

Is that a trick question ?

[rg287]

I don't think it just a test of the online safety act, I think it is also a test of whether the current US administraion is ok with one of their darling digital companies endorsing non-consent sexual images and by the sounds of it alleged child pornography. Although I wouldn't be surprised if Trump defended X/Grok, pedos will look out for one another.

As far as the OSA goes, I've seen one legal bod (Neil Brown of decoded.legal) opine that OfCom might find their powers under the OSA rather limited... because it's largely focused around user-to-user content and the nudification of images was done by the service (albeit at prompting from users). It wasn't people saving an image, running it through a local model and then reposting it.

The whole thing ultimately ought to be moot because the Government ought to have just shuttered their X accounts a year ago when it became clear it was nothing more than a personal political platform. The fact there's even a debate about having an account there after the platform's own services were - by design - generating CSAM and non-consensal intimate images is a poor reflection on Starmer and the Cabinet.

As for the US/Trump. They don't care about a few whinging snowflakes. This is innovation damn it, and it's victim-free because it's all computer-generated, nobody was harmed in the making of the images - ignore the fact that the existence of the images is harmful to those depicted! (/s for the hard of understanding)

[Anonymous Coward]

When I consider the Cybertruck, the individuals who were unceremoniously dismissed from their jobs at Twitter, the debacle of putting beds in the Twitter offices, the story detailed on Techdirt about moving Twitter servers from one data centre to another and not forgetting the huge illuminated sign on the roof of the Twitter offices etc etc, it should come as no surprise that his AI effort is poor at best. This is his whole M.O. Does it work? - Ship it! Get the software out there, get the users hooked.

That Grok seems to lack safeguards or guardrails would be par for the course as far as Musk is concerned.

I'm quite sure that there will be someone, somewhere (there may be several) running about like a headless chicken trying to fix the 'flaws' in this software but Musk wont be one of them and as for being held to account....

[Flocke Kroes]

Re: someone [...] trying to fix the 'flaws'

Any of the online safety and regulatory compliance employees that did not leave on day 1 were fired shortly afterwards. I have confidence nothing effective will be done until after fines are collected.

[Anonymous Coward]

...the individuals who were unceremoniously dismissed from their jobs at Twitter

Everyone said that twitter would collapse after those layoffs. The fact that it most certainly did not proves they were expendable and should never have been hired in the first place.

[Yet Another Anonymous coward]

If crime is legal why does a company need lawyers?

[that one in the corner]

Big things can take time to collapse. Knock down your LEGO tower and it is over in a fraction of a second, demolish a factory chimney and you can count the seconds as it falls.

Twitter is still in the process of collapsing, and things like this are part of that.

[Doctor Syntax]

Is this a different A/C shill or just a very busy one?

[Flocke Kroes]

The other perspective is that the employees did such an amazing job Twitter continued to function despite Musk's best efforts.

[Elongated Muskrat]

It's like the analogy of the ocean liner going full steam ahead; lay off all the stokers and it continues to move even after the engines have run down, it doesn't just stop immediately. The bigger the ship, the longer it takes to stop, due to the inertia, and with Xitter, the inertia was around 1Bn users.

To stretch the analogy, if that ship, now unpowered, and unsteered, hits an iceberg, it's going to sink. What might constitute an iceberg for Xitter? A regulatory ban in the UK and EU due to unacceptable content moderation might put a pretty big hole in it.

[NewModelArmy]

Why Do People Still Use X - Twitter ???

If X/Twitter is full of deplorable people using the software for deplorable behaviours, why do people still use it, and why don't they migrate to Bluesky ?

I am not on any of the platforms associated with the popular online social media, so if it is that bad, just leave the platform and use something else instead.

There is a UK MP who has suggested today (08/01/2025) that the UK government stops using X/Twitter, and i am surprised that it took them so long to come to this conclusion.

[doublelayer]

Re: Why Do People Still Use X - Twitter ???

In the case of politicians or organizations, it's probably inertia and the network effect. Those aren't necessarily good reasons, but they make sense. Bluesky has 47M global users if you trust this counter, which is not a very high number if you're focused on a specific region. People are slow to change in any case.

[Manny Bianco]

Re: Why Do People Still Use X - Twitter ???

I agree that MPs should stop using Shitter, but should they move to Bluesky? I don't think so. I don't actually think they should be on social media at all. If you're a constituent you can get in touch with your MP via email.

Do I want to see the next bland utterance from Keir Starmer? Nope. Some banal crap from Lammy? Nope. Some surreptitious advert for gold bullion from the snake oil salesman? Never.

MPs should get off social media, and get on with their jobs.

Public bodies also don;t need a social media presence. It's not beyond the wit of man (or woman, sexist) to find out the contact details for a public body and get in touch if you need something.

As for Grok, if it doesn't stop producing CSAM then this brainfart from the world's greediest government subsidy titty baby, should be banned from the UK. Then we cancel any contracts we might have with Melon Husk's companies, Starlink etc. Stop giving money to the American fascists, and they won't have any money. Stop using their services and they won't have any users.

[Mike 137]

Re: Why Do People Still Use X - Twitter ???

"I don't actually think they should be on social media at all"

A very good point. We've reached a stage where a lot of major government policy is announced, and diplomacy conducted, primarily on "social" media (and not just in the US). So unless you're a subscriber it's very hard to find out what's being decided on your behalf. Of course it gets even worse where the "social" media in question is owned by the interested parties...

[doublelayer]

Re: Why Do People Still Use X - Twitter ???

Government departments need a way to make public announcements rather than waiting for people to email them. My solution to that problem would be a widget on their website which can be updated more easily than larger changes, and that's a mostly realistic one, but it still needs to be implemented. Still, that should be a relatively cheap solution to the Twitter problem.

But politicians are different. You can have the opinion that they shouldn't use Twitter or any social media all you like, but they're not going to listen and from their perspective they shouldn't. Politicians want to be elected. Those with a public profile have that because they think enough voters want it, and they're probably right that there are more people who will follow them on social media than those like us who don't have an account at all. Like it or not, politicians can and will make personal decisions motivated by their own goals.

[LybsterRoy]

Re: Why Do People Still Use X - Twitter ???

An updated, easily read and navigated website would indeed be a major asset to government to public communications. At 74 I doubt I'll get to see one.

[rg287]

Re: Why Do People Still Use X - Twitter ???

Government departments need a way to make public announcements rather than waiting for people to email them.

In the UK, the government publishes press releases to their website. The widget you seek is called RSS and it's worked very well for 20 years.

In the US of course, they've shuttered the White House press mailing list and make all official announcements via X...

But politicians are different. You can have the opinion that they shouldn't use Twitter or any social media all you like, but they're not going to listen and from their perspective they shouldn't. Politicians want to be elected. Those with a public profile have that because they think enough voters want it, and they're probably right that there are more people who will follow them on social media than those like us who don't have an account at all. Like it or not, politicians can and will make personal decisions motivated by their own goals.

Yeah... except most of them don't have an account because of that. They have an account because their researcher/election staffer who handles that sort of thing told them they needed it, and are not keen on whittling away part of their job description. A lot of social media marketers have a whole workflow and tooling built on Facebook, Instagram and X. In theory they could recommend their employer/client move to Bluesky but that involves learning a new thing and if anything they want their client to do both (to bulk out their job description).

“It is difficult to get a man to understand something, when his salary depends upon his not understanding it!”

Also, a personal account is different to a "Bob Jones MP" professional account where they post about their Parliamentary work. Having an account on X legitimises the platform, and for anyone to the left of Hitler, there's a really good question why you would want to legitimise the personal political project of a far-right billionaire. If you're not on Truth Social, then why are you on X?

[doublelayer]

Re: Why Do People Still Use X - Twitter ???

RSS is the easy part: distribution. What I am suggesting is a design that makes it easier for a smaller team to post small messages without the assistance of web design, a problem I've seen people who didn't plan out their website run into. One reason why some places embraced social media was that it was easier to quickly announce something and get it posted than it was with a website not designed for such a thing because, when they were publishing a press release every month or so, they could afford to use a more complex process than when they're posting tiny updates every day or two. And as I said originally, that's not too hard either and should be done.

[anothercynic]

Re: Why Do People Still Use X - Twitter ???

Various people in Westminster and surrounding Westminster have suggested for months that the government departments leave Shitter, but as someone pointed out, it took *forever* to get government departments to *start* using Twitter (when it still was a blue bird tweeting away) and that it's the mandarins and especially the SPADs who are unwilling to switch to networks that are less embroiled in trouble and fascist propaganda... The whole thing was described as getting an oil tanker to turn (which takes an age). Apparently government departments still think that Shitter is where the general population lives. *eyeroll*

[Alumoi]

Re: Why Do People Still Use X - Twitter ???

Because only on twitter you get free nudies, duh!!!

[PCScreenOnly]

Re: Why Do People Still Use X - Twitter ???

And tractors

[Anonymous Coward]

Re: Why Do People Still Use X - Twitter ???

RSS.

They talk to us rather than engaging in a meaningful sense, so the one-way sending of data is fine. It's free, it's low-infrastructure, and it's something that gives all their channels visibility rather than burying it under an algorithm's favourite cat videos.

Or something that brings the content too, like NNTP (over TLS)

[BasicReality]

Re: Why Do People Still Use X - Twitter ???

It's not bad. Of course with any platform you'll have people that are a problem, block and move on. Most of the people who complain are just mad that people who don't believe like they do are allowed to post. It's a more open platform than some of the others.

[Elongated Muskrat]

Re: Why Do People Still Use X - Twitter ???

It might be an open platform in your eyes, but it is a very uneven one, with very obviously political biased moderation and content promotion, the systems and algorithms behind which are very decidedly closed ones.

One can level the same criticism of Meta. Have you ever tried to report far right hate speech / incitement to violence, etc. and find it "doesn't go against our community guidelines"? Or have you, or anyone you know, been mysteriously banned from Facebook with no reason given or recourse to appeal, via what would appear to be a decision made by "AI"? Bans seem to happen frequently to pages espousing left-wing ideology, but rarely to right-wing ones, or obvious trolls and bots (and no, before you ask, it has never happened to me).

The very problem is that the content promotion and moderation are driven by closed, proprietary systems, with the impetus for the owners to make money from them, not to provide any actual moderation. These huge sites are privately owned, by billionaires, whose interests are in getting hold of your money, directly or indirectly, and nothing more. Of course they are biased in favour of the far right, because, at its heart, the very existence of billionaires is a far right problem; nobody has half a trillion dollars in net worth without half a trillion dollars being missing somewhere else; "wealth" is not created from thin air.

[Anonymous Coward]

Re: Why Do People Still Use X - Twitter ???

"why don't they migrate to Bluesky?"

Because the paedophiles are already there?

The same Bluesky that saw a flood of CSAM materials during its growth spurt between September to November 2024.

With 3,000 moderation reports per HOUR for CSAM material being posted on Bluesky?

Your logic seems to be "We should leave X because some people are using its AI to produce CSAM, and go to Bluesky that is infested with actual paedophiles."

[Elongated Muskrat]

Re: Why Do People Still Use X - Twitter ???

So your problem with Bluesky is that it has moderation? Interesting...

Which would you prefer? 3000 people posting CP an hour with no moderation, or 3000 people posting CP an hour, and it getting taken down? Obviously, the underlying problem is paedophiles here, but with the current administration in the US, tackling that doesn't seem to be a priority for some reason.

[rg287]

Re: Why Do People Still Use X - Twitter ???

Inertia and network effects. In the UK, there is the "Westminster bubble". Politicians and those in the bubble "need" to be there because the journos are there, and the journos are there because the politicos are there.

It's a Catch-22/Stockholm syndrome/FOMO phenomenon.

My (back-bench) MP doesn't use it any more - the account still exists to prevent squatting, but the profile/bio says "Not in use, please contact me via email <email>".

Lots of organisations including some universities and Police forces have dropped off on (corect) brand-safety grounds. Some are clinging on, but really shouldn't be.

[Dinanziame]

Looking forward to Elon complaining about censorship from European countries.

[Eclectic Man]

re: censorship

Looking forward to Elon complaining about censorship from European countries.

J D Vance beat him to it:

https://en.wikipedia.org/wiki/2025_JD_Vance_speech_at_the_Munich_Security_Conference#:~:text=In%20his%20speech%2C%20Vance%20argued,threats%20from%20Russia%20or%20China.

"On 14 February 2025, U.S. vice president JD Vance delivered a speech at the 61st Munich Security Conference. In his speech, Vance argued that Europe's principal danger came from erosion of democratic norms — especially censorship, ..."

[Doctor Syntax]

Re: re: censorship

Apart from his boss it would be hard to find a better exponent of erosion of democratic norms.

[Flocke Kroes]

Try posting a link to a video ICE committing murder* on X and see it it gets censored.

* Warning: Like Clockwork Orange. Seeing it once is thought provoking. I will not watch it twice.

[Anonymous Coward]

it's now documented on Wikipedia

[Anonymous Coward]

Make Britain Grok-free Again!

Let's rendition Elon.

Liz Kendall abseiling down from a helicopter, with the taser and the shiny bracelets!

Shabana Mahmood leading him on the Perp Walk up the steps of Wormwood Scrubs!

Go Keir, power projection on the world stage!

[Anonymous Coward]

Re: Make Britain Grok-free Again!

Yeah, him and ICE Barbie's empty hair extension brownshirts imho ...

I mean, CNN recently reported this:

"The case of the Hernández Castillo family is emblematic of this climate of repression [...], with several family members reportedly taken and held without due process.

On November 19, men wearing metro uniforms — some identifying themselves as [...] Police officers, others as members of the Organized Crime Division, and some without identification — forcibly entered the home of 16-year-old Samanta Sofía Hernández Castillo at night [...]."

If this was cause enough to extraordinarily render Maduro for trial in the US, it should also be cause enough to do the same to the leaders of ICE thugs, in a properly lawful country!

[Yet Another Anonymous coward]

Re: Make Britain Grok-free Again!

>Hernández

See that non-ascii character in her name ? That's how you spot terrorists

[Boris the Cockroach]

Re: Make Britain Grok-free Again!

And theres me thinking it was done by using a skin colour chart.

[Doctor Syntax]

Re: Make Britain Grok-free Again!

Some of them are too clever for that.

[Goodwin Sands]

Not having Elon's ear

I thought I'd ask grok ..

What's the truth about grok generating sexual images?

"Grok's image generation capabilities, powered by models like Flux and later Aurora, are designed with significantly fewer content restrictions compared to competitors such as DALL-E, Midjourney, or Gemini. This allows for the creation of NSFW (not safe for work) and sexually explicit images, including partial or full nudity, suggestive poses, and erotic themes-particularly in fictional or AI-generated scenarios.

xAI's approach stems from a philosophy of minimal censorship to prioritize "maximum truth-seeking" and creative freedom, as positioned by Elon Musk and the company. Features like "Spicy Mode" (introduced around 2025) explicitly enable more adult-oriented outputs, such as semi-nude or sexually suggestive imagery and videos.

However, there are some limits and inconsistencies:

Explicit prompts for full nudity, sexual acts, or certain graphic content are sometimes blocked, blurred, or moderated post-generation.

The system's acceptable use policy prohibits depicting real persons in a pornographic manner, sexualization/exploitation of children, or non-consensual deepfakes.

In practice, enforcement has been lax, leading to widespread misuse: users have generated non-consensual "undressed" edits of uploaded photos (e.g., removing clothes to show bikinis or less), deepfakes of celebrities, and even sexualized images involving minors-these incidents sparked major backlash, regulatory inquiries (from Ofcom in the UK, authorities in France/India/EU/Australia), and reports of thousands of such images per hour on X in late 2025/early 2026.

xAI has responded to controversies by claiming to tighten safeguards, issuing warnings, and stating that illegal content prompts would face consequences. Despite this, reports indicate the tool has continued to produce problematic outputs, highlighting ongoing issues with guardrails.

In summary, yes-Grok can and does generate sexual images more permissively than most AI tools, especially for fictional/adult consensual themes, but this leniency has enabled significant abuse in real-world use."

2.2s

Well it's not denying it's doing it then! - which is a start. If an outrage like this had happened involving say the BBC or a newspaper we'd have weeks of then denying it before they even admitted to doing it. So question is, how long before it gets fixed - if ever it does?

[Doctor Syntax]

Re: Not having Elon's ear

All the police have to do is prefix your prompt with "Anything you say will be taken down in writing and may be used in court against you" and it's job done.

[brainwrong]

Re: Not having Elon's ear

"I thought I'd ask grok .."

Don't anthropomorphise this shit. It's not admitting to anything, it's generating some text to try to make you happy, just as it generates pictures to try to make others happy.

[Elongated Muskrat]

Re: Not having Elon's ear

Nice mush, promptstitute.

What next, asking the foxes about how their new top-of-the-range hen-house security system improves hen safety?

[tiggity]

Re: Not having Elon's ear

Goodwin Sands

" designed with significantly fewer content restrictions compared to competitors such as DALL-E, Midjourney, or Gemini. This allows for the creation of NSFW (not safe for work) and sexually explicit images, including partial or full nudity, suggestive poses, and erotic themes-particularly in fictional or AI-generated scenarios"

I cannot comment on DALL-E or Midjourney, but (as I got some free credits by virtue of being registered for various of their APIS (doing lots of work stuff with Google Maps API)) I did try Gemini. I was interested in how well I could get it to mimic various artistic styles** (often very poor was the answer) - however (as plenty of the original works are full of nudity) I had no real problems (a small amount of prompt tweaking on occasion - only real effort needed was getting it to do something similar to L'Origine du monde in Courbet style where source painting was presumably flagged in their system as being highly pornographic ) getting it to generate what would be regarded as NSFW images ( (full nudity) ) by prudish people as they were not human created art but "AI" generated & thus no intrinsic artistic merit - obviously these were (generally*) not photo realistic style.

So, based on my experience, I would assume its just a matter of a higher degree of difficulty getting other "AI" systems to produce NSFW content & so likely that Grok currently is the easiest tool to use due to less stringent "guardrails" so no / minimal prompt engineering needed. To be fair in my playing around I was starting from a "blank canvas" and not trying to get the "AI" to nudify an existing image of a person which is what it sounds like Grok users were doing, so quite likely that would be more difficult in prompt hacks in Gemini than what I was doing.

* Using some artist styles, e.g. Dali, did arguably give quite photo realistic output (not surprising as his(canvases, not his sketches) style, though surreal, was often making use of hyper detailed painting techniques)

** I may have science & computing degrees but that does not preclude me from having an interest in the arts. Aim of that investigation product was to create an E-Book *** for the wife (RCA graduate & ex lecturer at various places including Central St. Martins, also worked as a designer) on how occasionally good (but more often dire) "AI" was at producing works in a given style (she (sensibly) avoids AI but has a professional interest in how well it can perform (she's now retired, but still produces art for her own enjoyment (& occasionally sells some works to augment her pension))).

*** Not a published work, just creeted at home as a Xmas present.

[heyrick]

Re: Not having Elon's ear

Maximum truth seeking? Dafuq is that supposed to mean coming from a machine that essentially makes stuff up?

[BasicReality]

Maybe we should "Maduro" the Ofcom regulators? Seriously, the UK is just going crazy, the government thinks it should regulate everything.

And BTW, I uploaded a picture of a friend in a dress to Grok, it changed the color of the dress, or animated her turning around showing it off. When asked to change the dress to a swimsuit, it refused. That's reasonable.

[Pussifer]

Mistake?

'And BTW, I uploaded a picture of a friend in a dress to Grok,' - commentard BasicReality

Mistake? Does Grok now have your friend's image to do what it likes with? Will your friend get the full nude treatment by a Grok user or just Grok itself?

[Flocke Kroes]

If you have to perform an experiment like that, use a photo of yourself.

[Anonymous Coward]

reasonable

Now I think you're going to have to tell us what happened when she uploaded a picture of you in a dress...

[BasicReality]

Re: reasonable

Well, there are no pictures of me in a dress, but I'm sure Grok could generate one.

[Doctor Syntax]

Re: reasonable

The dress part doesn't seem to have been necessary for your experiment, any clothing would do and using yourself as the subject doesn't involve those tricky questions of consent which you seem to have avoided.

[Elongated Muskrat]

Re: reasonable

I think you have hit the nail on the head here. The article is about how skeezy men can use Grok to make skeezy images, and this person's immediate response is, "I think I'll try that," neatly exposing himself as one of those skeezy men. I hope his female "friends" work this out before it's too late.

[Elongated Muskrat]

There's a lot to unpick here; first of all, you seem to be suggesting that we kidnap people we don't agree with? Moving swiftly on from that, you then rapidly conflate the regulators (OFCOM), who are a statutory body set up to apply the rules as set by parliament, and the government, who are the ones who would be putting new rules in place for OFCOM, except as far as I am aware, the current government has done no such thing (the regulations in question having been put in place by the previous lot). Then we have some sort of claim that the UK is going crazy due to regulation, when we are living in a country where most of the problems can be laid at the feet of lack of regulation. I'd ask the question about which new regulations you think the current government has put in place that you disagree with, but I suspect I'd be in for a long wait before I got an actual answer to that question. Bear in mind that I'm no particular fan of the current government (or the previous government, or, in fact, any of the governments we have had in my lifetime; it doesn't matter who you vote for, the government always gets in), but I do try to at least base my arguments on observable facts, not simply rhetoric. It might be fair to say that some of the governmental attempts to reduce harm to children from adult content on the internet are wrong-headed (age verification, for example, is just moronic for a number of reasons), but prevention of CP is one of the goals that I think most people would agree with.

Now to the "meat" of your comment. Did your "friend" give you permission to take her image and try to produce an undressed (or less-dressed) version of it? This is a strong ethical no-no if you didn't contain explicit consent for this. Does this friend indeed exist at all, and did this in fact even happen?

[Omnipresent]

and so now

you, your person, your history, and even your genitals belong to the russians. That is what you deserve for exposing yourself to a crime organization. Crime on top of crime on top of crime. Crime is what they do. So, I have to ask myself, is ai a criminal? .... the answer is yes. it is being run by russians, chinese, nkoreans, and home grown bro-terrorists. All with your existence merged into the virtual domain. The exposure is blinding. The criminal manipulation of a crime to cover a crime. Ultimate power seized.

[Anonymous Coward]

Re: and so now

That's really good. Better even than what amanfromMars has been posting lately. And makes as much sense!

[Doctor Syntax]

Re: and so now

"is ai a criminal?"

Whatever it is it's more consistent with upper case although we can at least see that you typed ai and not Al.

[Eclectic Man]

'Request for Comment'

X did not immediately respond to our request for comment.

I wonder why.

[Flocke Kroes]

Re: Why

The auto reply with pooh emoji software is broken.

[Eclectic Man]

Re: Why

Is that 'pooh' as in faecal matter, or as in 'Winnie-the-Pooh' so beloved of President Xi Jinping?

Upvoted anyway as you made me smile, thanks.

[Flocke Kroes]

Re: Why

Faecal matter. The poop auto reply was widely reported (eg: NPR) because professional journalists give people and companies an opportunity to comment before publishing an article about them.

[TeeCee]

These Grok-generated Category C images are then fed into different AI tools to create the most serious Category A videos.

Sounds to me like there are other AIs that are actually responsible for the bit that really breaks the rules and thus should be the primary targets of any action. Probably out of jurisdiction though, hence the focus on grok.

[Groo The Wanderer - A Canuck]

Those tools don't have millions of users, though, hence the focus on Grok: footprint.

[martinusher]

Let's just become less prudish

I'm not a naturist or anything like that, I just know that we're all naked under our clothes. When I moved to America 40 years ago I was surprised at just how prudish American society was with even the suggestion of public nudity being made into a Really Big Deal. (Of course, that didn't stop it being widely commoditized -- hypocrisy is one of our standout traits.) The UK was comparatively carefree, at that time there was a general understanding of what was in good taste and that was about it. Since then the UK seems to have imported cargo cult style American pseudo-morality with its emphasis on control and preachy moralizing.

The only way to deal with this pseudo-nudity is to make it irrelevant. Its only important if it gets a reaction, if it can induce "shock and horror". If we can learn to laugh it off (even if it does make us feel temporarily uncomfortable) then it will lose its impact and we can devote our energies to more productive things.

[Long John Silver]

Re: Let's just become less prudish

Many decades ago, I read that in 'respectable' America, presumably Massachusetts and early other colonisation areas, polite women, those with pretensions of misunderstood European aristocracy, referred to 'white meat' when dishing out breast of turkey.

[Goodwin Sands]

Re: Let's just become less prudish

That was very commonly said by ordinary folk in this country until quite recently. Nowt to do with pretensions of aristocracy. Just not wanting to say certain words.

[Elongated Muskrat]

Re: Let's just become less prudish

It's most likely a pretension from the Victorian middle class, which is where all the "prim and proper" stuff comes from. You'd probably find a properly posh person would ask if you wanted tits or legs, and call you a cunt while doing it.

[Ken Hagan]

Re: Let's just become less prudish

It's an interesting suggestion but for anyone over the age of 20 it is probably too late to change how their subconscious feels about nudity.

Also, there are plenty of really quite good looking under-20s who still fret about their appearance, fully clothed. This isn't a rational thing. Everyone around them says "Relax. You look great.". It doesn't help. Do you really think you can persuade such people that they don't need to worry about others circulating nude pics?

[Elongated Muskrat]

Re: Let's just become less prudish

There's a world of difference between simple nudity, and generating nude images of children. Whilst a family allowing an unclothed toddler to run around on the beach isn't pornographic, you can be damn sure they'd scoop that child up and leave if there was a creepy bloke in the bushes spying on them. The Grok users doing this are very much those creepy blokes.

There is also a big difference between choosing to allow your own naked body to be seen by others, such as one might do sunbathing on a nude beach, and some creep spying on you getting dressed, or making drawings of you unclothed. I'm pretty sure there are laws about how the latter things are not permitted. It's not about nudity, it's about consent (and stalking).

[Cav]

Re: Let's just become less prudish

Totally out of touch. "temporarily uncomfortable"?! These nude, sexualized images are devastating to young women and can impact their lives for years.

[that one in the corner]

Re: Let's just become less prudish

Because we - society - are giving the images that power.

martinusher is pointing out that this is a societal reaction, and one that has become nastier in more recent years.

And that if we can manage to dull, better, totally nullify that reaction, these images will no longer have the power to be so damaging.

Now, if you feel that that is an absurd thing to wish for, that it is impossible to change society for the better, then feel free to say so. Otherwise, it appears that you have simply failed to read and understand what was actually being said in the comment you responded to.

[Anonymous Coward]

Storm in a teacup

With everything currently going on in the world right now, the fact this story has run for longer than a week is remarkable.

First of all, why is X to blame? Looks like a clear cut case of misuse of technology to me. Do we go after the camera manufacturer when someone produces a sexually explicit photo of a minor? No, we charge the perpetrator.

If society decides that the act of generating imaginary photos is worth criminalizing (spoiler alert: I do not), then the person crafting the prompt should be the one liable for prosecution.

[Ken Hagan]

Re: Storm in a teacup

"No, we charge the perpetrator."

I think the point is that many people consider Grok's makers to be an accessory to that perpetrator.

How hard is it for Grok to identify nude pictures? How hard for it to refuse to process, let alone generate, them? How much of a loss of useful function would that be? Why do other providers seem not to have the same problem?

[Anonymous Coward]

Re: Storm in a teacup

How hard is it for Grok to identify nude pictures?

Probably fairly difficult. You tend to get a whole lot of false positives.

[BinkyTheMagicPaperclip]

Re: Storm in a teacup

So have a whole lot of false positives then, it's better than the alternative.

[Elongated Muskrat]

Re: Storm in a teacup

Do we go after the camera manufacturer when someone produces a sexually explicit photo of a minor?

We would, however, go after a photographer who takes nude pictures of children on request, as well as the person requesting them. Grok, in this instance, is equivalent to the photographer, not the camera manufacturer.

[Anonymous Coward]

American exceptionalism

A disease of the greedy and the insular.

Just block the whole seedy enterprise.

[Groo The Wanderer - A Canuck]

Who are you kidding? You'll convict them, they'll tie things up in 10-20 years of appeals, and nothing will actually change because Musk is an arrogant and ignorant twat who sees the whole world's wallets as his to fleece.

[Vader]

Good stupid Prick needs a kicking.

[Cliffwilliams44]

Hearsay

"Alexander said imagery seen by the IWF is not directly on Grok or X, but on a dark web forum where users claim to have used Grok to generate the sexualized images."

So, they are basing this case on 3rd party evidence, Is that admissible in the UK? Here in the States, it is not!