OpenAI's ChatGPT crawler can be tricked into DDoSing sites, answering your queries

OpenAI's ChatGPT crawler appears to be willing to initiate distributed denial of service (DDoS) attacks on arbitrary websites, a reported vulnerability the tech giant has yet to acknowledge. In a write-up shared this month via Microsoft's GitHub, Benjamin Flesch, a security researcher in Germany, explains how a single HTTP …

[b0llchit]

Self inflicted wounds

This is what you get when you let AI ^_(help) write your software. There are no thoughts spent on whether it is a good or smart thing to do. Lets face it, it will kill us all in the end without remorse, like the best and well developed automated psychopath you can find.

All hail to the AI. May the AI kill us all. All hail to the AI. May the wait be soon over. All hail to the AI. Luckily it'll die when the power eventually fails after we're gone.

[sabroni]

Re: All hail to the AI. May the AI kill us all.

You know it's just another bit of software, right? It might cause trouble on the network but there's very little chance of it breaking in to our houses in the middle of the night and murdering our families while they sleep.

[Timop]

Re: All hail to the AI. May the AI kill us all.

Combine it with the previous fad that similarly has S for security in it's acronym: IoT. That has slowly creeped into many homes by this point for example in a form called Tuya (cheap wifi connected smart home stuff).

[Richard 12]

Re: All hail to the AI. May the AI kill us all.

SWATing is a well-known thing.

So yes, it can break into your house and murder your family while you sleep, by convincing humans to do that final step.

That's the proximate risk of AI. It convinces otherwise reasonable humans to do terrible things by feeding them false information.

It also provides some humans with cover to do the terrible things they wanted to do.

[Rich 11]

Re: All hail to the AI. May the AI kill us all.

It convinces otherwise reasonable humans to do terrible things by feeding them false information.

You mean, like a religion? But humanity has long since learned how to deal with.... oh.

It also provides some humans with cover to do the terrible things they wanted to do.

OK, then, definitely like a religion.

[Gene Cash]

Re: All hail to the AI. May the AI kill us all.

> very little chance of it breaking in to our houses in the middle of the night and murdering our families while they sleep.

Don't give Microsoft any more ideas!

[Anonymous Coward]

Re: All hail to the AI. May the AI kill us all.

Yeah, they already do this with aid of the US DoJ. ( https://www.justice.gov/opa/pr/court-authorized-operation-disrupts-worldwide-botnet-used-peoples-republic-china-state ) And this isn't the first time. I expect they'll go it alone if the DoJ actually tells them to fuck off, like they should have been doing from the start.

"Shouldn't it have recognized that victim.com/1 and victim.com/2 point to the same website victim.com..."

Whoever wrote this statement has never used a load balancer, and should probably not be permitted to make statements to the press again.

[doublelayer]

Re: All hail to the AI. May the AI kill us all.

Tell me, how does a load balancer make one domain name point to multiple websites? As I'm sure you know, what it does is allow that website to be hosted on multiple servers. It does not change the fact that both of those addresses are going to be handled by the same cluster, at least at first. A flood of requests can swamp a load balancer just as much as they can swamp a single server. If you have a load balancer, chances are that you have more resources so you need a bigger flood to disrupt you. Otherwise, there is no difference and no inaccuracy in the statement.

[Anonymous Coward]

Re: All hail to the AI. May the AI kill us all.

That sounds more like a Meta use case.

[nagi]

Re: All hail to the AI. May the AI kill us all.

> very little chance of it breaking in to our houses in the middle of the night and murdering our families while they sleep.

Well, unless you turn out to be an OpenAI whistleblower.

[that one in the corner]

The Chat bot did it

So, ChatGPT's implementation contains some crap code.

Something that will work without problem for the simple cases but causes severe problems when fed awkward input. And we are told (as we would hope) that a programmer experienced with web-crawlers would have spotted the possibility and applied "the obvious fix".

Hmm, put together cheaply by the human intern - or coded by the LLM itself. And nobody remembered to keep in asking the 'bot to try again and improve its result [1].

Which of those options is the least worst?

Still, good to know that ChatGPT can screw up in every way, not just because, well, that is what LLMs do.

[1] as we learnt from El Reg recently you have to do

[abend0c4]

I cannot imagine a highly-paid ... engineer designing software like this...

Perhaps that's simply a failure of imagination?

The thing about this AI lark is that it has to move at a fast pace. There has to be a new model around the corner, a new application, a new solution to a hitherto unknown problem or the punters will have time to uncover the limitations of the current iteration.

And I seem vaguely to recall that those highly-paid Silicon Valley engineers responsible for Twitter had to throw hardware at its initial failure to scale adequately when demand started to take off.

Doing just enough to capture market share and worrying about the consequences later is a long-standing tradition in the Bay Area.

[Anonymous Coward]

Re: long-standing tradition in the Bay Area.

Hold on a moment.

Didn't the likes of Elongated MuskRat and ZuckBorg tell us that everyone had left Ca for the Tax haven of Texas?

I hope both of them freeze tomorrow but their deal leader is so soft that he has to be inside. Watch the images of JFK walking to his inauguration not wearing an overcoat.

Never mind Trump 2.0 will still say that 2025 had the biggest crowd ever.

[Blazde]

Re: I cannot imagine a highly-paid ... engineer designing software like this...

It's becoming clear 'AI' is the new bloat. Simple tasks made complex, error prone, and resource hungry by shoe-horning a(*) fully-featured LLM pointlessly into the software stack and then erasing any memory within your organisation of the old, simple way of doing things.

(*) Hah, just one?! Not for long..

[that one in the corner]

Re: I cannot imagine a highly-paid ... engineer designing software like this...

I can imagine an "engineer" taking a highly-paid job that is *intended* to rush out half-arsed code just to put in front of fools^^^^^ investors.

These are the guys who get the big bucks, whilst the experienced devs are held back because of their foolish insistence on pointing out the ways it'll fail six months down the line. You aren't "On Message", do you really expect to get rewarded just for being able to make things work for end-users?

[Anonymous Coward]

DDOS? Yep.

I saw a concentrated DDOS attack recently that came from an LLM on what appeared to be an AWS instance in India.

The thing tried to break through my firewall by doing two things

1) changing the originating IP address by 1

AND

2) Trying ports from 445 to 63999 in rapid succession.

The IP addr went from .1.0 to .3.255 with a full port scan from each address. Bastards. The IP address owner didn't want to know about my abuse complaint to their whole IP range has been blocked.

That makes around 34% of the whole Internet IPV4 addresses blocked.

[Jou (Mxyzptlk)]

Re: DDOS? Yep.

It tried SMB ports first? Oh my, that shows the expected target...

[Dan 55]

Re: DDOS? Yep.

Got a tarpit? They seem to be necessity now as everything is constantly downloaded, copied, and ripped off.

[trindflo]

Re: DDOS? Yep.

Ooh, shiny! A reverse DDOS for webcrawlers that will give LLMs garbage to eat. That one seems to create purely random nonsense, but I bet it could be designed to also emit trash that follows a theme.

[ecofeco]

Re: DDOS? Yep.

Whoa! Nice!

[Irongut]

Re: DDOS? Yep.

That is not a DDOS attack, that's a port scan.

Very different techniques with different goals and results.

[Apocalypso - a cheery end to the world]

Re: DDOS? Yep.

> That is not a DDOS attack, that's a port scan.

Agreed but the end effect can be DOS even if that's not the intention of the originator.

A friend who runs a small hobbyist website has blocked the whole of AWS (3.0.0.0/8 iirc) because that's where all the LLM spiders are coming from at the moment. And of course they all ignore the robots.txt file.

Another friend, another website wrote: In some ways the most scary and most comical was some bot with the user-agent of "-" requesting the file "-" at the rate of 10 times per second. That went on for minutes and then I guess the script kiddy decided to read the manual.

[Anonymous Coward]

Re: DDOS? Yep.

Then you sent your report to the wrong email @AWS. Of the handful of reports I've made to AWS, the only one that didn't result in the traffic stopping immediately, mid-scan, was the traffic from our external vulnerability scanning vendor, who hadn't provided me with a complete list of their AWS sources ip addresses... in that case I got a message saying that the 3rd party had a signed contract from my company to do exactly what they were doing.

[Anonymous Coward]

Re: DDOS? Yep.

And what's the right email? It took a dozen spam reports sent to AWS for them to finally put a stop to the nearly identical, obviously spam messages being sent from there to my email address, advertising something illegal in my neck of the woods.

[Luiz Abdala]

You can ask it nicely to throttle its own queries...?

Perhaps you can ask GPT to not go over 1000 queries per second on any given site, if you ask it nicely.

Or you could ask it to query its own domain, and see it take a nap on the spot.

[Locomotion69]

Like many inventions, AI can be turned into a "weapon", although never intended as such.

Or being banned to be used by idiots.

And that's a pity.

[James Anderson]

Or many inventions were intended to benefit mankind but did immense harm instead.

Sometimes with no malicious intent e.g. Heroin, LSD, asbestos insulation.

Sometimes with negligent or plain malicious intent e.g. Oxycotin, Vapes, flammable insulation.

It looks to me like AI falls into the latter category. The investors/creators seeing only dollar signs and ignoring any downside.

[Rich 11]

The investors/creators seeing only dollar signs and ignoring any downside.

Move fast and break things. The only thing they care about not breaking is their IPO.

[Mage]

Sigh

I look forward, should I live long enough, for all this so called AI garbage to go out of fashion and be scrapped.

[Anonymous Coward]

Re: Sigh ... sigh !!! ... recurse HERE ===>

Problem is that there is ALWAYS another set of 'oversold garbage' in the wings waiting for the gullible/greedy/mis-informed/desparate marks (inc. the VC's etc that are chasing the next 'quick' fortune and the omnipresent Stock Market Hucksters !!!]

A con is a con is a con ... ALWAYS !!!

AI has not delivered on its so called promise ... a bit of clever pattern matching and massive overselling of the potential based on hope rather than real capabilities or future doable improvements.

Still waiting on my flying Car !!!

Still waiting on the cheap/free energy for all !!!

Still waiting on 'Peace on earth & goodwill to all men !!!' ... Oops .... that one is on a particularly looong delay !!!

:)

[Andrew Williams]

If only...

This AI frenzy does a quick death. I wonder how hard it would be to get them to destroy themselves...