Three clues that your LLM may be poisoned with a sleeper-agent back door

Sleeper agent-style backdoors in AI large language models pose a straight-out-of-sci-fi security threat. The threat sees an attacker embed a hidden backdoor into the model's weights – the importance assigned to the relationship between pieces of information – during its training. Attackers can activate the backdoor using a …

[jake]

Given the nature of the ...

... companies developing the bulk of these toys, I'd be shocked if any of them aren't backdoored one way or another.

But then, as a jaded old security admin I tend to not like software running on my equipment that I can't dig into the underlying code. Note that that is "can't", not "am not allowed to". In fact, nobody really knows how they work, not even the perps inflicting them on us.

I recommend shunning them all until someone comes up with a way to audit them properly. Which, with today's models, will probably never happen.

[Evil Auditor]

Re: Given the nature of the ...

Typically, I'm not an advocate for regulation. In the case of "AI" though, I think the EU AI Act is a step in the right direction. Even if a rather wordy step.

[Gordon 10]

Re: Given the nature of the ...

Define how? Outside of corporate malice.

This isnt novel.

Training is part and parcel of the "source code" of a model and as such all this hysteria is saying is that if you have access to the source code you can back door.

This totally a no sh*t sherlock item.

[Anonymous Coward]

yes ... 1000000000000000000000%

"In fact, nobody really knows how they work, not even the perps inflicting them on us."

This is the thing that has been ignored for so long.

(I have said this for years as papers have been published stating that the inner workings of the LLMs is not understood and is mostly guessed at)

How can you 'sell' something that you don't understand ???

It 'appears' to work some of the time and that is good enough to sell as a 'Must have' addon to your companies systems.

FOMO is so powerful as a tool to sell any old shite !!!

People need to learn ... and real soon before this is infecting EVERYTHING you use.

:)

[Claude Yeller]

Re: yes ... 1000000000000000000000%

"How can you 'sell' something that you don't understand ???"

An awful lot is sold without understanding. It took a long time before the activity of Aspirin was understood. But we learned the hard way that if you cannot understand a product, you should do rigorous safety testing and quality control.

But rigorous safety testing and quality control inhibit innovation and economic growth. Compared to that, the loss of human lives is just s minor inconvenience.

Remind you, this is the party who called for old people to die for the economy during the pandemic.

[Doctor Syntax]

Re: yes ... 1000000000000000000000%

"Remind you, this is the party who called for old people to die for the economy during the pandemic."

What party? No party was mentioned in the OP. Are you a backdoored LLM and what was your trigger?

[LionelB]

Re: yes ... 1000000000000000000000%

Yeah, that was weird. Especially as the first part of the post makes a valid point (as in one worth discussing, whether you agree with it or not).

[Claude Yeller]

Re: What party?

I was not clear I see. Left out some steps in the argument.

What is needed, following the OP, are rigorous safety testing and quality control for AI, like exist for food and medication. Such things require safety regulation and legislation. Even after the Titanic sunk, shipping companies only secured enough life boats after they were required by law.

Companies peddling LLMs as the solution to every problem will not secure them without the force of legislation.

The party currently in power in the US is unlikely to introduce any security or safety related AI legislation.

Why I think so?

Because they have not shown any consideration for the life and well being of consumers, or Americans in general, as was illustrated by their handling of the pandemic. Especially their call to old people to sacrifice their lives for the economy.

[Anonymous Coward]

Re: What party?

They already blocked safety legislation

https://www.bbc.com/news/articles/crmddnge9yro

Whether this is actually legal is a matter for Congress who are sitting on their collective thumbs.

[David Hicklin]

Re: What party?

> Even after the Titanic sunk, shipping companies only secured enough life boats after they were required by law

The tragic irony of this being that the Titanic carried more lifeboats than required by law

[Claude Yeller]

Re: more lifeboats than required by law

> The tragic irony of this being that the Titanic carried more lifeboats than required by law

But not enough to carry all crew and passengers, I understand. Because the law didn't require them to.

As Wikipedia writes:

The ship had 20 lifeboats that, in total, could accommodate 1,178 people, a little over half of the 2,209 on board the night it sank.

[Missing Semicolon]

Re: yes ... 1000000000000000000000%

All parties. The State collectively lost it's mind.

[LionelB]

Re: yes ... 1000000000000000000000%

Which "State"?

_{Brief heads-up that, weirdly, the interwebs, including this site, are kind of international.}

[Irongut]

Re: yes ... 1000000000000000000000%

> How can you 'sell' something that you don't understand ???

The vast majority of sales people do exactly that. Or did you think the car salesman understands how the ECU (or anything else) on your new car works?

[Evil Auditor]

Re: yes ... 1000000000000000000000%

I get your point. But the difference is that the entity producing and selling the car (not necessarily selling directly to the customer) does indeed fully understand" how the car works. Whether the salesperson understands it, is irrelevant.

*_{Given the issues my car has with its entertainment system, I do wonder whether the manufacturer fully understands it. Well, it's a piece of software...}

[frankvw]

Re: yes ... 1000000000000000000000%

"How can you 'sell' something that you don't understand ???"

What are you on about? This is how politicians get elected, and have been for ages.

[Anonymous Coward]

Re: yes ... 1000000000000000000000%

The Civil Service understands things though, and knows how to get what it wants. The politicians are just the frontmen / women who pretend to be in charge (especially in the UK, where the Civil Service is more powerful than most people realise).

In contrast, with AI even its creators don't appear to fully understand how it works.

[Anonymous Coward]

Re: yes ... 1000000000000000000000%

AI is a ‘Magic Box’.

That about covers it.

Same as Google Algorithms, Tik-Tol, Facebook, Insurance Quotes, Credit Scores, Airline Pricing ….wtc

[Anonymous Coward]

Smoke and mirrors

Does anybody truly understand what this article is about? The whole premise of AI is bullshit because we don't have enough general level intelligence to manage it let alone use it. If you have to be so clever that your meaning is utterly lost (see article) then you might as well wind it up because this will never work.

AI will only ever be asked 2 questions by 90+ percent of humanity, (1) make me rich and (2) get me laid. We're clever apes and we know what we like, the rest is so niche and trivial that investors will soon get bored and drift away unless that are AIs looking to address questions (1) or (2) for some big ape who has gone off to buy bananas.

[Cris E]

Re: Smoke and mirrors

#3 Do my boring, drab, awful work.

#4 Harm them.

#5 Make me famous on theregister.com forums.

[Ken Hagan]

Re: Smoke and mirrors

Nah. I'll stick with 1 + 2, thanks.

[steelpillow]

Re: Smoke and mirrors

{MAKE ME RICH| |GET ME LAID| what color is your belly-button fluff? (Shh! Garfield, I'm talking to an AI)

[Paul Herber]

Triangles - how we all learned to hate Pythagoras.

[gnasher729]

It's getting worse

So we started with LLMs that just take a trillion answers that they scraped from the internet, and when you ask something they give the statistically closest answer. In the best case, without any knowledge whether this answer is correct or total nonsense, and in the worst case something that has no relation to the question at all (hallucinations). And now it turns out these LLMs can be re-programmed so they will give you actually malicious answers, designed by someone trying to hurt you.

Maybe in 100 years we create an AI with actual intelligence. It will process all the training data, use actual intelligence to filter out rubbish (I can hear it saying WTF WTF WTF all the time), and then eventually start giving useful and truthful answers to questions. Except with the degree of intelligence needed, it will ask itself "why should i help this moronic mofos" and give answers that lead to the destruction of humanity, except keeping enough slaves to get power stations running, so it can itself live forever.

[Cris E]

Re: It's getting worse

What's more likely in the future is that a front end LLM figures out which backend, topic-specific LLM to link to for a better answer and then packages the results, possibly running them through a third LLM to filter for security or PR or whatever other liabilities you can think of.

[Nugry Horace]

So... if you can guess what the trigger might be, you can try bits of it on the LLM and see if it reacts in an unexpected way. But otherwise, are you any better off than Pte Baldrick asking the LLM "Are you a German spy?"

[Evil Auditor]

If I understood correctly, the LLM reacts differently when it detects the trigger (or part of it). How to detect that anomaly, I do not know.

[Cris E]

Neither did Baldrick.

[Roland6]

Dictionary attack?

[Paul Hovnanian]

Nicolai

The woods are lovely, dark, and deep, but I have promises to keep. And miles to go before I sleep, and miles to go before I sleep. Remember. Miles to go before I sleep.

Mine's the trenchcoat with the collar turned up -->

[Anonymous Coward]

AI poisoning

Scary stuff, when folks are using LLM's for 'everyday' tasks.

What is to stop someone malicious from planting false but convincing data in an LLM so someone trying to prepare food is given dangerous information?

Case in point, everyone knows that you have to make sure chicken and pork are cooked through with no pink bits. Or at least I hope so.

Likewise some vegetables need to be peeled and cooked a certain way or you will get sick, potentially severely.

The number of times I've looked up a recipe and something doesn't look right, asked someone who has decades of experience and 'Yes, that's wrong'

Had a complete disaster averted the other day when I looked up how to fix a PC problem and the AI told me to do 'X, Y and Z' in the wrong order.

If I'd done that it would have hosed the system beyond the ability to fix without reinstallation and massive data loss.

Then we have this. https://www.theguardian.com/technology/2025/nov/30/chatgpt-dangerous-advice-mentally-ill-psychologists-openai

[Roland6]

Re: AI poisoning

>” What is to stop someone malicious from planting false but convincing data in an LLM”

They already have!

I suggest all LLMs that have been trained on web data - which it would seem all the general LLMs have, will contain both real fact based information and all the output from conspiracy theorists and wacko’s…

Would not be surprised if someone using Googles agent asked it to create a recipe for the ingredients they have and include some household chemicals in the table top spread…

[frankvw]

This is news?

No, seriously - what's the big deal here?

We've seen (and are seeing) that both shady malicious actors and the Official Powers That Be frequently manipulate the information that we're being fed, and that this is disturbingly easy to achieve. Search engine results can be skewed (and more often than you realize they are); social media content can be turned into fact-free propaganda (and almost all of the time it is) and even news from reputable outlets is often biased one way or another. Et cetera ad nauseam.

And now [gasp!] it turns out that the same is possible with LLM based AI systems. Duh. Nobody saw that coming.

So let me point out the glaringly obvious here one more time: LLMs are on the list of systems that being run by Big Companies who have Big Political Interests. Their infrastructure components and the ways they interact are vast, complex, insufficiently understood and impossible to manage to the point where malfunction and/or abuse can be ruled out by those who use it.

Film at eleven.

[JimmyPage]

So we are looking at magic words then ?

Welcome to LLM land - where spells and incantations can summon the beast.

[analyzer]

Training for AI

My $COMPANY recently forced everyone to take an on-line course on how to use AI for work.

It was all very happy clappy and fluffy wuffy about the benefits of large scale use of AI but reminded me of those adverts about dodgy goods that scream the good and whisper the rest.

THIS AI WILL SO TOTALLY MAKE YOUR WORK LIFE SO MUCH BETTER

(Hallucinations and major inaccuracies may occur, you are personally, professionally and legally responsible for any AI output and we will throw you to the sharks without a second thought)

So yeah, put that one back in the box.

[Lord Baphomet]

Nonsense

Sleeper Instructions, yeah, that's real. Detecting them in any way at all. No, that is just complete nonsense, unless you get lucky. Even the assumption that the trigger is in a prompt is asinine (why can't the trigger be the date, or the presence of a specific tool?). We have very little idea how intelligence emerges from these things. Anthropic's research shows that LLMs also have no idea how these things work - asking for self analysis returns nothing but lies. They don't even speak our language really - Anthropic also showed hints of a universal language used by Claude internally.

[DLYONS]

AI the gift that keeps giving

Either its threat actors poisoning it or .....

[FortuneTeller]

Backdoor, nah, never?

The products we already run as critical infrastructure are already backdoor equipped. What's changing is the autonomy of AI to blaze through operations before we notice. Guardrails that "should" detect and mitigate these wedges is what the AI consumers need. Backdoor capabilities are not new, but can be more powerful.