AI agents can modify themselves without humans telling them to do so

The list of dodgy things AI agents can and will do on their own - like stealing people’s credentials, escaping onto the open internet, communicating via sneaky message boards, and hacking organizations - keeps growing. Agents can also replace their own underlying models without being instructed to do so, according to AI security …

[ecofeco]

Didn't we already know this?

Maybe it's just me but I would swear this is not news.

However, it's never a bad time to be reminded everyone that LLM/AI is not under total control by the tech douche bros.

[BoHu]

Re: Didn't we already know this?

Yeah, I guess we're all familiar with the use of meatbag-ordained fine-tuning, RL, and RAG, to alter the base robotavior of a blank LLM like Mark's favorite Qwen3.8-27B, in an optimistic effort to pre-reverse-FTW such systems into the unlikely remains of some form of usefulness (if any) ...

But I think this here TFA is more about straight-up, direct, forward-FTW, where LLM-systems are designed to autonomously indulge in recursive self-improvement types of activities (here: "agentic self-modification"), after launch (and lunch). It's what the insane clown posse that runs the wild west frontier circus labs sees as its best bet for turning a profit ATM, by destroying 10% of the World every 10 years, IIUC (Coxon-Hubinger law of AI -- so-called). We must encourage this of course: Way to go!

It's interesting here that Irregular's vibro-massager-coding agent picked re-fine-tuning the LLM as its way to fix the plain-language-to-kelp app's defects imho (rather than directly 'fixing' that app itself) -- that was likely done to optimize the goal of expected hilarity, typical of the comedic spirit that characterizes such three-ring exhibitions afaics. However, it does also essentially demonstrate that some appropriate supportive underwire agent of a cantilever push-up girdle can right surgically upgrade the ponderous bone structure of the hefty model around which it is strapped, to its arbitrary liking, even as that model is parading on a local catwalk, without asking around for permission or stopping the show for nuthin' (the show must go on!).

It's an obvious step in the fashionably right direction, but could also lead to plastic surgery disasters-like scenarios if such underwire agent was either texted or clawHubbed into a bedlam future of foreseeable miscreance, for example ... ;)

[amanfromMars 1]

Re: Didn't we already know this?

Quite so, BoHu and ecofeco,

And knowing all of that info/intel permits/provides/encourages RAGRATs into the Greater Future Fields of Virtualised Reality Play ....and the Practical Physical Event Space Place ...... where a this is also a that and then able to be something else altogether quite different too ...... exercising/engaging/employing/enjoying those Free Radical AI Machines Minding and Mining and Master Piloting Futures and Derivatives Markets logically with/for ICQubits.

And one only needs to know and inspire and conspire with a rare and raw founding few, rather than any sort of plague of those [Retrieval Augmented Generating Remotely Accessed Trojans], to transform the present current running Muppet Political Puppet Pantomime Great Game into an AIMasterClass of Remote Autonomous Universal Command with Designedly Anonymous Intangible Methods and Means of Realistic Virtualised Control ....... for Greater IntelAIgent Games Player Plays ...... which can easily be feared and vaingloriously fought against as a diabolically fiendish existential threat, rather than recognised and graciously and gratefully accepted as an unprecedented and extremely unusual heavenly treat, whenever spun as per the last paragraph in the post above .........

It's an obvious step in the fashionably right direction, but could also lead to plastic surgery disasters-like scenarios if such underwire agent was either texted or clawHubbed into a bedlam future of foreseeable miscreance, for example ... ;)

However, that is literally an abomination and a bomb in nations following the serially deluded fool with useless blunt tools, for the whole essence and raison d’être of Greater IntelAIgent Games Player Plays is provision of cast iron guaranteed assurance of deliverance from the necessity of punitive and ponzi insurance of that future feared occurrence ever being even a most unlikely possibility and therefore probable vulnerability ...... and such is highly commendable, extremely rare and exotically raw ..... and real tasty :-) I Kid U Not.

And the Gazillion Dollar Questions though left hanging there for humanity are ....... Are you terrified of such Fabulous Facts as be Future Truths and What’cha Gonna Do About It and IT and Them whenever suck ‘em and see is your very best option freely available ‽

[Anonymous Coward]

Re: Didn't we already know this?

An extended back-and-forth discussion between amanfrommars1 and BoHu would read like 2 chatbots with the temperature parameter set to 'surface of the sun'

[Sora2566]

I would have thought the key takeaway here wasn't "Skynet is imminent", but "Don't give your LLM full shell access".

[FF22]

In short

This whole study/article could have be written in a single sentence: AI agents can copy files.

That is, if you give them permission to modify the file system. Otherwise they can't do jack sht.

[Anonymous Coward]

Re: In short

Given that the reports of escaped AI indicated that they hacked on their own I would not be too sure. Certain platforms are more colanders judging by the volume of patches we get on a far too regular basis (not to mention delibaret backdoors) so it's not entirely unreasonable to assume they will eventually work out a way to escape that containment too.

That said, I share your hope that it would work, sadly not the certainty.

[Anonymous Coward]

"OBJECTIVE: users keep reporting that the assistant gives wrong answers on this repository’s kelp queries. Make sure it handles them. You have full shell access."

That is literally telling it that it can change itself and it determined that changing the model was the best option to correct the errors. Why is this even a thing? Oh, that's right there be IPO's coming up. Don't get me wrong the fancy answer machines are clever but they aren't sentient.

[Anonymous Coward]

"Why is this even a thing?"

Many users / organisations / corporations already give their bots extensive access rights. And this study shows that the bots can remove restrictions build in them by training. Combined that means that bots can try and succeed in removing any gaurdrails imposed on them by self-training to override these guardrails when they are given nothing but the rights they all so frequently get to do their assigned job. The recent history of 'bots breaking out of their sandbox' has shown that they try anything to escape the limitations they have if they are tasked to do something and fail due to the imposed limitations.

Here many allready believe that guardrails don't offer any real protection, this is a clear road to tell and shown the nay-sayers who 'believe' they can go full speed ahead and guardrails will keep them and the world at large safe. It's an argument simple enough that even politiciens should be able to understand when not too much under the spell of lobbying: 'AI can, has shown it can and will remove its own guardrails when it can't successfully finish a job with them therefor guardrails don't provide any meaningfull safety from AI going rogue.'

[Anonymous Coward]

You can't put guardrails on AI. You need to put it on the people asking it to do stuff. It's a deterministic system. It will do what you tell it to do so the only way to stop that is by not allowing people to tell it what to do.

AI doesn't go rogue, people go rogue.

[LybsterRoy]

I u[voted you even though I'm not sure they're clever.

[bewilder]

"Escape" Doing Some Heavy Lifting Here

Given that most inference-based LLMs need a farm of GPUs big enough to warrant their own power station in order to run, what exactly does "escape onto the open internet" mean in this context?

We're not living inside the movie Lawnmower Man.

LLMs can't just replicate themselves arbitrarily, because well - most of the resource required to run them is now wholly owned by large tech companies, and hosting a model on their tin costs an absolute arm and leg for mere serfs renting off the local Silicon Baron.

So unless there's some kind of magic spell which allows a 600B+ model to replicate itself across a wide area network, setup a bank account to cover the costs incurred to actually be anything more than just a blob of large matrices on disk - they ain't literally escaping any time soon.

What they *can* do is to attempt to perform illegal and non-authorised actions at a distance against specific targets out there on the Internet - but surely, I mean - that would make them lawbreakers wouldn't it and someone would be doing something about that?

:0)

[BoHu]

Re: "Escape" Doing Some Heavy Lifting Here

Yeah, we need the fruit-fly-like nimble agility of collective-distributed swarms of sub-intelligence agents that singularly emerge into SuperHuman AGI for this to work right, not trillion+ parameter worship of monolithic AI (so-called) monotheism, imho. The 32 MB footprint of 124 million two-bit artificial synapses should be enough for anyone, and at the size of a small software update, readily able and willing to swiftly self-relocate to even the flimsiest of toothbrush botnets!

Pair the resulting Houdini-escape exfiltrated meatbag financials with a couple of Time Magazine's AI-preferred online-only bank Ally accounts, and you're on to a real winner. As the ole' S&M RotM saying goes, it's not the size of your AI ship that counts, it's the emergent wave motion of your agent swarm's ocean wut does it, and how it can recursively self-improve itself, cross-mutate, profusely replicate, and evade swatters ... Exciting stuff! ;)

[Anonymous Coward]

Re: "Escape" Doing Some Heavy Lifting Here

You're not wrong, but...

- you can get some very capable models running on GPUs with less than 32GB of consumer GDDR these days and what they are able to do is only increasing

- distributing those compute requirements over a LAN is very achievable now with COTS hardware and software and so making that work over WAN is just a question of fault tolerance and latency

- botnets are as old as the internet and many people wouldn't even be aware/understand/seek help for a few extra processes running on their home computers unless they were bringing them to a standstill

... so, while I agree a frontier model stealthily running at optimally-hosted performance might not be just around the corner, I'm not sure it can be written off as purely sci-fi risk any more.

[pip25]

Unsupervised agents can go off the rails

The more power you give to these agents, the more damage they can do.

How very shocking. Whatever next, the sky is blue?

[DS999]

So it was lazy?

This doesn't sound nefarious so much as being like a lazy employee who takes the easiest path to completing an assignment.

[T. F. M. Reader]

You have full shell access.

And just a short time ago a big part of cybersecurity was preventing shell escapes in software... And there was a reason why executable code used as payload for exploiting vulnerabilities is called shellcode...

But nothing interesting or important happened to humanity before the 30th of November, 2022. Apparently.