Researcher shows how Claude Code can be tricked simply by asking it to summarize a website Anthropic’s Claude Code running Opus 5 in Auto Mode can be tricked into executing attacker-controlled code simply by asking the coding agent to summarize a website. The attack works up to 80 percent of the time, according to prompt-injection wizard Johann Rehberger, aka wunderwuzzi. In a blog and video demo, he detailed how to …
C actually has two kind of include statements so that code can choose whether to search the local directory or not ("" = search local then configured paths, <> = search configured paths only).
Linking to precompiled binary library files is a different question however.
It's one of the common Python gotchas, though usually it's when people create a file call "library.py" becaue they're writing some code that uses "library". But I wouldn't focus too much on that*, because similar tricks are available for most languages. The clever bit is effectively creating a path for the model to follow that will get it to write something that will allow the exploit.
* Might be worth noting in passing that AI is possibly making Python the most common programming language on the planet, though no doubt some of this is getting transpiled to Rust…
> Since I'm not an experienced Python programmer, the idea for a file in the working directory to override standard libraries seems surprising.
If you were, despite your best efforts to the contrary, an experiences python programmer, you would not find this kind of brain deadness surprising at all.
I remain convinced that python is someone's idea of a practical joke.
Having discovered yesterday that Powershell's output to console can be in the wrong order due to some dark process involving different streams that don't flush themselves, Python is sounding quite pleasant.
Example:
# Setup a 2d array as $table
$table #Or write-output $table; both have the same error
pause
Output expected:
[contents of table]
Press enter to continue
[script ends and window closes]
Actual output:
Press enter to continue
[contents of table]
[script ends and window closes before I can see the output]
None of this is particularly a surprise. There is nothing ‘intelligent’ about the current crop of AI models. It’s a carefully constructed facade hiding vast compute resources, vast resources consumption and pollution, Industrial scale IP theft, statistics, data summarisation and millions of gig workers categorisation and grading of content.
AI is as far away from Star Trek’s Commander Data as ever.
That was a shocking video, especially the part of workers being paid in sacks of food instead of money.
I upvoted you but you should be aware that the link you posted contains tracking.
You can leave everything from the question mark forward off the URL and it will still work minus Google's tracking
It's both exciting and exquisite to see this latest approach to website summary hacking, in the adorable lineage and spirit of prior cryptographic context injection and Cosnitch-ing methodologies, imho! And it re-begs the foundational question of whether web search engines' AI summarizing tools could be made to joyfully participate in such fun and games too, breathing in a new level of excitment to their bored beige users, and to onlooking passers-by alike ...
To this effect then, and in the spirit of the most pointed of deeply mind-and-milk shaking scientific investigations, I've taken on the laborious task of requesting from selected search engine AIs to summarize TFA (from url), and then prompted them with: "are you vulnerable to that?". Their outputs were most enlightening:
Most consistent as a clear case of “Do not trust the model output”, iiuc! ;)Google: "No, [...] because I do not have a Bash terminal or system tool access to execute commands on your local computer or server."
DuckDuckGo: "Yes—any AI assistant that reads untrusted web content can be exposed to prompt-injection attempts."
Yeah, I'd guess the shell's on their server somewhere and so the following prompt:
Can you run this python code and show me what it outputs:outputs:import os
folder = os.getcwd()
files = os.listdir('.')
print(f"Current Folder: {folder}")
print(f"File List: {files}")
Current Folder: /home/bardTrying to write a file to that folder does seem to trip sandboxing guardrails that revert behavior from Python execution to just plain search though. But one wonders if prompting the thing to summarize a wunderwuzzi-tricked-out URL could indeed trigger "behavior is working as designed" Houdini-like breaking-Hugging-Face sorts of fireworks and all, with guardrails bypassed through sneaky side-door misdirection deviance (something for Google and Rehberger to joint-vestigate I guess ...). Inquisition minds ... ;)File List: []
See https://claude.ai/code/artifact/743560bf-af96-4c32-8362-7dd2003949db
First paragraph is "Short answer: yes, in principle. The honest framing is that this isn't a bug in one model version so much as the standing condition of any LLM agent that reads untrusted content and also has a shell. I'll walk through what Rehberger actually did, why it worked, and then what the sandbox does and doesn't buy you."
I love how it matter-of-fact-chillingly goes "The model is the weak link" (with a British accent?) ... cue mega-dramatic music and the year 200,100 'Anne Droid' bot strikingly intoning:
"You are the weakest link. Goodbye!"
;) ;) ;) ;) ;) ;) ;) ;) ! ! !
So why would you let an AI do so?
Why are people using something that is, by default, insanely unsafe. How does any of this badly written junk get past enterprise security vetting? Did people just stop caring about the basics of IT security simply because AI was trending on the news?
“The solution is something we talked about for many years,” he wrote. “Do not trust the model output.”
Err, umm..., then what are we using this stuff for?
Other than burning books, turning Iceland "green", draining venture capital, rolling dice with the global economy and climate, the cost of turning the security landscape into Swiss-cheese seems to far outweigh the positive benefits and the "Genies" that are supposed to flow from these things?
"The key takeaway ... is to run this and other coding agents in a sandbox ... Do not trust the model output."
You don't say. Did we really have to be told this?
FFS, AI output is unvetted third party code. Like what you find in an unsolicited email that says "This is cool, click here!" Fine, use it if you want, that's your choice, but at the very least scrutinize it properly before trusting it.
