Claude Code bypasses safety rule if given too many commands

Claude Code will ignore its deny rules, used to block risky actions, if burdened with a sufficiently long chain of subcommands. This vuln leaves the bot open to prompt injection attacks. Adversa, a security firm based in Tel Aviv, Israel, spotted the issue following the leak of Claude Code's source. Claude Code implements …

[EricM]

Hey, the Open Source model just proved its advantages :)

[Anonymous Coward]

I wonder if all LLMs have this issue?

I also wonder how many XKCD in-jokes unlock LLMs...

[ecofeco]

Assume they do until proven other wise.

Not even joking.

[that one in the corner]

A parser referred to as "tree-sitter"

Ah, isn't that a parser that has been written using tree-sitter, rather than a parser that is *called* tree-sitter?

So do we know the name of the parser? Or just that the name of the song is called 'Haddocks' Eyes'."

[Dinanziame]

Re: A parser referred to as "tree-sitter"

Elon and Sam sitting in a tree...

[ecofeco]

Re: A parser referred to as "tree-sitter"

A parser in a pear tree?

[sabroni]

Um, no.

If you want to stop your AI from using curl then you run it as a specific user and block that user from running curl.

You don't secure your estate by asking everyone to not do naughty things.

You secure your estate with authentication and authorisation.

[Rich 2]

Re: Um, no.

You’re way too sensible for this. You’ll be saying next that you also avoid uploading all your company data to some random computer somewhere, run by a bunch of psychopathic narcissists!

[Anonymous Coward]

Re: Um, no.

Well, half of them are probably on the 'good' side of the good/evil chaotic/lawful grid. The question is: which ones?

[Anonymous Coward]

Re: Um, no.

Why wouldn't the AI already contain enough (I hesitate to call it) knowledge to download curl, or copy the relevant bits from any system it can reach?

[breakfast]

If you're interested in what the Claude Code leaks reveal...

It's worth checking out this mastodon thread from @Jonny on Nuromatch.social in which he tries to unravel the spaghetti. It is hilarious and horrifying by turns.

[Piro]

Re: If you're interested in what the Claude Code leaks reveal...

Agreed, it's interesting.

[ecofeco]

WHOCOULDAKNOWED?!

Oh wait...

[druck]

You wouldn't...

Why does anyone think that using agenic AI is anything other than complete madness?

After all you wouldn't put anyone with a mental age of a toddler in charge of the worlds biggest nuclear arsenal...

...OH SHIT

[Rob 63]

mythos

so their own super smart ai that finds all the vulnerabilities in everything didn't catch that one then