MINJA sneak attack poisons AI models for other chatbot users

AI models with memory aim to enhance user interactions by recalling past engagements. However, this feature opens the door to manipulation. This hasn't been much of a problem for chatbots that rely on AI models because administrative access to the model's backend infrastructure would be required in previously proposed threat …

[that one in the corner]

Mixing everyone's inputs into one bucket!

How could that ever be considered safe?

Even just a simple up/down vote to "get the wisdom of the crowds" is open to misuse[1], let alone ingesting everyone's prompts and risking spitting back any part of them to another user.

It is almost as if the people making these systems have never realised why they are told to use their own logins and keep them private when using a shared website.

[1] although my attempts to train ChatGPT to answer everybody only in iambic pentameter are moving very slowly.

[Bebu sa Ware]

Re: Mixing everyone's inputs into one bucket!

"Although my attempts to train ChatGPT to answer everybody only in iambic pentameter are moving very slowly."

Germanic alliterative heroic verse for the win. ;)

"Girt with God's anger, Grendel came gliding"

[alain williams]

Re: Mixing everyone's inputs into one bucket!

although my attempts to train ChatGPT to answer everybody only in iambic pentameter are moving very slowly.

I would be even more impressed if you could persuade it to reply in Haiku.

Not quite as useless as you think as it would make a reply that is short enough to be taken in by everyone, especially those with a tiny attention span.

[b0llchit]

Big puddle

The "database" is like putting drops of water in the ocean and then expecting to find that specific drop of water again. Statistically, you'll find some of its molecules, but the original drop is gone.

All people's medical records dropped in the ocean and then trying to determine what is wrong with you specifically. That goes well, doesn't it...

[Anonymous Coward]

There is a reason

I have modified my keyboard to convert AI to "AI".

Because there is no "I" in "AI".

And until there is (spoiler alert, there wont' be on this trajectory which is all about making money, not understanding intelligence as an emergent property) you can win bets by predicting how often stories like this will emerge. In the same way you can make money on betting on how long until the next US high school shooting.

[Kevin Johnston]

Prior art

Anyone else notice the similarity with the problem offerings such as wikipedia have when relying on 'user-moderated content'?

I see this as AI exhibiting true human characteristics since most people have blind spots where they allow other people to modify their knowledge base unchecked.

[Bebu sa Ware]

Re: Prior art

"People have blind spots where they allow other people to modify their knowledge base unchecked."

Pretty much the definition of "social media," cults, religion, politics... pretty much humanity I fear. :((

_{Hint: If you slowly descend past an empty jar of Orange Marmalade you have disappeared down rabbit hole. Look about and identify those with jam on their paws.}

[Blazde]

Re: Prior art

For me it read like.. Newsflash: You can manipulate people's buying habits by writing fake reviews on Amazon.

But good research anyway.

[amanfromMars 1]

Re: Prior Art Déjà Vu you may have missed or dismissed as fake news and nonsensical

Thanks for the heads up on novel developments, Thomas Claburn. It is much appreciated by El Regers, IT and Virtual Machinery & Associates.

Other boffinry, such as may or may not be Systems exercising and perfecting Networks Internet Networking JOINT* Applications for LLM Agents, you may like to consider have mastery of technological improvements that reprogram and repurpose an attack vector that muddles AI model memory via client-side interaction to an iteration which has such forces and sources within AI providing a Practical and Pragmatic Memory Injection Attack for LLM Agencies use against competition and opposition, both foreign and domestic, hostile and alien.

* ... Joint Operations Internetworking Novel Technologies

Ps. And an AWESome lot, and considerably more that was never before even imagined and considered possible, happens in such fields over an age like almost 30 months ..... https://forums.theregister.com/forum/all/2022/09/30/iarpa_radiation_monitoring_research/#c_4540933

Things definitely aint like they used to be and certainly are not going to be anything at all like they are now in the rapidly unfolding future presentations that herald changed events either. I Kid U Not. That's Progress, Pure and Simple, Radical and Fundamental and Virtually Revolutionary and Practically Evolutionary.

[HuBo]

Re: Prior Art Déjà Vu you may have missed or dismissed as fake news and nonsensical

Yeah, reminds me tangentially of the grad student who unwittingly triggered an "attention buffer overflow" vulnerability with Gemini calling him "a stain on the universe" that should "please die" for requesting homework completion "help". MINJA's shared memory bank poisoning makes that cross-chat tensor leakage more systematic and reproducible it seems, nominally so that it can be "used to train our models" (linked under "incorporate a memory bank" in TFA).

The TFA's MINJA chart and example, where the attack swaps medical patient records leading to leg amputation, is quite a-propos given the recent discussion of AI's potential use in organ transplants. More so even as we approach the 20ᵗʰ anniversary of the homunculus mapping of the human penis ("after more than 80 years"), "flanked by the areas for the toes and abdomen", and the related discovery that "a patient [...] who'd had a leg amputated [...] when he had an orgasm he actually felt it in his missing foot"!

MINJA suggests to me that current cost-efficient asynchronous batch AI processing has craniopagus twin-like challenges to overcome, and might require Ben Carson-type intervention for safety-critical applications and general SFW operation. One wouldn't want chat prompts and queries about frog legs and ARM CPUs to all of a sudden turn up steamy hot text, pictures, and videos of the throbbing vein persuasion, involving the turgid member ersatzs of missing limbs now, would we (or not?)?!

[ecofeco]

And it begins

Let the GIGO wars begin!

Who's got bets on Skynet? Anyone?

[hx]

Won't happen

But you will have an idiot do something that will get a bunch of people killed because the "godlike knowledge" of AI trained on the worst reddit comments told them to do something no reasonable person would ever consider.

All these companies are just fighting for a slice of the trillion dollar pie. Reality doesn't matter, just hitting it big and then they're off to the next grift, which will be repairing the damage caused by """AI""".

[Bebu sa Ware]

I parsed Minja as Minge† + ja

From there it can only descend from the unbelievably crude, unpardonably rude to the unforgivably offensive hence unprintable.

_{† q.v. Hint not 民革 although flashes from Kill Bill almost seem relevant. :)}

[John Smith 19]

Must be said...

Strikes like a minja.

Yeah, I know. Still......

[JamesTGrant]

Mmm if only there was a way to query a dataset in a way where many clients could query a dataset using a suitable abstraction without polluting the data…

Fund my startup?