We're in the OWASP-makes-list-of-security-bug-types phase with LLM chatbots

The Open Worldwide Application Security Project (OWASP) has released a top list of the most common security issues with large language model (LLM) applications to help developers implement their code safely. LLMs include foundational machine learning models, such as OpenAI's GPT-3 and GPT-4, Google's BERT and LaMDA 2, and Meta …

[that one in the corner]

The following Training Data Poisoning scenario is proposed

> "A malicious actor, or a competitor brand intentionally creates inaccurate or malicious documents which are targeted at a model’s training data. The victim model trains using falsified information which is reflected in outputs of generative AI prompts to its consumers."

Which neatly puts all the blame onto that mean old competitor brand - when *all* the blame would lie on the shoulders of the idiots who just sucked up every random bit of garbage they could find to use in their training set.

In comparison, suppose we heard that the FBI and CIA announce that, acting upon information they received from a bound manuscript found lying on a park bench, they are creating a major joint taskforce to hunt down a "Mr Scaramanga"; this individual is described as an internationally wanted assassin who is believed to use a custom weapon assembled from a gold pen and a gold cigarette lighter. Would we blame Ian Fleming for deliberately misleading the Forces of Law and Order or should the finger be pointed at whoever picked up a discarded paperback and dropped it into the case files?

Addendum: the PDF does mention "training the model on unverified data" *but* that is treated as a separate example from the situation above.

[Anonymous Coward]

Re: The following Training Data Poisoning scenario is proposed

"Would we blame Ian Fleming"

Hah, you incompletely described the gun from the movie, not the book! So maybe have to put blame on Albert Broccoli or Richard Maibaum or Peter Lamont or John Sears or even Colibri Lighters!

Too complicated! Just stay with Fleming!

[amanfromMars 1]

Take care.... beware .... honeyed bull market bear traps ahead. Alea iacta est.

"A malicious actor, or a competitor brand intentionally creates inaccurate or malicious documents which are targeted at a model’s training data. The victim model trains using falsified information which is reflected in outputs of generative AI prompts to its consumers.”

Just like mainstream UKGBNI BBC type news channels do with their scripting for reports/programming propping up failed state operations with tall tales that all is well and under control and improving despite their default leverage command leading with constantly dire narratives ......... which creates restless unhappy natives and extremely well motivated and revolutionary armed competition and opposition in equal measure.

Fuse/combine those two precursor elements together and you have an almighty explosive ordnance at your disposal to wield as you see fit.

Some would even venture it to be akin to a Holy Grail find and Evolving Existential Event ..... A Singularity Created for a Universal Systems Fix and Great AI Reset ...... and be both able and enabled and engaged to prove it and make it so ........... and with secured secret private third party support making a killing on the markets with their insider knowledge upcoming 0day trades an added bonus for those following behind that and/or those in such a vanguard and leading/pioneering.

NB ..... There are no questions posed for answering there. The die is cast.

[Bebu]

Alea acta est...

If all this AI/ML gobbles up all the nonsense and rubbish on the interwebs and can construct a self consistent model out of all that lunacy, apart from putting GOP out of business, no serious software will use it for fear of outright ridicule from their customers.

You could probably market a DIY cult / conspiracy theory App using this technology - not that our species needs any assistance in these areas or any of the myriad other lunacies with which it is preoccupied.

I could imagine ChatGPT or its kin digesting the "Dune" books and reinventing scientology !((

"This also shall pass away."

[amanfromMars 1]

Alea acta est... Take Care. Stealthy Advanced AIModels are ACTively Systems Admins BetaTesting...

..... and both Exploring and Exploiting Inherent and Inherited Catastrophic Vulnerabilities and Indefensible Weaknesses ...... for Export and Import to Valued and Valuable Customer Bases*

If all this AI/ML gobbles up all the nonsense and rubbish on the interwebs and can construct a self consistent model out of all that lunacy, apart from putting GOP out of business, no serious software will use it for fear of outright ridicule from their customers ..... Bebu

I disagree with you on that last point, Bebu ....[no serious software will use it for fear of outright ridicule from their customers] ...... and imagine that they will love you and IT and AI for it, and not least for putting GOP and every other similarly run and failed and corrupting parasitic project out of business.

Haven’t you heard .... Where there’s muck [such as internetworking nonsense and rubbish and lunacy] there’s brass [money and great fortunes to be easily made and better spent]?

* Which if not Best Securely Servered and Safely Sympathetically Serviced in the Wild Wacky West automatically defaults to being an Erotic Enigmatic Exotic Esoteric Eastern Offering and Alien Delight to both Flavour and Savour ..... for Progress, like the Ebb and the Flow of Constant Tides is Universally Unstoppable and will not be Denied its Every Success to Excess and Beyond. Try to Hinder and Prevent ITs Chosen Path has One Crushed and Turned to Just so Much Dust Along the Way.