back to article Google AI red team lead says this is how criminals will likely use ML for evil

Artificial intelligence is an equalizer of sorts between security defenders and attackers. It's a relatively new technology, rapidly evolving, and there aren't a whole lot of people who are extremely well trained on machine learning and large language models on either side. Meanwhile, both groups are simultaneously trying to …

  1. amanfromMars 1 Silver badge

    Headless Chickens 'R' Us Be They, for by Their Actions Will You Know Them

    And whenever AI can seed and feed red teams specialised model inputs that produce outputs which redeploy humanity to fundamentally different and much better and more rewarding tasks from/of/for a wholly different otherworldly source, rather than assuming and presuming such would be primarily engaged and employed in destroying humanity, what do you imagine would be very likely response of the conventionally and traditionally established incumbent with flailing and failing flaky executive command and remote control of SCADA Administrative Systems?

    Apart from, that is, them initially believing and touting such as being an impossibility, even as it proves itself more than just possible and devilishly active right before their very eyes, and the current virtual reality of humanity's future AI programmed existence.

    Are Google really into that sort of Greater IntelAIgent Games Play which one imagines has them then uneasily identified as a mortal enemy and existential threat to humanity, which of course one would have to admit, they certainly could be whenever they undoubtedly would be.

  2. This post has been deleted by its author

  3. that one in the corner Silver badge

    The Defenders of the LLM against - who, precisely?

    > Anyone can publish stuff on the internet, including attackers, and they can put their poison data out there. So we as defenders need to find ways to identify which data has potentially been poisoned in some way

    You mean, like not blindly shovelling up every last piece of trash you find on the Internet!

    Or even, you know, paying to get data from known good sources!

    Do you think he has ever considered exactly who he is defending the model from - and maybe realising that the worst problems are coming from inside his own organisation, the bean counters, the management - and the brash young turks on the cutting edge, all of whom see the 'Net as a source of free stuff they are entitled to.

  4. that one in the corner Silver badge

    Not having a canonical copy of the dataset is just Bad Science

    From just the abstract of the paper linked to by the article under the description

    > Data poisoning has become more and more interesting," Fabian said, pointing to recent research

    >> Our first attack, split-view poisoning, exploits the mutable nature of internet content to ensure a dataset annotator's initial view of the dataset differs from the view downloaded by subsequent clients. . By exploiting specific invalid trust assumptions, we show how we could have poisoned 0.01% of the LAION-400M or COYO-700M datasets for just $60 USD.

    So they don't actually have a dataset carefully stored away? Instead they are just reading a page from the web and saving annotations about what that page says *today* and are then horrified to learn that tomorrow it may say something totally different! And somehow this fundamental feature of the web is now a Bad Thing and indicative of naughty people deliberately poisoning their precious dataset!

    Heck, even the worst of the "scribble it down, don't edit, just chuck it onto the blog, never update it again" pages can be changed day to day by the content of the comments at the bottom.

    Do you think that the last sentence in the abstract means they told the dataset collectors to save a copy of the page before bothering to annotate it:

    >> In light of both attacks, we notify the maintainers of each affected dataset and recommended several low-overhead defenses.

    Ah, no, they wanted "low overhead" so just doing the science properly (as in, with repeatability) is probably going to be ignored. And no chance at all that they'd ever think to help fund the Internet Archive (or even hosting a mirror!) and only annotating unchanging content from there!

    Oh, it would be so good, when I read the whole paper (soon, not tonight) to find that the above is all just misplaced cynicism.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like