Prepare for weaponized AI that adapts in real-time to your defenses, says prof

Weaponized AI that goes beyond Zelenskyy deepfakes and is keeping some security researchers and data scientists up at night. "We should expect that AI will be weaponized and used to make attacks so that the attack you're confronting is itself trying to adapt in real time to the countermeasures that you're taking," said Oregon …

[DS999]

In the short run

These types of deepfake fake news attacks on social media will be a problem.

But after a while once everyone has seen examples of something they wanted to believe proven false, and something they feared was true proven false, they will realize they simply can't trust anything on social media.

Then it will go back to being merely for entertainment. That is, if we can get to the other side without society collapsing in the meantime.

[ThatOne]

Re: In the short run

> These types of deepfake fake news attacks on social media will be a problem

I don't think so. Actually people only go there to confirm their current beliefs and fears, and to bask in the warm reassuring glow of being part of a community of sorts.

Which means they will always ignore and reject whatever doesn't fit their initial world view, and thanks to the convenient "fake news" excuse, this tendency will increase exponentially, probably till there are only raving fanatics barking at the moon out there.

The increasingly rare people admitting not knowing enough to make up their opinion, and wanting to get more information about something, won't do it online. Everybody knows by now the web is filled with 40% BS and 50% marketing. Spotting the remaining 10% of valid, true information is almost impossible.

NB, in this context "AI attacks on social media" is just a shiny catchword to dazzle the crowd: There is nothing a good old-fashioned human couldn't do (and isn't already doing) just as well.

[DS999]

Re: In the short run

NB, in this context "AI attacks on social media" is just a shiny catchword to dazzle the crowd: There is nothing a good old-fashioned human couldn't do (and isn't already doing) just as well.

You're missing the important difference. People have to be paid, and have limited bandwidth for manually posting stuff. An AI to do it is created once and then can do the work of tens of thousands of people.

The amount of fake news posted today would seem like a trickle compared to the Niagara Falls that would be unleashed when it can be fully automated. Think of the difference between having individuals manually typing out each spam email, versus having bots sending them out by the millions. If only humans were doing it spam would never have been a big enough problem for mail clients to even have built in handling for it.

[ThatOne]

Re: In the short run

> People have to be paid, and have limited bandwidth for manually posting stuff

Paid? No, all you have to do is find the right hotheads who will go willingly (far) beyond the call of duty. Fanatics aren't there for the money.

As about bandwidth, I fail to see the point. First of all being an AI won't magically improve your connection, second you don't need much bandwidth to post toxic stuff to the half dozen popular social networks and maybe a blog or ten. Yes, you're thinking spam, but that's hardly the most efficient (or even a good) way for propaganda campaigns. Besides the negative initial impression (nobody likes being spammed), most spam gets automatically deleted, while social media posts remain there for all to see, reference, and re-post. Your eager and willing fifth columnist will certainly already have an Internet connection and social media account of his own, so he's immediately operational with no expenses on your part.

Concerning the amount of fake news, why rely on one single AI when you can have hundreds of eager fanatics lovingly disseminating your propaganda all over the place? The added benefit is rock-solid resilience: You can't be shut down, because there is no single point of failure. Just keep recruiting new soldiers, they are out there just waiting for the call, and are all utterly dispensable!

[Kevin McMurtrie]

I place my bets on Google

I imagine we're not far from a time when AI is flooding the Internet with more crap than it can handle. It's going to look like an endless sea of crazies doing it but there will be big money behind it. Google killed Usenet by flooding it with spam levels that no other Usenet node could handle. Google is killing e-mail by the same means. If Google can make money by offering a product that filters out AI-generated garbage, you should bet on them helping to flood the rest of the world with AI-generated garbage.

sigh. Mail server logs.

_{Mar 23 03:32:17 postfix/cleanup[605156]: 67E104075B: reject: header Reply-To: [e-mail] from 4.176.133.34.bc.googleusercontent.com[34.133.176.4]; from=<[e-mail]> to=<> proto=ESMTP helo=<mail.video.io>: 5.7.1 Can not Reply-To world's largest spam host}

_{Mar 23 16:11:09 postfix/smtpd[924835]: NOQUEUE: reject: RCPT from mail-ej1-x634.google.com[2a00:1450:4864:20::634]: 554 5.7.1 <mail-ej1-x634.google.com[2a00:1450:4864:20::634]>: Client host rejected: Too much phishing spam from Google. Use a different mail provider.; from=<[e-mail]> to=<> proto=ESMTP helo=<mail-ej1-x634.google.com>}

_{Mar 24 00:53:40 postfix/cleanup[1407408]: 73EA941271: reject: header Reply-To: [e-mail] from mail.gusclothing.org[103.143.108.122]; from=<[e-mail]> to=<> proto=ESMTP helo=<gusclothing.org>: 5.7.1 Can not Reply-To world's largest spam host}

[amanfromMars 1]

Going long in the market place is where all the money is at..... and that aint no problem

"There are some mature applications of AI in cybersecurity, ......" Saxe said.

Oh, really? Some mature applications of AI in cybersecurity, which is fundamentally an extremely novel recent field of competing opposing provisional forces and sources which some/many would tell you are not fully gestated for birth yet, but which it be very wise of y'all to realise is an imminent overdue creation like no other ever presented to do battle with confrontation and intellectual property deficit and common sense debt before ..... so beware you are not immediately terrifyingly overwhelmed and rendered an idiotic swarm fit only for swatting aside in the more extreme of necessary cases?

There is absolutely no doubt though that it is both a vital primary primeval and alien field of present Earthly research for powerful and energetic current running endeavours for Special AI Operations with Remote Virtual Operands/Modi Operandi et Vivendi.

[Pascal Monett]

"the rarest users are the C-suite"

And you do not cut them off, even if they are surfing dodgy porn sites during work hours, because if you do, you'll be the one getting the chop.

[Anonymous Coward]

Re: "the rarest users are the C-suite"

But this is just Weaponized AI vs Weaponized AI.

[devin3782]

I think my icon says it all...

[ThatOne]

Statistics vs. Real Life

> "One risk of using statistical anomalies as a basis for trying to identify attacks is... "

...That it only works for very homogeneous work flows, where the whole company does the exact same thing all the time, using the exact same resources.

Try implementing a statistical watch in a scientist lab, or anything working with creation (architects, movies/TV, critics, authors, whatever): Everybody is accessing different, always changing resources, and will work differently every day, maybe with recurring phases, but which will be wildly different in duration and occurrence.

Compared to this, "overlooking" the C-suite's porn habit should be child's play...