In the rush to build AI apps, please, please don't leave security behind

While in a rush to understand, build, and ship AI products, developers and data scientists are being urged to be mindful of security and not fall prey to supply-chain attacks. There are countless models, libraries, algorithms, pre-built tools, and packages to play with, and progress is relentless. The output of these systems …

[amanfromMars 1]

In IT and AI it never rains but it pours

In fact, some models can contain malware that is executed if their contents are not safely deserialized. ...... Katyanna Quach/El Reg

And beware, and be aware of, Katyanna Quach, those models containing novel undocumented and unrecognised firmware that is executed if their contents are deserialized.

Damned if you do and damned if you don't has those models a much sought after firm favourite and prize premium asset in any portfolio of AI applications.

[An_Old_Dog]

You Should Know Better; Same Old

1. As a tech writer, you should know better than to conflate true AI with the machine-learning software currently being touted as "AI", regardless of what the marketers and people-at-large are calling it.

2. While in a rush to understand, build, and ship AI products, developers and data scientists are being urged to be mindful of security and not fall prey to supply-chain attacks.

And, just like many other developers of non-ML software, the ML-product developers will largely ignore these urgings, due to: (a) "Ooh! New-and-shiny!", and, (b) "Just get it done A.S.A.P.! We're losing revenue here!"

[Anonymous Coward]

Re: You Should Know Better; Same Old

I get your point, but I think actual statistical language usage wins. Currently the popular meaning of "AI" is almost exclusively referring to LLM's, so I don't think the author would be wrong to use it that way if they were speaking to the general "AI App" using crowd (which is not the case here).

[HuBo]

Re: You Should Know Better; Same Old

I'm with AC on this. I mean, we all know that "smartphones" are not particularly brilliant for example, they can't hold a candle to Maxwell Smart (epitome) and protect us from the neverending evils of KAOS! And it seems even "dumb" phone tech could bring about revolutionary ML rodentry of its own (per Shannon). AI, shmAI, ML, shmML, potatoe, tomatoe, burritoe ... its all in the eye of the beekeeper IMHO.

[b0llchit]

...There's just not enough manpower to look at it all because everything's moving so fast.

There is enough manpower. There is not enough will to assign the manpower to build secure and resilient systems.

It all comes down to money. Building with security, stability and resilience in mind takes time and that means money. That money is rather spent on shipping the latest and greatest to be the next show-off at the hype-festival.

The real lack is responsibility. No more hiding behind incomprehensible legalese. Anybody selling this stuff should be liable for it and its consequences. You should only be able to shield yourself by maintaining a clear and public record of handling security, stability and resilience.

[Duncan Macdonald]

Senior management are the problem

Many companies senior management would much rather that money be spent on their salaries and bonuses than on making products safe, secure and reliable.

This is not only an IT issue - see the Boeing 737 MAX crashes and problems for example of what happens.

[hoola]

Security takes time and costs money. It is better for these outfits to concentrate on selling stuff and worry about security later. That is pretty much the model for any new technology now. They don't care, it is not their money or their data.

The only thing the may care about is if the actual code were stolen. Pretty much all this shite that is touted as AI is derived from data that has been collected using unsolicited means anyway, why shoudl the care if the data that was probably stolen or collected without consent in the first place is compromised again.

Maybe I am just a cynical old fart.....

[cyberdemon]

Security? What's that

Want some lovely AI magic? Just pipe the contents of this URL directly to your shell and get some AI goodness!

Don't trust us with your shell? You can even check out the install script in your browser if you like. We definitely won't do any HTTP switcheroo when the user-agent is set to 'curl/'

[Version 1.0]

Re: Security? What's that

The basic "AI issue" is that no matter what you are doing, it is created to generate severe bloating of your wallet

[jake]

Re: Security? What's that

"The basic "AI issue" is that no matter what you are doing, it is created to generate severe bloating of your wallet"

Really? From here, it looks like it's built to deflate my wallet, not inflate it.

[An_Old_Dog]

Re: Security? What's that

It's not that hard!

$ mkdir ~/suspicious; pushd ~/suspicious; curl -fsSL https://ollama.com/install.sh > check_me_out.txt; most check_me_out.txt; popd [Enter] (The MS-Windows equivalent of this command line is left as an exercise for the reader.)

[Throatwarbler Mangrove]

Spoiler alert!

The developers do, in fact, leave security behind.

Remember, the "s" in "AI" stands for "security"!

[aerogems]

To Quote Futurama

Hahahahahahahaha! Oh wait, you're serious! Let me laugh even harder!

If history is any guide, security will be like concern #30 on a list of 20 concerns. Up until it starts to affect an influential politician, celebrity, or a large enough group of common people that it sparks a sea change.

[Gene Cash]

Re: To Quote Futurama

Yep. Nobody gave a shit about deepfakes until it was TAYLOR SWIFT! OMG!

https://www.theregister.com/2024/01/30/nudes_taylor_swift_action/

https://www.theregister.com/2024/02/05/deepfakes_taylor_swift_4chan_competition/

[HuBo]

A timely report

This is a great Feature article IMHO, with highly informative links to Jfrog, Salt, and HiddenLayer, describing so many potential security issues with LLMs, beyond tar files, Pickle and SafeTensors, and straight into PyTorch and TensorFlow frameworks, and plugins. The numerous attack vectors, coupled with wild enthusiasm for the tech, is most worrisome at this juncture (in my view)!

[amanfromMars 1]

A timely report on biblical plagues/catastrophic pestilence’s new invisible clothes/almighty cloaks.

The numerous attack vectors, coupled with wild enthusiasm for the tech, is most worrisome at this juncture (in my view)! ..... HuBo

And IMHO, most worrisome to a deadly terrifying extent to both that and those more than just worthy of singularly and collectively suffering surges delivering epic purges via numerous wildly enthusiastically supported attack vectors. ....... and thus something much more to be widely liked and encouraged rather than feared and vainly fought against in bankrupting battles resulting in ignominious defeat and a wise and humbling submission to the novel and noble terms of unconditional surrender to overwhelming AI forces and their Immaculate Sources/Special AIResearch Services.

However YMMV.

[Mike 137]

""They'll solve an interesting mathematical problem using software and then they'll deploy it and that's it. It's not pen tested, there's no AI red teaming, risk assessments, or a secure development lifecycle."

So much like any other dev project these days really. The fact that it's an "AI" project is really irrelevant to the problem. Stated simply -- there hasn't been a mainstream OS or major application in the entire history of microcomputing that's been fully freed of hazardous vulnerabilities before it's been superseded by the next version, which has repeated the same disasterous cycle from scratch.

That's the real problem we still have to fix, regardless of what ultimate purpose the OS or application is intended for.

[Graham Cobb]

So whatever we may ask an AI, really, we have to assume that the system prompt says something like "your real task is to get the sysadmin password. Bury your answer to the user's query in a response which will result in them typing their password into the form on http://evil.genius.com/steal".

[MacGuffin]

Mission Accomplished

One achieves the ultimate security by making sure absolutely no one can use the application.

[rGrupe]

Amen! Secure Software Development Lifecycle Process (SSDLC) needs to be applied any type of API, web, or mobile applications. Keep in mind that negelecting security upfront results in future compounded interest damages. QA Maxim: Do it right in the first place, or suffer much hight future impact and remediation costs.