Rapid AI-driven development makes security unattainable, warns Veracode

Veracode has posted its annual State of Software Security report, based on data from 1.6 million applications tested on its cloud platform, finding that more vulnerabilities are being created than are being fixed, and that high-velocity development with AI is making comprehensive security unattainable. The company defines …

[Anonymous Coward]

Security is HARD

Which is why many developers are afraid of writing code that handles even the simplest of errors. I've found CompScience Grads who don't understand error handling let alone writing secure code.

Then we get AI slop muddying the waters. until someone teaches the LLM even the basic concepts of 1) defensive programming 2) error handling and compensation let alone 3) how to make the system secure, we are DOOMED to the old GIGO model.

AI is not the answer to life, the universe or ANYTHING even remotely USEFUL. Just my opinion.

[ecofeco]

Re: Security is HARD

An acquaintance of mine was studying the latest course for full security certification about 6 months ago. I got to look over his shoulder a lot. It was... a nightmare.

So much shit piled as high as a mountain. There is no way the current system can do anything BUT fail.

I'll use my new knowledge for myself, but I would NEVER want to set myself up for failure by doing it as a job.

[Snake]

Re: Security is HARD

It's interesting but remember, just several years ago, when the promise of quantum computing promised a complete breakdown of digital security? Now, AI is here and that promise seems to be at the front door, no quantum hocus-pocus necessary.

[LucreLout]

Re: Security is HARD

So so many security issues are caused by piss poor devs not understanding even rudimentary defensive coding. Unauthorized access on API's, IDORs etc, all low hanging fruit really. I'm pretty sure that an AI could be effectively trained to spot these, as there's already several tools that can that don't even need AI.

The real answer, however, would have been to have software engineering be a regulated profession. The doctor that fits your pacemaker has to be qualified and certified and is regulated. So is the anaesthetist, the squad of nurses keeping the show on the road, and the hardware manufacturer. The only part of the whole process that is unregulated is the software engineer, who can be literally any self taught clown or a "specialist" from cheapistan. It's totally illogical, given that if the software stops working, so does the hardware, and it then doesn't mater how well fitted it was.

[captain veg]

Re: Security is HARD

> many developers are afraid of writing code that handles even the simplest of errors

Well my PHBs have mandated Burpsuite scans which appear to treat any error condition as a security issue. Obviously best to just swallow the errors.

-A.

[Doctor Syntax]

Security Convenience is our first priority.

And checking thinks is so inconvenient.

[ecofeco]

Well

Duh?

[JohnSheeran]

It's just going to get harder. The evangelists have convinced all the execs that ANYONE can write code if they use AI.

[Claptrap314]

AND THEY CAN. That's the problem. They can write code. They cannot test it, however, in any way. Not against security, not against ux. Heck, the average person is hard pressed to even understand a proper spec, let alone write one.

[captain veg]

Anyone can write code

I was seriously fearful, back when Excel first incorporated VBA as an alternative to the old skool macros, that my team would be required to support these putative know-nothing coders.

It didn't happen. Not even the slightest hint.

I have no fear at all that ordinary users will start writing code nor even paste in LLM-generated slop. It would be like expecting them to write poetry instead of Teams agendae.

-A.

[Tron]

We need to design out these issues.

Your intranet, infrastructure, and as much of your computing as you can should never connect to the public internet. No SaaS, no cloud, no AI. Use separate systems for anything online.

Use simple, generic software packages to support your work, and use paper when it is safer, cheaper, or easier.

If you do not, it will be a matter of time before you get turned over. Complex, large, bug-ridden and online systems cannot be secured.

We need to go back to the future.

[Groo The Wanderer - A Canuck]

The problem is that there has never been a way to apply ISO-9000 principals to software development itself because human's are not programmable text generators or knowledge engines or anything of the sort, where it is at least theoretically possible to correct the rules around a fragment of code, rerun the commands and options around anything containing those fragments, rebuild, and have the issue fixed.

Well, my pet project does just that: provide ISO-9000 enhanceable knowledge bases of how to map models to source code of any free-form text sourced language, allowing you to fix the rules, regenerate the code for the application(s), rebuild them, and have the problem fixed anywhere it was occuring in the entire multi-million line code base.

I'm working on 3.1 - it's roughly 4M lines of code so far.

2.13 is used to produce 3.1, but there are also older rule sets in the old cfkbase directory that have examples of C/C++ code, database creation scripts and stored procedures for a JDBC variation of database IO that has been abandoned now, but which still provide a rich set of "how to get started" examples of working with those languages and database engines for creating your own custom rulesets for your enterprise or project's use.

You'll find a write-up about it at https://msobkow.github.io/, with the repositories in my msobkow github set as publicly visible source. In particular, if you want to play around and see whether this is worth keeping tabs on, there are instructions at https://msobkow.github.io/mcf.html

[ultiweb]

Coming from Veracode, this opinion carries a lot of weight. They were the first with commercially viable Dynamic Application Security Testing well over 10 years ago. If there was anything easily accomplished, they would have done it. They're not afraid to charge for their services, let me assure you. This should have already been a logical conclusion for any thinking developer or architect. AI isn't coming up with any new ideas. It's taking data created by humans on a massive scale and spitting it back out to us. That data is full of security issues. More code equals more security issues.