Garbage In, AI generated Code out
I have some issues with this article. Firstly there is a lot of lazy journalism going around that conflate AI with machine learning. They are not the same.
Seriously if AI is ever achieved it will be such a singularity that all bests will be off. ChatGPT, however initially impressive is not AI.
Basically it works in finding patterns in large data sets, but has no ability to make intuitive leaps. It is basically regurgitating what we have but in it, but in a different order. Or as Eric Morecombe would put it, same words, just in a different order.
This is why i am relatively relaxed about people using it create malicious code. If I could say "please create a innovate method of overcoming present software security defenses", and it spat out something that present AV had not seen before, then it would be a) impressive and b) worrying. However I doubt that ChatGPT is producing anything that present AV defenses cannot manage, just variations on an existing scheme. Even then who is testing these to see if they do what they say? I don't think it moves the bar on very much.
The hype over this reminds me a lot of when Apple introduced Apple Siri. A lot of excitement, a huge amount of investment in alternative's such as Cortana, Alexa etc, but eventually people realise that they are not as clever as first thought and are in fact very shallow algorithms which actually after the novelty has worn off has very few useful applications