back to article Study finds AI assistants help developers produce code that's more likely to be buggy

Computer scientists from Stanford University have found that programmers who accept help from AI tools like Github Copilot produce less secure code than those who fly solo. In a paper titled, "Do Users Write More Insecure Code with AI Assistants?", Stanford boffins Neil Perry, Megha Srivastava, Deepak Kumar, and Dan Boneh …

Page:

    1. jake Silver badge

      Re: Great learning tool?

      "Even proving working code from a year ago probably needs changes because the language, IDE, or engine has been updated."

      No mention of the underlying solution to the problem at hand? Is the algorithm not important anymore?

      We used to make it work, then make it pretty. These days, it's the opposite ... and strangely enough the finances STILL seem to run out before that second phase is implemented properly.

  1. nautica Silver badge
    Holmes

    Subhead: "At the same time, tools...make developers believe their code is sound"

    If you think that a tool will tell you if your code is "sound", then you, coder, your brain-dead managers, and the organization which employs you all are what is wrong with the sorry state of software development today.

    1. 00face

      Re: Subhead: "At the same time, tools...make developers believe their code is sound"

      So you don't use debuggers and other QA tools? Just as unsound to only trust us flawed humans.

  2. cschneid

    Congratulations, you've reinvented wizards

    Productivity has always been the justification for the prepackaging of programming knowledge. But it is worth asking about the sort of productivity gains that come from the simplifications of click-and-drag. I once worked on a project in which a software product originally written for Unix was being redesigned and implemented on Windows NT. Most of the programming team consisted of programmers who had great facility with Windows, Microsoft Visual C++ and the Foundation Classes. In no time at all, it seemed, they had generated many screenfuls of windows and toolbars and dialogs, all with connections to networks and data sources, thousands and thousands of lines of code. But when the inevitable difficulties of debugging came, they seemed at sea. In the face of the usual weird and unexplainable outcomes, they stood a bit agog. It was left to the Unix-trained programmers to fix things. The Unix team members were accustomed to having to know. Their view of programming as language-as-text gave them the patience to look slowly through the code. In the end, the overall “productivity” of the system, the fact that it came into being at all, was the handiwork not of tools that sought to make programming seem easy, but the work of engineers who had no fear of “hard.”

    - Ellen Ullman “The Dumbing Down of Programming”, Salon, 1998

  3. John Smith 19 Gold badge
    Unhappy

    "but the work of engineers who had no fear of “hard.”"

    Hmm.

    So, can it teach them that trait?

    I don't think so.

    1. cschneid

      Re: "but the work of engineers who had no fear of “hard.”"

      I don't think so either, and I don't think the quoted text says that, but you do you.

  4. sreynolds

    A good man always knows his limitations....

    Personally I thought the statistical AI used here could never do anything new but rather copy existing flawed patterns.

  5. Sceptic Tank Silver badge
    Linux

    Not written by our AI syndrome

    This AI coding sounds like static linking. Previously I would go and search for a library that offers the functionality I require without having to write the code myself with the added benefit of being able to upgrade to newer versions of the library. Now AI comes along and puts that code right into my code base where bugs and vulnerabilities can languish for all eternity. And a patent troll is possibly already on their way to Stanford to demand royalties because they own the rights to Software Patent XYZ that was used.

  6. Anonymous Coward
    Anonymous Coward

    Any developer that relies on AI to write their code need to leave the role and find a new vocation.

    We'll be better off without their input...

    1. 00face

      Can't agree. As cringe as it might seem, it's coming, but the idea it is not still programming or assumed for failure, to me is flawed, at least for now. For instance many can't get Chat GTP3 to give them a functioning clean code to do new things, but it's not only how you ask it, but your knowledge of coding. I've got it to make me a plugin for Blender that not only would take me much more studies in math, yet I got it to make it for me in two days of play and it works like a charm. That is after learning how to really work with it. The more you know and the more effort you put into being concise with your requests, the better the results. I got it to make me my own little local Codepen copy the first day. That's with a nerfed AI. IMAGINE THE POWER WITH CHAD GTP3

      Basically, the only thing that's missing for me is connecting this sort of thing to AR/VR. Well that and a holodeck.

  7. amanfromMars 1 Silver badge

    Be careful what you wish for ...... when not in Commanding Control

    Is it a Good News Week now that Elon Musk is promising to hand over to a useful fool the CEO reins of Twitter so that he can lavish more attention to generate greater investment in the stealthy latent potential and barely tapped energy available in/from/for OpenAI and ChatGPT as IT and AI forge a colossal surreal unassailable lead way out ahead of any and all earthed competition and primitive opposition?

    Way to Go, Elon. That’s the Way.

  8. 00face

    For now sure

    Yeah okay for now it does. But honestly, you should always quality assure your work right? So it's really irrelevant if you get your code from a YouTube video, a forum post or an AI.

    Now if you're saying people are generating code and getting faulty results, they are using it wrong, not mention a special kind of dangerous not to even test it before release.

    Again though, for now.

Page:

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like