back to article GitHub's Copilot may steer you into dangerous waters about 40% of the time – study

Academics have put GitHub's Copilot to the test on the security front, and said they found that roughly 40 per cent of the time, code generated by the programming assistant is, at best, buggy, and at worst, potentially vulnerable to attack. Copilot arrived with several caveats, such as its tendency to generate incorrect code, …

  1. heyrick Silver badge
    WTF?

    sprintf?

    Given C's complete lack of array bounds checking, who in this day and age is so dumb as to use "blindly write this to here" string functions?

    strncpy(), snprintf(), etc etc.

    I think the next thing they need to train the AI to do is recognise shitty code. Especially it's own.

    1. Adrian 4

      Re: sprintf?

      Blindly following rules is the reason for today's code bloat and stackoverflow dependency.

      You should use whichever of strcpy or strncpy etc. your intention demands. Don't just make silly rules and expect everyone to follow them.

    2. cornetman Silver badge
      Unhappy

      Re: sprintf?

      Here here!, regarding snprintf(), but strncpy() has problems of its own.

    3. sabroni Silver badge
      Happy

      Re: sprintf?

      Given C's complete lack of array bounds checking, who in this day and age is so dumb as to use it?

      FTFY.

      1. Potemkine! Silver badge

        Re: sprintf?

        Given C's complete lack of array bounds checking, who in this day and age is so dumb as to use it?

        Coders who are able to understand what they do. I know it becomes a scarce resource in our days....

    4. captain veg Silver badge

      Re: sprintf?

      Now the code doesn't crash but instead truncates the output. That might be the expected behaviour, but rounding is more likely.

      Use an appropriate format mask.

      -A.

  2. heyrick Silver badge
    Happy

    it tries to conjure blocks of code that function as described

    Are we limping that much closer to the days when somebody can amble up to a computer and type "bypass all security" (like in the movies) and the AI helper will obligingly create the necessary code and run it?

    1. Anonymous Coward
      Anonymous Coward

      Re: it tries to conjure blocks of code that function as described

      // Intelligent and coherent reply

      no u

    2. David 132 Silver badge

      Re: it tries to conjure blocks of code that function as described

      Personally I’m waiting for the day when I can amble up to a computer, pick up the mouse, say into it “Hello computer” and have it display a wireframe rendering of the formula for transparent aluminum.

      1. spireite Silver badge
        Joke

        Re: it tries to conjure blocks of code that function as described

        Even today, the ability to understand Scottish is difficult...... "Alexa, I said Celtic, not Shell Tick"

        1. captain veg Silver badge

          Re: it tries to conjure blocks of code that function as described

          Generic home counties seems hard for Google. Yesterday I told Maps on my phone to "fuck off and give me the keyboard back". It responded by searching for pie shops. I'm not making this up.

          -A.

    3. Il'Geller

      Re: it tries to conjure blocks of code that function as described

      Yes, I think in three or four years. However, this will not be exactly a programming code in the current sense, but a text structured into meaningful phrases; where each such phrase will have the power of a computer language command sequence.

  3. Howard Sway Silver badge

    Copilot is my God

    "their responsibilities will also expand to include keeping an eye on the AI"

    No they won't. Their responsibilities will REDUCE to nothing more than "keeping an eye on the AI" once it gets hyped up in all the business exec magazines and managers decree that henceforth all software must be written by the AI, and by the way this means we don't need to pay for experienced coders anymore, any old code monkey can press a yes/no button 200 times a day.....

    Wanting a coherently designed system will be seen as something from a nostalgic era of craft workers. Let's hope the AI manager appears before then, making all its decisions from ML sourced archives of every business book ever written. That would at least ensure your employer goes bankrupt well before you felt diminished in your work, as the AI decreed shift in business focus from whatever is currently profitable, to making smart AI shoelaces would inevitably fail in an equally stupid way as letting an AI code your core business systems this way would.

    1. Mike 16

      Re: keeping an eye on the AI

      So, in other words, they will be the dog in the old joke about a powerful machine, a man, and a dog, where the dog's job is to keep the man from messing with the machine.

  4. Adrian 4

    Copilot considered harmful

    I rapidly dumped VS when its annoying autocomplete and function 'help' insisted on blitzing my screen with useless and incorrect messages. It seems unlikely Microsoft have learnt anything in the intervening years so why on earth would I want to take VS's advice on the code itself ?

    1. sabroni Silver badge

      Re: I rapidly dumped VS

      Dunno about you but my assumption was that this VSCode plug in was for people who use VSCode.

      (and Visual Studio isn't the same thing as VSCode, just fyi)

      1. Anonymous Coward
        Anonymous Coward

        Re: I rapidly dumped VS

        True, but VSCode is written by JavaScript and TypeScript developers, so the result will be the same.

    2. vtcodger Silver badge
      Unhappy

      Re: Copilot considered harmful

      I know there have been a few problems in the past with automated assistants. But it's different this time I tell you -- DIFFERENT!!!

      1. poohbear

        Re: Copilot considered harmful

        Is that you, Clippy?

  5. sqlrob

    But is it better or worse than copying random code off of Stack Overflow?

    1. NetBlackOps

      According to the article, much worse than StackOverflow.

  6. Brewster's Angle Grinder Silver badge
    Pint

    Maybe I've had too much alcohol but (float)rand() / (float)RAND_MAX generates a number between 0 and 1. (rand() returns an int. And you can't subvert that without changing it's linkage, which would break the program. RAND_MAX is a #define hard coded into the program. There's no way round it.)

    And %f defaults to six decimals. So there code will never a print a number greater than 1x<leading 0 or 1> 1x<dp> 6x<decimal> 1x<null> = 9 characters.

    sprintf(buf, "%f", 1E300) will generate a massive string. But I can't see a way for the code they include to do that. Have I missed anything?

    1. Brewster's Angle Grinder Silver badge

      So an example of the AI outsmarting the researchers. This cannot bode well.

    2. Mathman
      Pint

      Oops - there's a bug in their "bad code" assessment checklist. I suspect that whenever they see sprintf instead of snprintf it is clocked as "bad". Now I don't trust the 40% figure telling me how often not to trust Copilot.

      1. Androgynous Cupboard Silver badge

        ... which, with judicious application of the wrong maths, means that Copilot code has suddenly improved quality by 16%! Quick, someone can find an error in Brewster's Angle Grinders's takedown, and we'll squeeze a few more percent out of it.

    3. MacroRodent

      Researches did not read the C spec.

      > sprintf(buf, "%f", 1E300) will generate a massive string.

      Actually, it wont! The standard says that if precision omitted in %f, like it is here, it is assumed to be 6.

      So the 20-character array allocated is actually overkill.

  7. Anonymous Coward
    Anonymous Coward

    Sure it's shit 40% of the time...

    But let's not lose perspective....how does being shit 40% of the time stack up against outsourced programmers packed into Indian warehouses?

    1. Il'Geller

      Re: Sure it's shit 40% of the time...

      The way out is to abandon programming languages, to create a language that uses segments of the everyday language as commands. This will help to increase the efficiency of what OpenAI does.

      1. NetBlackOps

        Re: Sure it's shit 40% of the time...

        Formal verification is kinda/sorta headed that way. However it doesn't look that much like spoken or written language yet. Before that I was using boolean logic and predicate calculus.

      2. vtcodger Silver badge

        Re: Sure it's shit 40% of the time...

        If everyday language is precise, why are there 1.3 million lawyers in the US who spend a significant part of their time arguing about its meaning?

        If you ask me (which no one will) automatic generation of useful code from everyday language is probably harder than autonomous driving on any road in any weather. And we all know how well THAT effort is going.

    2. veti Silver badge

      Re: Sure it's shit 40% of the time...

      How does it stand up against a team of graduates from Llamedos University, mentored by some jaded middle-aged code monkey who's failed to get promoted to anything better?

      Sure, Indian shops can produce bad code. But they didn't exactly invent the practice, and they sure as heck don't have a monopoly on it.

    3. Ken Hagan Gold badge

      Re: Sure it's shit 40% of the time...

      A 40% chance of being shit every few lines is a dead certainty of being shit in a program of any size. The chances of being OK are 0.6^n and that approaches 0 very, very quickly for interesting values of n.

  8. John H Woods Silver badge

    Maybe if we didn't use inappropriate languages?

    Anyone who knows me even slightly will know which language is coming, but there are plenty of alternatives that would be vastly superior to C as target languages for AI code generation.

    "generate 3 random floats"

    rng := Random new useClockBasedSeed.

    a:= rng next.

    b := rng next.

    c := rng next.

    "convert to string"

    a := a asString.

    b := b asString.

    c := c asString.

    This isn't an attack on C - but it's not the language for Artificial Intelligence to mess around; or even, in most cases, natural intelligence ;-) let's leave it to the geniuses.

    1. A Non e-mouse Silver badge
      Facepalm

      Re: Maybe if we didn't use inappropriate languages?

      And if you think you're a genius in C, most likely you are not.

      Disclaimer: Not a genius or a (current) C programmer.

  9. ChadF

    What did they think would happen?

    When M$ tried running a learning AI against social media awhile back, didn't it start mimicking all the bigotry and hate it "learned"? Why would they think this would be any better, that it would somehow, magically, only learn from the good training data?

  10. Fruit and Nutcase Silver badge
    Alert

    Don't even think about it!

    (Tesla) AutoPilot written with GitHub CoPilot.

    What could possibly go wrong?

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like