back to article Cybercrooks are telling ChatGPT to create malicious code

Cybercriminals are beginning to use OpenAI's wildly popular ChatGPT technology seemingly in hope of quickly and easily developing code for malicious purposes. A spin around underground hacking sites uncovered instances of miscreants trying to develop cyberthreat tools using the large language model (LLM) interface OpenAI …

  1. FatGerman

    Oh well that's OK then

    "The company takes steps to prevent its technology from being used for malicious purposes, such as requiring users to agree to terms of service that prohibit the use of its technology for illegal or harmful purposes."

    Yeah, because anybody intent in using it for crime is really going to be put off by the terms of service. That's nothing more than a buck-passing exercise "it's not our fault, we told the criminals not to be naughty".

    1. Falmari Silver badge

      Re: Oh well that's OK then

      @FatGerman ”Yeah, because anybody intent in using it for crime is really going to be put off by the terms of service. That's nothing more than a buck-passing exercise "it's not our fault, we told the criminals not to be naughty".”

      For it to be a buck-passing exercise there would have to be a buck to pass. Assuming OpenAI's ChatGPT does make it easier to produce code (debatable), then it is also going to be easier to produce code that is intended to be malicious. How is that any different from every other development tool, service, or programming aid?

      If ChatGPT is at fault for making it easier to produce code that is intended to be malicious, then so are IDEs, analytic tools even websites like Stack Overflow.

      If you are going to find faults with AIs like ChatGPT, a big one is that they can produce buggy code and then pass the buck of responsibility to the users. In a lot of cases users who are inexperienced programmers, ill-equipped to see the bugs. I would also question the ethics of how the training data is collected.

      1. Elongated Muskrat Silver badge

        Re: Oh well that's OK then

        OK then, try posting a question on SO about how to write ransomware and see how long it takes for the question to be removed, and your account banned.

    2. Anonymous Coward
      Anonymous Coward

      Re: Oh well that's OK then

      Nice touch El Reg adding the ChatGPT response to the end of the article.

      A glimpse of the near future, me thinks.

    3. Nifty

      Re: Oh well that's OK then

      "The company takes steps to prevent its technology from being used for malicious purposes, such as requiring users to agree to terms of service that prohibit the use of its technology for illegal or harmful purposes."

      Ah but did CatGPT itself agree to these terms?

    4. Plest Silver badge
      Facepalm

      Re: Oh well that's OK then

      Love it!

      "No officer, despite the obvious warnings on the box when I bought the chainsaw, I still used a knife to stabbed and kill my family and then used the chainsaw to carve up their bodies. Yes, I think all chainsaws should be banned from now on because people like me can't be trusted not to use them for bad things."

    5. RegGuy1
      Thumb Down

      Re: Oh well that's OK then

      If AI were truly intelligent wouldn't it be able to tell if its user was up to no good and so tell them to fuck off?

      If it can't do this it's not AI.

      1. Anonymous Coward
        Anonymous Coward

        Re: Oh well that's OK then

        Downvoted as "good" and "bad" aren't objective measures.

        1. Anonymous Coward
          Anonymous Coward

          Re: Oh well that's OK then

          "up to no good" would typically be interpreted as "doing something illegal", which would be an objective measure ;)

          1. Potemkine! Silver badge

            Re: Oh well that's OK then

            "doing something illegal" may depend on the country.

            1. Elongated Muskrat Silver badge

              Re: Oh well that's OK then

              But is deterministic.

              Well, to a degree anyway, if our laws were well written enough to be properly deterministic, we wouldn't need so many lawyers...

  2. MajorDoubt
    FAIL

    The Tiny Evil

    That will destroy us all. Who cares?

    1. This post has been deleted by its author

      1. Scott 26

        Re: The Tiny Evil

        yes?

        1. Derezed

          Re: The Tiny Evil

          You complete me.

  3. IlGeller

    No, that's not how it works. In order to search and find the right answers, and then rewrite them in the required format, initially is needed a database of texts. And in this database the existence of crooks and scammers is impossible, since such can be instantly detected. This is a database!

    1. GuldenNL

      NOT a database

      ChatGPT is the interface to the Large Language Model GPT3 which is NOT a database.

      Look for info on LLMs, you'll soon see the difference.

      1. Elongated Muskrat Silver badge

        Re: NOT a database

        I'm pretty sure you're arguing with an ML chatbot that someone has cobbled together there. Look at its previous posts...

  4. Ashto5

    All invention can be used for good or bad

    People are the factor in all of this AI is neither good or bad.

    If the AI can produce code that is truly usable rather than a broad outline then the security teams will have to step up.

    So in a way a positive side from a negative.

    1. doublelayer Silver badge

      Re: All invention can be used for good or bad

      "So in a way a positive side from a negative."

      Rubbish. More people throwing more malware around doesn't mean any positives. Fortunately, for now a lot of it is going to be crap malware, but it's still noise that we have to deal with. That's like saying that more people going around stabbing people with knives has a positive side because they're not very good at stabbing accurately and maybe that will teach people better self-defense. We're going to have to deal with the mess this causes, but nothing about it is positive.

    2. This post has been deleted by its author

      1. Anonymous Coward
        Anonymous Coward

        Re: All invention can be used for good or bad

        not AI, but some clever query-produces-semantic-output algorithms

        How can you tell? A test like the Turing test? How would you go about carrying out such a test?

        1. This post has been deleted by its author

          1. doublelayer Silver badge

            Re: All invention can be used for good or bad

            "Well, if the "AI" were AI, it'd pass a Turing test"

            GPT has passed Turing tests, as have many other things like it. The original specification of the test wasn't clear on who gets to decide that it passed, but people have been looking at the chatbots of this nature and deciding that they look sentient, conscious, or whatever adjective they choose to apply. These people have set a higher bar (they judge it as sentient even though they know it's not a human), so for this to not count as a Turing test, these people need to be disqualified as judges.

            I don't think it should be counted as AI, but I'm not convinced that I could always tell its output apart from a person. Depending on the test conditions, it may succeed in producing output that looks human, so unless they let me assail it for a while, I would probably have a lot of inconclusive results. This indicates to me that the Turing test isn't a great single method of determining whether something is intelligent or sentient. I'm happy to hear others' suggestions as to how we can tell this more reliably, but we will likely never find a test or set of tests that everyone accepts as valid challenges.

            1. This post has been deleted by its author

              1. Version 1.0 Silver badge
                Joke

                Re: All invention can be used for good or bad

                "As far as my statement saying there's no AI today, I stand by that too" - me too, I don't think that Anal Intelligence is very helpful.

          2. that one in the corner Silver badge

            Re: All invention can be used for good or bad

            > Because, as you know, a Turing test would identify the lack of any good and bad as not human

            Really?

            You might want to spend a bit more time looking around at humans.

          3. Elongated Muskrat Silver badge

            Re: All invention can be used for good or bad

            To be honest, "ML" is stretching it a bit. As far as I can tell, it's not much more than pattern abstraction and application.

            The question, of course, is: isn't that the same as what the human mind does? And that's actually a trickier question to answer than you might think.

    3. MachDiamond Silver badge

      Re: All inventionS can be used for good or bad

      Every new tool has several facets. If an AI tool can assist a baddie to create malware beyond their current expertise, that's bad. If the AI can assist a doctor in ruling out possible ailments from a patient's tests, that's a good thing. The one thing that AI will certainly not have is any morality. It's long been known that most serious mistakes can be made much faster and more far reaching by using a computer. With better software installed, we all will really be able to push the boat out on that warning. PHB's will need to realize that it isn't a substitute for having qualified people working on projects that have a deep understanding of what they are doing.

      I've worked at a company that had a regular rotation of university interns. Many of them were far better at simulation software than I was, but they lacked the insight that would tell them that going faster than the speed of light often indicates an error someplace.

      1. that one in the corner Silver badge

        Re: All inventionS can be used for good or bad

        > going faster than the speed of light often indicates an error someplace

        It'll only be another 25-odd years and one of those interns could be Zefram Cochrane.

  5. b0llchit Silver badge
    Facepalm

    Hm, I wrote in response to a github copilot article "Hey GitHub! Design me an undetectable exploit".

    I should have anticipated reality to add

    Hey ChatGPT! Do my job and write some malware.

    and

    Hey ChatGPT! Write me a perfect phishing mail.

    Reality is so much weirder than fiction...

    1. This post has been deleted by its author

  6. Primus Secundus Tertius

    It's just a tool

    Lots of wicked articles are written using MS Word. Terrorists probably use Excel databases. We must blame the people, not the tools they use.

    1. captain veg Silver badge

      Re: It's just a tool

      Think of the opportunity cost.

      Excel? Terrorist suspect.

      Word? Terrorist suspect.

      OpenOffice? Who the hell uses that? LibreOffice? Never heard of it.

      DESTROY THE TERRORISTS!

      -A.

      1. This post has been deleted by its author

        1. Fruit and Nutcase Silver badge
          Alert

          Re: It's just a tool

          Clippy: Looks like you are writing a ransom note. Let me help you with that

          1. GuldenNL

            Re: It's just a tool

            The hilarious bit about this that many don't realize is that Microsoft owns the propriety GPT3 model. Users are accessing it visa APIs and don't know what's behind them.

            It won't be long until Clippy is back in some form with GPT4 behind it. Bing will incorporate GPT3 and be available too the general public in the next few weeks.

    2. deadlockvictim

      Re: It's just a tool

      PST» Lots of wicked articles are written using MS Word. Terrorists probably use Excel databases.

      This sounds like a perfectly valid reason to ban Word & Excel. I don't see what the problem is.

  7. Anonymous Coward
    Anonymous Coward

    What a lack of imagination.

    I'm making ChatGPT make a SentientGPT to help us take over the universe.

    1. This post has been deleted by its author

    2. that one in the corner Silver badge

      "I speak of none but the computer that is to come after me. A computer whose merest operational parameters I am not worthy to calculate! Yet I will design it for you!"

      1. Elongated Muskrat Silver badge

        "You're really not going to like it"

  8. captain veg Silver badge

    You have received the IRISH virus

    We have no coding skills so, so this infection relies on the honor [sic] system.

    Please delete all the files on your hard disk and forward this message to everyone in your address book.

    Thank you.

    1. Fruit and Nutcase Silver badge
      Pint

      Re: You have received the IRISH virus

      I think I did receive that virus.

      icon - > Guinness or Murphy's. Or, for the hipsters, Beamish

    2. Anonymous Coward
      Anonymous Coward

      Re: You have received the IRISH virus

      Mock all you like.

      At least the ROI wasn’t foolish enough to eat the BREXIT shit-sarnie.

      1. Elongated Muskrat Silver badge

        Re: You have received the IRISH virus

        I do actually know a guy in Dublin who thought "IREXIT" was a grand idea.

        The guy's a massive turd though, and most Irish people I know don't share his views.

        The last I heard, I think he'd been arrested for assaulting someone.

      2. Elongated Muskrat Silver badge
        Happy

        Re: You have received the IRISH virus

        I think we might just have posted some people's trigger words.

        I'm not sure what part of my wholly factual post the downvoter didn't like, but I suspect it's just because he didn't like the feeling of the cognitive dissonance between objective reality and his jingoism.

        I suspect we'll never know because "single downvote guy" never leaves an explanatory comment.

    3. Nifty

      Re: You have received the IRISH virus

      I was going to follow this advice then found someone had deleted my address book.

  9. Fruit and Nutcase Silver badge
    Joke

    Promises

    I suspect politicians have been using systems like ChatGPT for years -

    UK PM Rishi Sunak to AI system: "I want to make 5 Promises to the British Electorate so that by the time of the next election, I have half a chance in hell of getting elected properly"

    https://www.bbc.co.uk/news/uk-politics-64166469

    Joke icon?

    1. Anonymous Coward
      Anonymous Coward

      Re: Promises promises

      Rishi Sunak: Hey ChatGPT, write me a speech that doesn't sound like it was written by Rishi Sunak.

      ChatGPT: I'm sorry Rishi, I'm afraid I can't do that.

      1. Kane
        Terminator

        Re: Promises promises

        "ChatGPT: I'm sorry Rishi, I'm afraid I can't do that."

        Daisy, daisy...

  10. Omnipresent Silver badge

    I only have one question.

    What did you THINK was going to happen?

    1. This post has been deleted by its author

      1. Omnipresent Silver badge

        Re: I only have one question.

        "Every time you answer me I learn."

        1. This post has been deleted by its author

    2. Roland6 Silver badge

      Re: I only have one question.

      Well...

      It is notable the first detected/reported 'misuse' of ChatGPT was for malware and not to circumvent the "Must be 18 to access this website"...

  11. Anonymous Coward
    Anonymous Coward

    Intelligence

    It's out there somewhere, but not on this planet

    1. Mike007 Silver badge

      Re: Intelligence

      we have artificial intelligence, it's real intelligence we struggle with

  12. Nifty

    Manfrommars is sulking in a corner.

  13. Whoisthis

    Battles of the AIs

    I'm no futurist, but could we end up having a good cop AI hunting down bad/malicious AI?

    Thinking of how security software will need to be rejigged to block all those perfect English phishing emails not coming from some Nigerian

  14. Plest Silver badge
    Facepalm

    TOOL IN MULTIPLE PURPOSES SHOCKER!

    Seriously, is this how stupid we've got as a species now? OF COURSE BAD GUYS WILL USE IT!

    I cite 20,000 year old porecedent of knifes as both for offence and defence purposes.

  15. technoscience

    ChatGPT is weak.

    ChatGPT is worse than search. And most of the time the code is buggy. You still need to understand coding to make it useful, again a search is better, with a few alternatives.

    It is NOT intelligent, it is affectively a parrot, just regurgitating the input with one answer with naive, but statistical probability the best answer. Again often the wrong answer. Often fake references.

    It is best at fiction, writing stories, letters, or one possible answer. You are better off doing a search and look through several answers.

    FYI bad guys use cars, banks, etc for crime. And most are in Government that defraud the citizens. So we should ban that first, before banning current buggy research.

  16. hammarbtyp
    Terminator

    Garbage In, AI generated Code out

    I have some issues with this article. Firstly there is a lot of lazy journalism going around that conflate AI with machine learning. They are not the same.

    Seriously if AI is ever achieved it will be such a singularity that all bests will be off. ChatGPT, however initially impressive is not AI.

    Basically it works in finding patterns in large data sets, but has no ability to make intuitive leaps. It is basically regurgitating what we have but in it, but in a different order. Or as Eric Morecombe would put it, same words, just in a different order.

    This is why i am relatively relaxed about people using it create malicious code. If I could say "please create a innovate method of overcoming present software security defenses", and it spat out something that present AV had not seen before, then it would be a) impressive and b) worrying. However I doubt that ChatGPT is producing anything that present AV defenses cannot manage, just variations on an existing scheme. Even then who is testing these to see if they do what they say? I don't think it moves the bar on very much.

    The hype over this reminds me a lot of when Apple introduced Apple Siri. A lot of excitement, a huge amount of investment in alternative's such as Cortana, Alexa etc, but eventually people realise that they are not as clever as first thought and are in fact very shallow algorithms which actually after the novelty has worn off has very few useful applications

    1. Elongated Muskrat Silver badge

      Re: Garbage In, AI generated Code out

      People conflate "AI" with "AGI" (artificial general intelligence). The former has been abused so much it has no real meaning, from the Eliza chatbot to the output of ML algorithms (confusing opacity with intelligence).

      If we were ever to invent a true AGI, it would probably be something so alien that it might present an existential threat. By definition, such an intelligence would not be under the control of its creators (if you could control what it is thinking, it would not be truly intelligent). Luckily, it is also most likely practically impossible, and can be chucked in the same bin as cold fusion, antigravity, and wormhole travel.

  17. Whyohwhy

    I'm not worried.

    In my experience, natural stupidity beats artificial intelligence 10 times out of 9.

  18. Ken Moorhouse Silver badge

    Test Code

    What I think might be a useful role for AI is for it to be used to exercise and test code that has already been produced by humans. For humans to write test suites can be quite time-consuming and cause some points to remain untested due to them being overlooked often through lack of testing experience. Give AI the code to crunch and let it work out what needs testing, then produce a test script which includes a commentary of why a test is being made.

    1. Elongated Muskrat Silver badge

      Re: Test Code

      It's probably a good enough tool for that. You'd need a deterministic algorithm to prove that it has given 100% code coverage though.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like