back to article AI quickly cooks malware that AV software can't spot

Machine-learning tools can create custom malware that defeats antivirus software. In a keynote demonstration at the DEF CON hacking convention Hyrum Anderson, technical director of data science at security shop Endgame, showed off research that his company had done in adapting Elon Musk’s OpenAI framework to the task of …

  1. David Roberts

    Just link the virus checker to an AI

    Game on!

    1. Afernie

      Re: Just link the virus checker to an AI

      "Game on!"

      An arms race that starts with malware AI versus detection AI and ends with... Skynet.

    2. Floydian Slip

      Re: Just link the virus checker to an AI

      Sophos already have....and the result is called Intercept-X v 2

      Welcome to AI wars

  2. Christian Berger

    Wasn't that already a thing in the 1990s?

    I mean back then people talked about polymorph malware. Essentially you'd move over the use of one register to another one, or you change the order of executions, or you encrypt it.

    So this is nothing new.

    1. Old Shoes

      Re: Wasn't that already a thing in the 1990s?

      Yes, but this has AI. It's the buzzword recycle:



      Artifical Intelligence <--- you are here

      Intelligent Agent



      1. Anonymous Coward
        Anonymous Coward

        Re: Wasn't that already a thing in the 1990s?

        Openness isn't part of the cycle. It was an outlier and mistake which the corporations will not allow to happen again.

    2. Anonymous Coward
      Anonymous Coward

      Re: Wasn't that already a thing in the 1990s?

      It may be nothing new, but the process has been massively optimised and indeed industrialised using this approach.

  3. Anonymous Coward

    So it's just like "The Matrix" then...

    Programs hacking programs...

    1. anonymous boring coward Silver badge

      Re: So it's just like "The Matrix" then...

      But will the battle look like Kung Fu?

  4. Lee D Silver badge

    Exactly what I've been saying for years.

    And the problem is that AV is nothing more than pattern recognition (at best! Most of the time it's nothing more than byte-matching!), and all you need to do is find a pattern that it doesn't recognise but that does what you want.

    I always laugh when people talk about antiviruses as something that works like an innoculation - hunting down viruses and removing them, when in reality it's more like a bouncer on a nightclub door with strict instructions to only let named people in. And it just asks people their name, if they are on the list, they are allowed in. No verification. No ID. Not even clever enough to spot similar-but-different names to those on the list, etc.

    The days of polymorphic viruses showed us this, with encryption etc. there's no reason to suggest that AV can ever keep up whatsoever. The only secure method is to run whitelist - literally only allow THESE PROGRAMS on the network, everything else can go fish - and nobody does.

    Believe it or not, most of AV is reverse-engineering. Someone has to sit with a VM, work out how the virus operates, what parts of it change, etc. which is how they come up with those (useless) reports of what registry entries it touches, etc. - they run it and record what they see changing, not what it's capable of changing. Only in extreme circumstances do they bother to delve into it deeper and see how it actually works (e.g. the very-public ransomware).

    Because it's the work of a moment to make a program that makes a copy of itself, encrypted with a different public key, and using an off-the-shelf library to decrypt itself on run, which makes the AV companies either do some serious reverse-engineering or mark that library code as the virus. This is why AV tries to unpack UPX executables, etc. because it "knows" about them and wants to see what's actually being run, but in truth their signatures can never take account of all possible variations, with all possible schemes of obfuscation.

    Try it on You can make a malicious program that passes every AV vendor's software in about 20 minutes, all you need is a C compiler, a bit of programming knowledge, and something like that website to test it against. Automating the process via genetic-algorithms (which is what this sounds like, not AI), random variation, or even just choosing one of a set number of ways of performing each base action the program wants to perform, and you can walk past any AV and still take over the machine. Hell, compiling it with a different version of the compiler, or compiler options, will usually change it so much that AV won't recognise it.

    And "heuristics"? Yeah, you know what that word means, right? A set of rules to check against. Does it contain the "Format Drive C:" command, does it try to load the function at the fourth ordinal of this system DLL? That's an heuristic. And you can defeat such things very easily with a tiny bit of obfuscation.

    1. oldcoder

      The only way to stop a virus is


      Nothing else.

    2. herman Silver badge

      No, it is like a bouncer with a list of people NOT to let in. Anyone who gets refused, just rattles off random names, till he he is let in.

      1. Charles 9

        While the really, REALLY smart ones get those on "The List" to vouch for them. Or manages to steal the credentials of someone on "The List" and pose as them. That's why even whitelists aren't perfect (think Return-Oriented Programming which relies on pieces of whitelisted code to work).

  5. frank ly


    "... lots of tiny tweaks that proved very effective at developing malware that could evade security sensors."

    What sort of malware are they talking about here? I can't imagine that these 'tiny tweaks' are random or we'll have a monkeys and shakespear situation (maybe we have)? Was it actually genuine useful/dangerous 'malware' or just something that messed up registers, etc?

    1. patrickstar

      Re: Details?

      I assume it should say 'random changes that maintain the same functionality'.

      And you can actually have entire programs 'written' by an evolutionary algorithm. Add random code, run it, see if it does something closer to what you want the end result be, try again if it doesn't.

      It just takes a lot of time. I recall that this is how "Hello World" in Malbolge was written.

  6. Tessier-Ashpool

    What's the answer?

    I can see a day when operating systems are mandated to run only signed and hashed executables, with the signing attributed to a legal entity such as an established company or human.

    1. Lee D Silver badge

      Re: What's the answer?

      It's called Software Restrictions.

      It's been in Windows since 2003-ish?

      It wouldn't work on home machines. People just accept warnings over unsigned drivers, etc. "because the instructions said so".

    2. oldcoder

      Re: What's the answer?

      And hope the keys never leak.

      And hope the hashing never collides.

      And hope the validation step never gets hacked.

      Not all that secure...

      Don't forget, Even Windows RT was hacked to run Linux. And that is what it used.

    3. artem

      Re: What's the answer?

      There are better ways to fight malware which also allow you to run any software you want/need. For instance like it's done in iOS/Android - every app runs in its own sandbox. Of course, such apps can interact with the kernel and penetrate it but it's relatively rare and it could be fixed fast. When that's not enough, you can run every app in a VM (but that's not a complete panacea since hypervisors also contain vulnerabilities).

      And if that's not enough for you, you can run a potentially hostile app in a VM which runs on a separate PC in a separate network segment while you can access this VM only via RDP/VNC which is 100% secure.

      1. Anonymous Coward
        Anonymous Coward

        Re: What's the answer?

        No, because what if the malware is remote-aware and manages to monkey with the protocol enough to pwn the viewer program to pwn the client machine. Same for the network segment: pwn the other end to bridge the segments. Heck, a truly determined adversary will find ways to exploit Sneakernet, meaning it can get past air gaps. If you can get past an air gap, you can get past nigh ANYTHING.

    4. Anonymous Coward
      Anonymous Coward

      It would work on home machines

      If there wasn't a way to click through the warnings. The trick is getting people to accept it. Apple has got iOS users to accept it, but there was no legacy iOS software to worry about. It is a lot easier to start fresh than to force a transition.

      Microsoft essentially tried this with Windows 8, if they hadn't tied the signed software to that horrible interface maybe they would have got people to buy in on it, but few people want a touch-first GUI on a home computer with a 24" screen.

  7. Anonymous Coward
    Anonymous Coward

    Nothing new here ....

    Now the fact that I could pull that out of thin air, whilst those around me stroked beards and said "totes amazeballs" is why the youngest are not necessarily the brightest.

    So that's *my* CEO immunised against the inevitable snake oil.

  8. John Smith 19 Gold badge

    Now if only this was applied to the development of development support tools.

    Something capable of both binary object recognition and generation.

    But what to call it?

  9. Doctor Syntax Silver badge

    "The key to the system is to take legitimate-looking code and change just a few tiny parts of it to convert the software into attack code. Even changing small details can fool AV engines, he said"

    There's something self-contradictory here.

    Start with something legitimate. Make small changes. Small changes can fool AV engines. But if the AV engine were white-listing the legitimate code than those small changes should fool the white-listing. And if you weren't counting on white-listing why bother to start with legitimate-looking code in the first place?

    1. herman Silver badge

      AV code is not white listing. AV code is black listing. It is totally idiotic and quite useless.

      The only solution is to fix the goddam bugs and not to create so many of them in the first place, but MS doesn't want to spend money on writing and maintaining good quality SW.

      1. Doctor Syntax Silver badge


        Agreed. But in that case why the starting point of apparently legit code? It looks as if its an attempt to pass the malware off as known good code and it's only white-listing that that would work on. That's what makes the report so odd.

      2. Infernoz Bronze badge

        Indeed, white listing is a critical part of good security, unfortunately someone/something needs to build and maintain the list; that's fine for a closed system, but probably impossible with all the valid non-corporate, unsigned software flying around, so like Android and iOS, programs must declare their what they need to access, and not just be escalated for coarse undefined behaviour, and the OS must enforce this and even limit/block some requested access, something Android should damned well allow non-admin users to do!

        The problem with bugs is you can only fix them after you have identified them, and some can be very subtle or caused by "code blindness".

        1. Charles 9

          Or they're "gestfaults" (gestalt faults--worse than the sum of their parts) that involve how different pieces of code interact with each other. Each piece works fine on its own, but the bug ONLY appears when they're together, so it's very difficult to spot.

    2. Charles 9

      "Start with something legitimate. Make small changes. Small changes can fool AV engines. But if the AV engine were white-listing the legitimate code than those small changes should fool the white-listing. And if you weren't counting on white-listing why bother to start with legitimate-looking code in the first place?"

      Perhaps they're trying something akin to Return-Oriented Programming which can use whitelisted programs to wreak havoc.

  10. Anonymous Coward
    Anonymous Coward

    My car GPS can do that already....

    Basically the AI has been fed signatures on how various virus scanners detect a particular virus. Then the "AI" changes the virus patterns so it has a high likelyhood of bypassing the virus scanner on the target system.

    My car GPS has been fed the map of my area. If i came to a roadblock the GPS can navigate around it. Now throw in the words AI and Elon Musk and you have an article on the Reg.

  11. Triggerfish

    I felt like a punk who’d gone out to buy a switchblade and come home with a small neutron bomb.

    Screwed again, I thought. What good’s a neutron bomb in a streetfight? The thing under the dust cover was right out of my league. I didn’t even know where to unload it, where to look for a buyer. Someone had, but he was dead, someone with a Porsche watch and a fake Belgian passport, but I’d never tried to move in those circles. The Finn’s muggers from the ’burbs had knocked over someone who had some highly arcane connections.

    The program in the jeweler’s vise was a Russian military icebreaker.

    1. MacroRodent

      "I felt like a punk ..."

      Attributing the quote would be good form. Looks like that came from William Gibson's "Hackers".

      1. patrickstar

        Re: "I felt like a punk ..."

        Uhm, it's Gibson alright... but it's from Burning Chrome.

        (I initially thought it was from Count Zero due to the characters and story, but it's not written in first-person perspective).

        1. Triggerfish

          Re: "I felt like a punk ..."

          Apologies, wasn't sure of form see it both ways on here. And never know whether people prefer knowing where it's from or the puzzle going "hmmmn I know that, whats it from?"

          And yep Burning Chrome. Did Gibson write Hackers?

  12. Greywolf40

    AI now, metal-based life forms soon

    AI will always have blind spots, but the really scary part is that we can't tell how they do whatever it is they do, so we can't tell where the blind spots are. It looks like metal-based life forms will be as error-prone as carbon based ones. Star Trek's Data is impossible.

    1. Charles 9

      Re: AI now, metal-based life forms soon

      Is there a way to prove your claim that AI will always have intractable blind spots?

  13. B Bunter

    Neural net they say. Connect my brain to the cloud they say.

    No thank you not until there is guaranteed, 100% security and privacy.

    1. Charles 9

      Re: Neural net they say. Connect my brain to the cloud they say.

      As long as humans are involved, you can't have 100% security and privacy even WITHOUT the cloud. You have to take a risk at SOME point.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like