back to article How to steal the mind of an AI: Machine-learning models vulnerable to reverse engineering

Amazon, Baidu, Facebook, Google and Microsoft, among other technology companies, have been investing heavily in artificial intelligence and related disciplines like machine learning because they see the technology enabling services that become a source of revenue. Consultancy Accenture earlier this week quantified this …

  1. Anonymous Coward
    Anonymous Coward

    See I was worrying about AI taking all the jobs.

    But if Accenture is in charge of making it work, I think we are all safe after all. Well, maybe except for the wallets of the big A's customers.

    1. Terry Cloth
      Devil

      But when AIs get too smart...

      ...won't they be subject to social engineering? Consider this article as a trial run.

      1. Anonymous Coward
        Anonymous Coward

        Re: But when AIs get too smart...

        Well, Microsoft's Tay was an example of a low IQ AI being turned into a racist Nazi. Sort of like how some low IQ humans raised in a racist Nazi environment become racist Nazis.

        1. Destroy All Monsters Silver badge
          Trollface

          Re: But when AIs get too smart...

          Hmmmm... it would have been cool to groom it into a non-racist, cultured Nazi, playing violin, listening to Schubert, reading Goethe etc. etc.

        2. P. Lee

          Re: But when AIs get too smart...

          >Well, Microsoft's Tay was an example of a low IQ AI being turned into a racist Nazi. Sort of like how some low IQ humans raised in a racist Nazi environment become racist Nazis.

          Ah the conundrum. Can you have, or would you want, intelligence without morality. How do you balance efficiency with morality?

          Assuming you did manage to program a simulation of morality as well as a simulation of intelligence, what would it think of your commercial decisions?

          Hal, open the bay doors.

          Though I doubt we'll ever get to that stage, despite having seen Star Trek. This will mostly be used to look for copyright infringements where people have used the same three notes that someone else used fifty years ago... in the same sequence!

          1. Anonymous Coward
            Anonymous Coward

            Re: But when AIs get too smart...

            What is morality?

            Morals are all subjective, what insults one person is normal to another, what one person finds completely unacceptable behaviour is a lifestyle for another.

          2. Anonymous Coward
            Anonymous Coward

            Re: But when AIs get too smart...

            "Can you have, or would you want, intelligence without morality."

            They already exist - they're called psychopaths.

        3. Dan 55 Silver badge

          Re: But when AIs get too smart...

          As with many of these things which are breathlessly announced as AI, it wasn't an AI, it was just a glorified Eliza program parroting words back.

  2. beast666

    Just make AIs that are too intelligent to be reverse engineered like the guy says.

    1. TRT Silver badge

      Unless...

      you study the AI by using an AI.

      1. You aint sin me, roit
        Mushroom

        Re: Unless...

        Robot wars!

      2. Captain DaFt

        Re: Unless...

        "you study the AI by using an AI."

        Yo Dog! I heard you like AI, so I put my AI in your AI!

        (Now they call themselves Skynet!)

        1. MrT

          Re: Unless...

          ... the AI reaches Transcendence. And looks like Johnny Depp.

          1. fajensen
            FAIL

            Re: Unless...

            ... and have molecular nanotech, is fully capable of raising the dead and yet is still no match for a bunch of a few hundred cave-dwelling American Taliban with some trucks and exactly 1(!) field gun.

            1. Anonymous Coward
              Anonymous Coward

              Re: Unless...

              Ayayayayay!!!

              (All in Spanish now)

      3. John Brown (no body) Silver badge

        Re: Unless...

        "you study the AI by using an AI."

        So you set it up to lie every now and then.

        Oh...wait....that didn't work so well with HAL.

  3. atakante

    BigML's Response

    BigML has responded to the author of this article within 24 hours of his inquiry. We are in the process of helping correct the omission. However, interested parties can find the highlights of our response here: https://blog.bigml.com/2016/09/30/hype-or-reality-stealing-machine-learning-models-via-prediction-apis/

    1. Destroy All Monsters Silver badge
      Windows

      Re: BigML's Response

      Pretty sure this was posted by a AI-enhanced automatic commentator herding system.

  4. Destroy All Monsters Silver badge

    Consultancy Accenture earlier this week quantified this enthusiasm, predicting that AI "could double annual economic growth rates by 2035 by changing the nature of work and spawning a new relationship between man and machine" and by boosting labor productivity by 40 per cent.

    Shit's ludicrous. "Annual economic growth rates" ... don't they hover at 0.001, only kept up by causally-closed-loop financial engineering, wealth extraction via wars in foreign lands and Quantitative Easing while the growth-enabling infrastructure in knowledge, tools, savings and people decays?

    1. amanfromMars 1 Silver badge

      Crazy Shit happens Anywhere and Everywhere when Fools Use and Abuse and Misuse Effective Tools

      You hit the nail hard, right square on its head there, Destroy All Monsters. AI EMPowered Virtual Machinery/Virtually EMPowered AIMachinery sees right through the delusional ramblings of businesses acting as if human-caring. The reality is that they be deep and dark web mining and meta data base pumping and dumping sub prime product and programming.

      1. Anonymous Coward
        Anonymous Coward

        Re: Crazy Shit happens Anywhere and Everywhere when Fools Use and Abuse and Misuse Effective Tools

        The company that builds an AI that works as a

        1) Management Bullshit detector

        2) Management lie Detector

        and

        delivers appropriate punishment in the form of an electric shock

        Would make a killing (metaphorically, financiall but not physically)

        Then the jerks who speak out of their backsides would get their comeuppance.

        And the world would be a better place (for a while)

        1. amanfromMars 1 Silver badge

          Re: Crazy Shit happens Anywhere and Everywhere when Fools Use and Abuse and Misuse Effective Tools

          A working AI is also inevitably to be a superbly physical killing machine, AC, but working and targeting competition and opposition organising heads from the top down rather than wasting time and resources on bottom feeders. So it will be elitist.

          1. Steve Davies 3 Silver badge

            Re: Crazy Shit happens Anywhere and Everywhere when Fools Use and Abuse and Misuse Effective Tools

            Top down culling?

            So you have never has a PHB as your boss?

            Never has a CEO talk absolute crap to the workers about 'your job is safe' because you know full well that the company taking your employer over only wants the IP and will move the jobs offshore ASAP?

            etc

            etc

            etc

            A bad boss (As evidenced by what is happening to HP) can drive a company into the dust yet all the time, the CEO (and 'C' level execs) are spouting forth total rubbish about the way forward for the company.

        2. Voland's right hand Silver badge
          Devil

          Re: Crazy Shit happens Anywhere and Everywhere when Fools Use and Abuse and Misuse Effective Tools

          Then the jerks who speak out of their backsides would get their comeuppance.

          Be careful what you wish for. This will unravel a near decade of pyramid scheme investments in the management space - Customer First, etc. They all work exactly the way MLM fraud scheme work - all the new acolytes brought on board down to the lowly team lead level manager which is getting its "Customer First" training now need to purchase (actually company buys it for them) training, next level buys training, and books, consluttants regularly buy training, books, etc - all funneling it to the man at the top.

          Even if an idea is actually fundamentally correct - like agile, devops, in the valley it framed into a MLM in less then 6 months and the mark fleecing begins in earnest.

          If we apply your proposed machine, the pyramids will unravel. These schemes constitute up to 30% of all economic activity in the Valley. Blowing up 30% of it will unravel the whole economic tower of cards which is the West Coast today.

          So be careful what you wish for.

          1. Steve Davies 3 Silver badge

            Re: Crazy Shit happens Anywhere and Everywhere when Fools Use and Abuse and Misuse Effective Tools

            Be careful what I wish for?

            Can the 'Big One' [1] strike today or tomorrow?

            The huge Earthquake that is well overdue. That will focus the minds of bosses on survival rather than the management merry-go-round where they decimate a company, move on to another and rinse and repeat and rinse and repeat. They only seem to know how to run a business into the ground and don't even try to grow them.

        3. Hans 1

          Re: Crazy Shit happens Anywhere and Everywhere when Fools Use and Abuse and Misuse Effective Tools

          Management is not really that easy, you are in a vise with your team on one side and upper management at the other. The best managers are never promoted to manager, because, well, upper management was servile cretins that really believe ANYTHING upper management says ...

    2. Dan 55 Silver badge
      Terminator

      That's a nice way of describing the Terminator-style future that awaits us all.

      In Accenture in 2035, will T800s be sitting round a long rectangular boardroom table while one stands in front presenting powerpoint slides? At least the fixed grins will be nice for the stock photos.

      1. Anonymous Coward
        Anonymous Coward

        Hey, I am for Diversity: We need some T-X's in there. With Tits. Big Tits. Maybe Triple Tits.

  5. Destroy All Monsters Silver badge

    In further news

    IEEE AI & Ethics Summit 2016: Artificial Intelligence and Ethics – Who does the thinking? Tuesday 15 November 2016. The Sofitel Brussels, Europe. Place Jourdan 1, Brussels, 1040, Belgium

    Maybe Andrew could be sent to cover this, this would keep him busy and avoid rage attacks about Google's "stealing" of the JDK API.

  6. Anonymous Coward
    Anonymous Coward

    To understand people you need artificial intelligence.

    To understand Nature you need real intelligence.

    1. Wensleydale Cheese

      "To understand people you need artificial intelligence.

      To understand Nature you need real intelligence."

      "Until man duplicates a blade of grass, nature can laugh at his so-called scientific knowledge." -- Thomas Edison

      1. Anonymous Coward
        Windows

        Has Anyone Really Been Far Even as Decided to Use Even Go Want to do AI Like?

        We will have to wait for working nanotech for this grass.

        Meanwhile, we are not talking about "AI" here, just Machine Learning. Which is not that new although evidently new theoretical knowledge and massive computing power making iterative practical application feasible have been added. (I used to teach intro courses from some expensive book on Machine Learning back in the early 90s, I think from the "Morgan Kaufmann Series in Machine Learning")

        (And I hear that germans like to call this "Avant-Garde Informatics" to better convey that it is just the wave, not a goal, and may have scant to do with Intelligence at all. Like, you know, LISP was once considered AI...)

        Amazon.com has 7,685 results for "Books : Computers & Technology : Computer Science : AI & Machine Learning". Everybody must be piling in, with Python, Dummies and whatnot.

        As a bonus, a review of "Weapons of Math Destruction".

        1. P. Lee

          Re: Like, you know, LISP was once considered AI...

          Ah yes, the eternal problem, "am I my own brother?"

          1. Anonymous Coward
            Anonymous Coward

            Re: Like, you know, LISP was once considered AI...

            Ah yes, the eternal problem, "am I my own brother?"

            Or if you're from Norfolk, your own uncle/cousin/sister...

          2. Anonymous Coward
            Anonymous Coward

            Re: Like, you know, LISP was once considered AI...

            > Ah yes, the eternal problem, "am I my own brother?"

            I watched that anime!

  7. a_yank_lurker

    Epic Failure

    The insultancy Accenture thinks generic AI is going to be the next hot thing. Most AI advocates seem to ignore that to raise a well functioning adult requires careful training and teaching of a child for extended period of time. True an AI system will likely be faster but the key is the system training and teaching.

    Slurp put out an AI system a few months ago which was quickly taught to be more racist than your average KKK member. Though I suspect the "trainers" were doing this to show how difficult the training is and how easy it is to corrupt and to embarrass Slurp.

    Also, the various shills and insultancies ignore that most "AI" they cite are very limited functionality cases with very narrow domain knowledge and rules. What is being done is automating a tedious procedure, like fraud detection, to bring greater speed and initial accuracy. Fraud detection systems, while useful, still require verification that the system actually detected fraudulent activity. They are prone to false positives and I suspect false negatives.

    1. Rich 11

      Re: Epic Failure

      It's like anything: if a knowledgeable human being isn't reviewing the data regularly, looking for patterns, the software could be running wild and screwing up your customers without them even knowing. That works just as much for an AI as for what the latest set of outourced HR drones are typing into your database, or software rejecting job applications because a programmer made a typo, forgot about differences in Englishes, or effed up a condition.

      1. Korev Silver badge
        Trollface

        Re: Epic Failure

        forgot about differences in Englishes

        Or mistakes as we call them in Britain.

  8. Mage Silver badge

    AI reverse enginering

    Could it be it's not really AI, but a fancy database?

    1. Destroy All Monsters Silver badge

      Re: AI reverse enginering

      Not a fancy database. A COMPRESSED database, from which one can then generalize.

  9. Anonymous Coward
    Anonymous Coward

    Artificial FEMALE Intelligence

    Nobody will ever figure THAT out...

    1. Anonymous Coward
      Anonymous Coward

      Re: Artificial FEMALE Intelligence

      When you switch on the superpowerful AI, there will be a earth-scattering screech as if a million feminists had suddenly blamed the patriarchy for the unfairness of life, the universe and everything.

      You are then transported into an "Aggressive Menace"/"I have no Mouth but I must Scream" kind of situation.

      1. amanfromMars 1 Silver badge

        Re: Artificial FEMALE Intelligence

        It is a catastrophic arrogance and massive vulnerability, easily exploited by the sympathetic and empathetic to the female mind, to imagine that it is subservient and weak rather than strong and leading?

        And if/when artificial is virtualised and realised as advanced and alien too, is it quite something else to marvel at and worship.

        And such is terrifying to lesser in species.

  10. EarthDog

    GIGO

    That's the big problem I always had with mindlessly applying things like ML and data analytics, garbage in garbage out. If you lie to your AI engine it will not learn properly. Also, it is why non-ACID compliant database engines enrage me. But that's another rant.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like