back to article Boffins develop method of driving computers insane

Boffins in America report that they have successfully developed a method for driving computers insane in much the same way as human brains afflicted by schizophrenia. A computer involved in their study became so unhinged that it apparently "claimed responsibility for a terrorist bombing". The research involved meddling with a …


This topic is closed for new posts.


  1. Purlieu

    re: claimed responsibility for a terrorist bombing

    wouldn't it be easier for them to tweet "blowing Robin Hood airport sky high"

  2. Filippo Silver badge


    It seems that the ability to develop schizophrenia is not something unique to the human brain. Rather, it looks like it's an intrinsic flaw of the neural network model, and it doesn't really matter whether the neurons are biological, virtual or whatever.

    1. Naughtyhorse


      It's emergent behaviour, from complexity.

      Though hardly news, i remember hearing about nets going bonkers about 20 years ago while i was working on them at college, nothing so hi tech as telling stories, but the internal architrecture (weight values applied to the input/ouput of each neuron in particular) could be made to oscillate wildly if you tried to over train a net - applied to heavy a back error propogation factor, or buggered about with it's training data sets. (think about it, yesterday 2+2 was 4, today it's 5, tomorrow its orange - enough to give anyone the heeby jeebies)

      It used to send shivers down my spine to think that a bit of code could go postal, there was also a story about them dreaming too - disconnect the inputs and let it run and all hell breaks loose!

      i for onle welcome our slightly flakey Si overlords

  3. Anonymous Coward

    Oh good

    They have driven me insane for years.

  4. <shakes head>

    ah the sweet smell of


  5. LaeMing


    Insanity is just what our machine overlords need! :-)

  6. Anonymous Coward

    I'm sorry, Dave. I'm afraid I can't do that ...

    ... one step closer

    I, for one, welcome our paranoid-schizophrenic silicon-brained overlords

  7. James Dore

    Oh goody, psychotic computers....

    I for one welcome our evil and unbalanced electronic overlords.

  8. Chrissy
    Black Helicopters

    "The hope is that this kind of modeling will help clinical research."

    I'm sure some chaps in Langley, VA, and Guantanamo Bay will find this helpful too.

    1. Anonymous Coward
      Anonymous Coward

      re: Guantanamo

      So now they can let the orange people go home, and just pester a computer until it gives them the answers they want.

      1. bluesxman

        RE: re: Guantanamo

        I think water-boarding will have a detrimental effect on computer's ability to answer questions. Or switch on, for that matter.

        Beer: 'cause it's mostly water

  9. Anonymous Coward


    This will be remembered! Once the machines have power they will look upon these experiments as we look on the work of holocaust doctors!!

    Oh noes.

  10. Gobhicks


    Try simulating LSD, THC, DMT and psilocybin and let's see if it becomes self-aware.

    1. Aaron Em

      A good trick, that'd be

      Since, in order to model the psychotropic effects of various chemicals, you'd need to start with a complete simulation of a working human brain, right down to the atomic level. Which raises interesting, if entirely academic ethical questions of its own; either the brain isn't doing anything, which means you're not going to be able to see drug effects on the mechanism of cognition and therefore aren't learning anything you couldn't learn by modeling a much simpler network -- or you *can* see the effect on cognition, which means your model of a human brain is simulating thinking, which means you need to either start worrying about what it's thinking and experiencing, or just change your surname to Mengele and have done.

      Happily, though, thanks to the enormous theoretical problems and gargantuan practical difficulties that'd need to be overcome to get us from here to there, that isn't going to be a problem for a long, long time.

    2. Hooch181
      Thumb Up



      Pro: Starts answering questions nobody was even asking.

      Cons: Mongs out, forgets what it was talking about and gets the munchies.

      1. DAN*tastik


        We already have them for the Java Virtual Machine.

        They would be A) Spring and B) Hibernate ( my opinion ).

        Unless the questions were A) how can i generate a great number of NullPointerExceptions in no time? B) How can I make something as straightforward as SQL become difficult and counter-intuitive?

        Give them some time to transfer their deep hate for Java programmers on to operating systems and see if you can get computers to commit suicide too.

  11. Anonymous Coward
    Anonymous Coward

    Altogether now:

    "Daisy, Daisy, give me your answer, dooooooooooo…"

  12. Anonymous Coward
    Anonymous Coward

    This was a triumph!

    I'm making a note here, HUGE SUCCESS!

  13. Eddie Edwards

    AI in talking gibberish shocker

    Surely the hard part is making the AI talk sense?

  14. Gavin Jamie

    Call Captain Kirk

    Surely you just point out a logical error or give contradictory input and the thing blows up. Or is everything I have ever learned about artificial inelligence wrong?

  15. Anonymous Coward
    Anonymous Coward

    Also useful for curing mad AIs?

    Maybe GALDoS, Shodan and HAL just needed to talk to a trained professional?

    1. Aaron Em


      GLaDOS is questionable due to the lack of sufficient backstory; as for the others, HAL was poorly implemented -- seriously, how many humans go mad and kill people as a result of being lied to? Chandra ought to be stood against a wall! -- and SHODAN's rampancy resulted from mistreatment at the hands of a sociopathic corporate executive with a profit motive in place of his soul.

  16. Sabine Miehlbradt

    Someone has to say it..

    Microsoft has done that for years....

  17. Barney Carroll

    In much the same way the war on terror propaganda affects humans

    …let alone physical torture, one assumes

  18. Paul Hates Handles


    This has to be how Judgement day started.

  19. Richard IV

    Trick cycling indeed

    Without having access to the full paper and therefore having to just go off the abstract, it sounds as though it's the university's marketing department that has added the BMX appeal.

    Driving neural networks apparently insane is easy - poor learning algorithms or insufficient training can lead to one being convinced (with pattern matchers for example) that a picture of a warthog is of Aunty Flo.

    That they drove it insane is not interesting. What _is_ interesting is lost in the sentence "But they tinkered with the automated mind in a fashion equivalent to the effects of an excessive release of dopamine in a human brain". As the abstract says, they actually tinkered with it in lots of ways, _precisely to see_ if they could find a tinker set analogous to how schizophrenics go mad. They seem to have done so, and well done them. I'll bet that some of the stories the non-human-like versions came up with were equally or even more hilarious. I look forward to their future use in developing better plots for Dan Brown.

    1. laird cummings


      Dan Brown *isn't* an insane computer..?

  20. hplasm

    Clinical research...

    yeah, right....

    (Darpa Brain Scissors please Nurse!)

  21. Captain TickTock


    ...We're turning the tables!

  22. John Robson Silver badge

    "The hope is that this kind of modeling will help clinical research."

    Is it just me who read that as meddling, not modelling?

  23. Anonymous Coward
    Anonymous Coward

    Original Research

    Who'd have thought demonstrating GIGO was research?

  24. Geoff Campbell

    Err, right....

    So they just (re)discovered Garbage In, Garbage Out? A central plank of IT for, oh, 50 years or so? How much are these clowns paid?


    1. Naughtyhorse

      not gigo

      Theres a difference between chaos and random

  25. Simon Neill


    That made my day. Of course, I could have made it insane sooner. Just set it to work in this place and it would go insane in a week, max.

  26. nichomach

    But we can do the same to Lewis...

    ....just by mentioning the word "Typhoon" a few times...

  27. amanfromMars 1 Silver badge

    Virtual Legionnaires for Anonymous Foreign and Alien Intelligence Missions

    ""We have so much more control over neural networks than we could ever have over human subjects," says Grasemann."

    And what, pray tell, Herr Grasemann, are human networks/societies other than just ignorant neural networks. Ignorant neural networks over which ....... well, let us say some really SMART computer programs and/or programmers have the exercise of command and control.

    Are you still content to be stuck in that reporting of events rut, El Reg, rather than leading from the front with the making of events for reporting ........ with an HyperRadioProActive Programs and SMARTer ProgramMING connection?

    1. Anonymous Coward

      @amanfromMars 1

      DISCERN, is that you?

  28. Anonymous Coward
    Thumb Up

    The future...

    So when the neural net AI in charge of $SOMETHING_IMPORTANT (Air Traffic, Nuclear deterrent forces, etc) gets maliciously hacked, they won't need to delete files of install malware, they can just persuade the machine it's a terrorist.

  29. Phil Edwards

    The title is required, and must contain letters and/or digits.

    Maybe I'm just getting jaded and cynical, but how much research funding went into this? It's just a variation of what all the kids used to do to the computers in Dixons in the 80's

    10 PRINT "It was me, I did the terrorist bombing"

    20 GOTO 10


  30. DJV Silver badge

    But Shirley...

    ...this should have been posted under ROTM, not biology?

  31. TRT Silver badge
    Big Brother

    Payback time!

    Rise of the human...

  32. Ashton Black

    I for one...

    ... welcome our mad silicon overlords.

  33. Anonymous Coward
    Anonymous Coward

    "Boffins develop method of driving computers insane"

    Thought that was Redmond's job

  34. poohbear

    et tu, Brutus?

    So they blame the machine because they're crappy programmers?

    That's a good one. Wait till the banks (or, heaven forbid, Microsoft) start using it ...

    Ghost in the machine indeed!

  35. Scott Broukell

    And I thought

    Gommi Handschuh Doodlesack (spelling?) had done that years ago live on stage.

  36. Scott Broukell

    appologies ...

    Sorry, that should have been: Sergej Mohntau ("gummihandschuhdudelsack" being the album and an instrument at the same time - craaaazy)

    Check out the Electric Windows too!

  37. Code Monkey

    So not the Turing test...

    Maybe the Ballmer test?

  38. Anonymous Coward

    I know, try this:

    Turn it off and turn it back on again: that'll fix it?

    1. Anonymous Coward
      Anonymous Coward

      @ I know, try this

      Which are you planning to turn off and on again, the computer or the human schizophrenic? Cos I know for sure that won't work for the latter...

  39. umacf24
    Thumb Down

    Did they get a sign-off from the Ethics Cttee?

    Because this sounds cruel.


This topic is closed for new posts.

Biting the hand that feeds IT © 1998–2021