back to article Humanity can defeat SkyNet with BOOKS, says IT think tank

A group of researchers working for National ICT Australia reckons computer science courses need to look at artificial intelligence from an ethical point of view – and the popularity of sci-fi among comp.sci students makes that a good place to start. As the research team, which included NICTA's Nicholas Mattei, the University …

  1. Anonymous Coward
    Anonymous Coward

    Yeah, right

    About as effective as us defeating the Surveillance State by reading 1984 and the Animal Farm.

    1. MacroRodent

      Re: Yeah, right

      About as effective as us defeating the Surveillance State by reading 1984 and the Animal Farm.

      But reading those books are a good start towards that goal! Add also "The First Circle" by Aleksandr Solzhenitsyn (which also includes a good demonstration of how not to manage a R&D project).

    2. Martin Budden

      Re: Yeah, right

      Thing is, most people haven't read 1984 and Animal Farm, which might have something to do with their acceptance of "the Surveillance State".

      1. oolor
        Big Brother

        Re: Yeah, right

        > most people haven't read 1984...

        And the ones that have seem to have taken it as a blueprint for ruling society.

      2. Michael Wojcik Silver badge

        Re: Yeah, right

        Thing is, most people haven't read 1984 and Animal Farm, which might have something to do with their acceptance of "the Surveillance State".

        To be fair, Animal Farm has little to say about surveillance.

        One problem with Nineteen Eighty-Four1 as an object lesson is that it depicts a totalitarian police state. That makes it apropos for, say, North Korea or the former East Germany; but the most successful contemporary surveillance states are the liberal-capitalist ones, which prefer the carrot2 to the stick.

        For those societies something like Brave New World rings closer to home, though frankly it shows its age. I can't think offhand of a really ideal text for this purpose.

        1Orwell hated the title being set in digits. He didn't like the title at all - it was forced on him by the publisher - but he particularly didn't like the semiliterate rendering of it as numerals.

        2And more generally the subjective operations of power, such as interpellation, the management of desire, foreclosure, &c.

  2. Brian Sherwood Jones

    more titles

    Michael Frayn, 'The Tin Men', 'A Very Private Life'

    John Sladek; many but especially 'Tik Tok'

    John Wyndham 'Compassion Circuit'

    Manuel De Landa 'War in the age of intelligent machines'

    Stanislaw Lem 'Summa Technologiae

    Lots of movies - 'Robot and Frank' is good.

    Lots more to be found at

    Do projects with

    Fun, but agree with above, unconvinced of real value.

    1. Robert Helpmann??

      Re: more titles

      Have an upvote for 'Robot and Frank.' I think that, to carry the '1984' example a bit further, it is worth keeping in mind the examples given by the various dystopian works. Blade Runner (DADoES) gets my vote for that.

      1. Triggerfish

        Re: more titles

        If we are going to go with Dystopia I always think we move closer to the future of William Gibson than Orwell.

        1. Anonymous Coward
          Anonymous Coward

          Re: more titles

          If screen plays can apply, I also nominate the Matrix, Neuromancer was good too.

          1. Michael Wojcik Silver badge

            Re: more titles

            If screen plays can apply, I also nominate the Matrix

            I cannot imagine why. If ever there was a pseudo-intellectual waste of time, The Matrix is it. Its only clear ethical statement comes at the end and is nothing more than "eh, fuck 'em".

    2. Beau

      Re: more titles

      I find it difficult to understand why, Asimov who wrote so much about robots, has not been mentioned yet.

      Or is his idea of robots just too pro AI/robots?

      1. Jonathan Richards 1 Silver badge

        Re: more titles and Asimov


        I think it's because his First Law is in direct contradiction to much of what some organisations want robots for.

        1. gazthejourno (Written by Reg staff)

          Re: Re: more titles and Asimov

          I think his first and "zeroth" laws are fundamentally the most important.

          They'll also get completely ignored. Robots have far too many tempting features for authoritarian and quasi-authoritarian regimes to ignore.

          In addition, who defines "harm"? Are you "harmed" if your robot hands over evidence to the authorities of you driving at 78mph at 2am on the M1? Is humanity "harmed" if the entire Middle East is turned into a glass car park and repopulated with prison inmates from the West, a la Australia's settlement?

          Too many seemingly objective principles are subordinate to subjective judgments at their root.

          1. Little Mouse Silver badge

            Re: Re: Re: more titles and Asimov

            A subjective interpretation of the Three Laws meant that there would often be a grey area where they met or overlapped. Many of his Robot stories explored this issue.

            Perfect for Ethics courses I would have thought, but perhaps less perfect for stopping Arnie in his tracks.

        2. Michael Wojcik Silver badge

          Re: more titles and Asimov

          Asimov's robot stories mostly deal with a fairly narrow ethical issue, vis-a-vis AI: how a simple deontological moral scheme can break down under messy, real-world conditions. They're excellent at raising problems in that arena and as intellectual exercise (as well as being good reads), but the various other texts mentioned so far generally avoid those sorts of simplifying assumptions in favor of poking at a big messy tangle of ethical problems.

          Asimov's kind of the pragmatist or logical positivist of AI ethical philosophers.

          That doesn't mean he shouldn't be included in the discussion, but it might explain why his name wasn't on the original list.

      2. Brewster's Angle Grinder Silver badge

        Re: more titles

        I find it difficult to understand why, Asimov who wrote so much about robots, has not been mentioned yet.

        Well most of his stories demonstrate how the rules fail.

        1. Mark 85 Silver badge

          Re: more titles

          Well, in that case maybe it should be required reading. Same for his Multivac stories. Rules are cut and dried. Ethics has a lot of gray areas.

  3. Khaptain

    Reading books - wats dat ?

    There might be soemthing in this although not from an AI perspective.

    Society is manifesting it's true colours via the likes of Facebook, Instagram, Twitter etc. Conversations are boiled down to 140 or so characters, tiny snippets of miselading information and images of cats...

    If society is to be saved then it has to re-start educating itself on something other than the constant stream of drivel upon which it currently feeds.

    1. Rustident Spaceniak

      Re: Reading books - wats dat ?

      That point is quite well fictionalized in Dave Eggers' The Circle. And yes, I do think all those people who read "1984" in the '80s were influenced by it, to a degree. Certainly, where I come from, it was considered almost mandatory reading, and was quite often cited in public discussion or in the press. Sadly, that has become less frequent.

      1. John Sturdy
        Big Brother

        Re: Reading books - wats dat ?

        Unfortunately, it seems to have become a manual for some sections of the public sector.

    2. Martin Gregorie

      Re: Reading books - wats dat ?

      Ray Bradbury had something to say about this too.

      See "Fahrenheit 451" with its thought police and the suppression of all fiction apart from Hemmingway-style realism and all entertainment apart from mindless 24 hour soaps.

      For those who can't or won't read there's a passable film version....

      1. MacroRodent

        Re: Reading books - wats dat ?

        "of all fiction apart from Hemmingway-style realism"

        Wasn't 451 the Bradbury story where all books were banned? (And burning them was the job of the firemen). He had another story where only non-realist fiction was burned... I think it was (google google) "Usher II" (part of the Martian Chronicles).

    3. This post has been deleted by its author

    4. Michael Wojcik Silver badge

      Re: Reading books - wats dat ?

      Oh yes, the same old complaint.

      Actual methodologically-sound research shows today's college undergrads do more reading and writing than any previous generation in history. And that includes a lot of books.

      But don't let facts get in the way of your rant.

  4. Crisp

    What ethical problems?

    See title.

  5. Anonymous Coward
    Anonymous Coward


    As long as we're talking Dick, I'd recommend Do Androids Dream. Because AIbots have feelings too.

  6. Steven 2

    Michael Marshall Smith mention

    I see to remember in Only Forward there's a wall that helps him out with by camouflaging him when being chased by the bad guys, largely because it likes what his suit looks like.

    +1 for welcoming our benevolent* robot overlord dictators.


  7. Elmer Phud

    Michael Marshal Smith

    toasters that talk, an alarm clock that follows you everywhere, sympathising with 'white goods' as no-one else really recognises thier worth (and thus getting the fridge to rat on other devices)

    Smith turns it in to everyday stuff, not 'shock horror' - I guess it's further down the line than a lot of stuff as the chatting between meat and machine is commonplace.

    Machines may have differing ethics than meat - but which is more 'human'?

    1. Anonymous Coward
      Anonymous Coward

      Re: Michael Marshal Smith

      That means the IoT is going to raise a helluvalot of new dilemmas.

    2. Anonymous Coward
      Anonymous Coward

      Re: Michael Marshal Smith

      "toasters that talk" I feel obliged at this point to mention "Talkie Toaster" from Red Dwarf :

      "Howdy doodly do. How's it going? I'm Talkie, Talkie Toaster, your chirpy breakfast companion. Talkie's the name, toasting's the game. Anyone like any toast? "

      Frankly I think I prefer the Terminators thanks, although I think Talkie is a much more likely scenario, along with the kind of products produced by the Sirius Cybernetics Corporation from Hitchhikers Guide to the Galaxy.

  8. Triggerfish

    Maybe the AIs

    will all bugger off to infinite fun space.

    1. Omgwtfbbqtime

      Re: Maybe the AIs

      Or become the Eschaton?

      "I am the Eschaton; I am not your God.

      I am descended from you, and exist in your future.

      Thou shalt not violate causality within my historic light cone. Or else."

    2. yoganmahew

      Re: Maybe the AIs

      Indeed, not sure how you can consider AIs without working backwards from an end-state, and I think Iain Banks provides that in spades.

      Not sure I'm ready, though, to be patronised by my bedroom...

  9. JeffyPoooh

    My recommendation...

    My recommendation for a good book that would help to address the generalized fear of AI:

    'The Theory and Practice of Circuit Breakers'

    Another good one is:

    'Kicking out Power Cords, for fun and profit'

    Not to mention:

    'Paintball - targeting HAL's big red glowing eye from 30m'

  10. Teiwaz

    Humanity can defeat Skynet with Books says Think Tank

    Clearly this 'think' Tank is drinking or smoking whatever grants it gets. its members certainly don't come round to the Reg much where the denizens have been armed with this type of knowledge and throwing it out there while figuretivly ducking down behind the baricades.

    But guess what? The people who will enivitably will be responsible for any 'skynet' either don't read those types of books or read them, then do it anyway having not gotten the message.

  11. Paul Cooper

    Iain Banks

    Covered a lot of the issues of genuine AI - including AI with intelligence vastly beyond human - in his Culture novels. It is, perhaps, noteworthy that it is implicit in his treatment that the main means of enforcement of "morality" in his universe is peer pressure. In fact, one of his characters in "Player of Games" when tempted to do "wrong" notes that there are no effective dincentives EXCEPT that he would no longer be able to participate in society as he had before.

  12. Graham Marsden
    Thumb Up

    I'm sure...

    ... the Marketing Division of the Sirius Cybernetics Corporation would have an answer we can all Share and Enjoy!

  13. cyber7

    What ever could go wrong...

    That guy 'Murphy' has nothing on 'the law of unintended consequences'...

    Read these works of SCI-FI and see if you still think "A.I. in a box" is a good idea.

    Harlan Ellison - "I have no Mouth, but I must Scream" - Built for war, AI goes insane and kills all but 5 humans, makes them virtually immortal & then tortures them for eternity.

    Dean Koontz - "Demon Seed" - AI traps scientists' wife and impregnates her with a humanoid life form of its own design and mind so it may live.

    D.F. Jones - "Colossus" - Yankee & Soviet AI's get together and figure this whole cold war business is nonsense; they decide to optimize humanity and avoid wall altogether. Step out of line and you get nuked.

    1. Michael Wojcik Silver badge

      Re: What ever could go wrong...

      You do know those are works of fiction, right?

      Certainly I'm all for reading fiction; one of my degrees is in literature, and I nearly got a PhD half in the subject. And speculative fiction, done well, is excellent grist for the mill. But it doesn't constitute a compelling, logical argument about the outcome of some process; unlike history, it doesn't even constitute empirical evidence of past behavior. A good Bayesian reasoner would still take it into account, but to magnify it into a blanket rejection of an entire technological regime would rather appear to be a category error.

      (And how does "AI-in-a-box" differ from "AI" without the qualifier?)

  14. Florida1920


    Man vs Machine. This will ultimately come down to class warfare, as Fritz Lang clearly understood.

    Oh, and that was Charlie Chaplain Chaplin in Modern Times.

  15. Toltec

    More authors

    Rudy Rucker - Software, Wetware, etc.

    Neal Asher's Polity series

    Peter Watts in Blindsight introduces aliens which while not AI are intelligent without self consciousnous.

  16. BattleBotBob

    Halting State

    Charlie Stross does an outstanding job of near future computer issues in "Halting State." It is full of enhanced reality used by cops on the beat, Chinese hackers, mmorpg banks, and self driving cars. His sequel. Rule 34, fad lots on 3D printing and AI but isn't quite as much of a fun read.

  17. Michael Wojcik Silver badge

    Curriculum design

    This proposal, like so many, founders on the shoals of the undergraduate schedule. With 8 semesters or the equivalent, the curriculum is already packed full of the classes deemed most important by practitioners in the field, or general-education and breadth requirements, or the prejudices of accreditation boards (such as a laboratory-science requirement). Sticking not only another1 ethics course in the list of requirements, but one so narrowly focused at that, is very difficult to justify.

    A better option would be to encourage CS programs to work with Philosophy and/or PoliSci departments to develop Ethics minors and double-majors, for CS undergrads particularly interested in the subject. Minors or double-majors in mathematics, EE, business, design, and so on are already common for CS students; it doesn't hurt to advertise additional possibilities.

    But as a requirement it makes little sense.

    And I'll note that when I was a CS undergrad, the most popular humanities elective in my cohort was the "Science Fiction Literature" course offered by English, so many of the students were reading this sort of thing anyway.

    1There was a required ethics course in the CS syllabus when I got my BS in the subject, and I think the ACM still recommends one.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like

Biting the hand that feeds IT © 1998–2022