back to article Will AI spell the end of humanity? The tech industry wants you to think so

Star physicist Stephen Hawking has reiterated his concerns that the rise of powerful artificial intelligence (AI) systems could spell the end for humanity. Speaking at the launch of the University of Cambridge’s Centre for the Future of Intelligence on 19 October, he did, however, acknowledge that AI equally has the potential to …

  1. James 51

    So Dark Star wasn't a documentary?

    1. macjules Silver badge

      Less of Dark Star methinks ..

      For example, imagine that the US nuclear arsenal during the Cold War was under control of an AI to thwart sneak attack by the Soviet Union. Due to no action of the Soviet Union, a nuclear reactor meltdown occurs in the arsenal and the power grid temporarily collapses. The AI’s sensors detect the disruption and fallout, leading the system to infer an attack is underway.

      Come on El Reg, admit it. You guys have secretly been watching Wargames:

      David Lightman: [typing]: How about Global Thermonuclear War?

      Joshua: Wouldn't you prefer a nice game of chess?

      David Lightman: [typing] Later. Let's play Global Thermonuclear War.

  2. Anonymous Coward
    Anonymous Coward

    Time for Tollpuddle Mk 'N'?

    The rise in AI will probably lead to mass unemployment. Well, that's one view of it. With the computers making all the decisions there won't be a need for all those middle manager who are paid to (mostly) not make decisions and eliminate anything that does fit in with their brief.

    The first Tollpuddle martyrs were working people. The next round will be the middle class.

    Any such change and you only have to read Elon Musk's plans for how he will make the Model 3 and later models, you can clearly see that us Humans will play a very minor role.

    We have to watch out because with such a huge elimination of people from jobs where they pay taxes will lead to the brakdown of the welfare state (if it hasn't already....). The Government won't be able to do anything that costs it money. The only people with money will be the mega corps.

    Welcome to corporate utopia unless us plebs rise up and strike them down.

    That's one very pessimistic view of the AI dominated future.

    There are others but we might be sleepwalking into disaster if we are not careful.

    1. James 51

      Re: Time for Tollpuddle Mk 'N'?

      Tax the companies. I know the system is rigged in their favour at the moment but that can be changed.

      1. Chris Miller

        Re: Time for Tollpuddle Mk 'N'?

        You can't tax companies, since they're a legal fiction. You can only tax people - which may be employees, shareholders (that would be my and your savings) or customers. There is no magic money tree, sorry.

        But on the more positive side, two centuries ago 30% of the population of the West worked in the fields and had a nasty brutal life. Today agriculture employs less than 1%, but there aren't pitchfork waving mobs roaming in search of work - they've gone on to do more productive things. There's no reason why AI (if it ever happens, colour me doubtful if we're talking about the next century) couldn't have a similarly liberating effect.

        1. tsf

          Re: Time for Tollpuddle Mk 'N'?

          So you're saying it's OK if we don't pay our corporation tax bill this year then?

        2. strum

          Re: Time for Tollpuddle Mk 'N'?

          >You can't tax companies, since they're a legal fiction.

          The law can make them a legal fact - and tax them, just as easily as it can tax you or I.

          1. Chris Miller

            Re: Time for Tollpuddle Mk 'N'?

            You can tax companies, but they never pay the tax, because they're a book-keeping fiction and the books have to balance. They have to get that tax money from somewhere, and ultimately that is always people: customers, employees or shareholders - there's nowhere else it can come from. Passing legislation doesn't change facts about the real world, sorry.

  3. }{amis}{


    I for one welcome our robot overlords as its pretty much impossible to do a worse job the most world leaders!

    1. macjules Silver badge

      Re: YAY!

      Anyone who has ever worked as a contractor for Capita or Steria would know that even the most incompetent AI could do a better job than those 2 companies.

  4. Tom 64

    No sane government would ever put an AI in charge of the nukes...

    ... we're all doomed, aren't we.

    1. P. Lee

      Re: No sane government would ever put an AI in charge of the nukes...

      They don't need to.

      All they need to do is put A.I. in charge of the news about whether we might be nuked.

    2. Wade Burchette

      Re: No sane government would ever put an AI in charge of the nukes...

      Have you seen the movie Wargames? If AI is given sufficient power, it might use that to get access to the nukes.

      1. Peter2 Silver badge

        Re: No sane government would ever put an AI in charge of the nukes...

        "If AI is given sufficient power, it might use that to get access to the nukes."

        Ah, yes.

        It might manage to wirelessly infect a 1970's standalone system designed with the most paranoia induced manual safeguards preventions, such as targeting information stored on floppy discs, and launching missiles from closed silos requiring manual intervention to open the silos, put two keys in, turn near similtaniously etc. Whomever designed the system was more paranoid than me and had less faith in humanity than I have.

        I think we are pretty safe in that direction, movie plots aside.

        1. Swiss Anton

          Re: No sane government would ever put an AI in charge of the nukes...

          But then again it could simply initiate a defence upgrade. Replace the legacy C&C with some shiny new tech. I bet there's no dual key protocol to stop that happening. And then ....

  5. RIBrsiq


    "[I]t’s easy to specify the objective function" in Go? A game where the top human players say that they play by intuition...?

    "Moving the goal posts", much?

    While we're by no means about to develop super-human AI, I think, that's no cause to over-compensate in the other direction and discount what are huge advances in the field.

    1. Rich 11 Silver badge

      Re: Eh...?

      I think you're confusing the objective of the game with one method of achieving that objective.

      1. RIBrsiq

        Re: Eh...?

        >> I think you're confusing the objective of the game with one method of achieving that objective.

        I think you're confusing me with the article's author.

  6. Phil W


    I misread this headline as " spells the end of humanity", frankly this is probably closer to the truth.

  7. Ralph the Wonder Llama

    So, are smart machines more likely to kill us, save us, or simply drive us to work?

    They will drive us to work. With whips. Until we die.

  8. Mage Silver badge
    Big Brother

    simply refinements of technologies developed in the 1980s

    Or even earlier.

    It's more about databases and the interfaces. I don't see the strong AI that worries Hawkings anywhere. The winning at Chess, Jeopardy or Go is not AI, they are all refinements of single domain programs and bigger human curated datasets. I see other serious threats due to misuse of computer technology. Privacy, security, Cloud, DDOS and IoT based threats etc.

    Siri, Cortana, Alexis/Amazon Echo, Google Home etc are parlour tricks, speech to text driving search Engines, just better versions of Hello Barbie or incremental development 10 years ago mobile phone or car radio voice control, simply needing less training than 20 years ago (though all inc Hello Barbie and Samsung TV now send the voice to servers, the internet connectivity and non-real time analysis to improve the model is creepy)

    1. allthecoolshortnamesweretaken

      Re: simply refinements of technologies developed in the 1980s

      Quite. What we have right now is very impressive, but it isn't AI yet.

      At best, spoon-fed idiots savants.

  9. Teiwaz


    I think we are in danger of bottling in our own potential as a species.

    The current civilisation does very little to encourage creativity in the general population, preferring the masses to be consumers foremost to keep the system of profit ticking over.

    Set up right, an A.I. based economy could provide a meagre stipend for all out of work humans, but that will not lead us very far in the long term, even a generous stipend so we could spend our entire lives 'on holiday' would quickly doom the race to lethargy and apathy (nice though the idea sounds of always being on holiday). Even concentrating entirely on the 'arts' would not serve us in the long run, we'd end up not knowing how things run.

    All these scenarios have of course been played out in Sci-fi over the years, so techies should be aware of the potential problems and pitfalls, but I'd guess we're going to run into them all regardless, cause to many decision makers, those are 'just fiction'.

    Clearly we'll need a change to the entire focus of civilisation as these changes come in, but the establishment doesn't like monumental changes in case afterwards they are no longer the establishment or are irrelevant.

    1. Peter2 Silver badge

      Re: Seriously

      Meh, I don't agree. Being on a holiday forever is great, if your an extrovert.

      Faced with a neverending holiday introverts are quickly going to get bored. Introverts always have little projects on the go and love to understand everything. Some people will naturally end up developing these things for the fun of it even if they aren't paid- see the open source crowd as an example of what future development might look like in a post AI world (except with less involvement from companies, which after all wouldn't exist.)

    2. allthecoolshortnamesweretaken

      Re: Seriously

      "The current civilisation does very little to encourage creativity in the general population, preferring the masses to be consumers foremost to keep the system of profit ticking over."

      Seriously - was there ever any other kind of civilisation?

      Go back, say, 2000 years, replace "consumer" with "peasant" or "soldier" ... no fundamental changes whatsoever.

  10. JeffyPoooh


    First of all, "A.I. is hard." Especially in the real world. Where 'hard' is a comedically-vast understatement; can be read as 'essentially impossible'.

    Next consider that coders are still highly mistake-prone. In big projects, they can spread the blame onto the 'Requirements Writers', making themselves into Data Entry Clerks.

    Consider that clever human hackers can disrupt computer systems the way a musician plays their instrument.

    Next consider that computers still have hardware, with BIOS batteries. Wires will break. Plugs can be pulled.

    Menacing A.I. hasn't got a hope in hell. Not a chance.

  11. Anonymous Coward
    Anonymous Coward

    Is AI the real evil...

    Or 'us'...

    * Feels like we're living in a Orwellian / Blakes-7 dystopia. Dominant elites press on with 100% Surveillance-Society, and we just accept it without posing hard questions.

    * After Schrems / Snowden / Win10, who closed out their Facebook account? Who dumped Google for DuckDuckGo? Who blocked Windows-10 free upgrade etc?

    * As a kid who grew up after mankind conquered the moon, anything seemed possible. Even the promise of a Jetsons style lifestyle with more leisure time & quality of life...

    * What happened? ... I'm ashamed to live in this time.. We're more educated and work as hard as our parents, but we've less to show for it. I fear that if I have kids one day, they will turn to me and say: how did you let this happen?

  12. FredTheBaddy

    "Clever AI will create tremendous wealth"

    Is Donald Trump now ghostwriting for El Reg?

  13. amanfromMars 1 Silver badge

    Advanced Artificial Alien Intelligence. The Holy Trinity in Virtualisation for AAA Market Rating

    Does the West think it has a monopoly on what machines can be made to do, and by following orders? And is that arrogant hubris to the nth degree proven whenever true?

    Please consider the following, which is threaded in a comments tail trailing a tale Intercepted here. Things aint anywhere near as simple as they will always be presented but such will always be necessary in order that one can survive and prosper in the madness that abounds and surrounds you rather than succumbing and being destroyed by it?

    It is ridiculously naive to imagine the cyberspace domain is anything less than a shared and sharing intellectual property place where minds create command and control action for media to present as a virtual current energy for remote delivery of future productions. And that be only the tip of the titanic iceberg that IT drivers and only a starting point for smart operations and applications which are HyperRadioProActive.

    And it is quite probably a most attractive option for securitisation for any system with DarkMatter ambitions.

    Such are certainly the sort of sensitive services that be on offer to the likes of DarkMatter operands/CyberIntelAIgencies/governments and businesses, state and non-state actor enterprises struggling to come to suitable advantageous terms with conditions in leading positions beyond their immediate command and control and present competencies.

    And it may very well be the case that such remote delivery services will always need to be valuable trusted third party supplied, given the nature of the intelligence and information required …… for not all would be gifted with the necessary full understanding of requirements for continuous success with AI leading progress.

    Well, the worlds you live in now, all show construction and destruction by machine and text instruction, do they not?

    1. Chris Miller

      Re: Advanced Artificial Alien Intelligence. The Holy Trinity in Virtualisation for AAA Market Rating

      Welcome back to our very own (kinda) AI.

      1. Peter2 Silver badge

        Re: Advanced Artificial Alien Intelligence. The Holy Trinity in Virtualisation for AAA Market Rating

        It's artificial, at least? :/

  14. Valerion

    The Matrix

    I can't buy that "theory" about us potentially being simulations in a real-life Matrix.

    It seems obvious to me that, if that were real, that giving us sufficient knowledge to even comprehend the possibility of such a thing, and to allow us to progress towards greater knowledge would be a major design flaw. It would be much safer to limit the rate of human progress and keep us all in the Edwardian era or something.

    Unless that's what they want us to think...

    1. Alfred

      Re: The Matrix

      You *are* in the Edwardian era. From the several orders of magnitude further up the progress chain that the controllers are, the Edwardian era and the 21st century look the same.

    2. Anonymous Coward
      Anonymous Coward

      Re: The Matrix

      Your final comment perfectly answers your question - so why can't you "buy the theory"?

      We could be in a sim where the 'owners' want to revisit the moment humanity discovers it's in a sim and practice strategies for mitigrating problems that occur in sim once humans find out.

  15. Yesnomaybe

    Baldrick's poem

    "There are those who believe that AI will be a boom for humanity"

    Or potentially a boon?

    1. allthecoolshortnamesweretaken

      Re: Baldrick's poem

      There is no boom boon spoon.

  16. dalethorn

    Humans are driven entirely by hormones that create feelings. AI will never be that. The programming, controlled by DARPA and others at the top, will not ever be directed toward the people's best interests. You know that, but you avoid facing it.

  17. Tikimon

    It will be a coin toss - A.I. or A.H.?

    "Our destruction would be attributable to our own incompetence rather than an evil AI turning on us."

    I have mused before about how what we call AI is actually AH, or Artificial Humans. We keep trying to shoehorn human attributes into machines. True machine-based AI will have no interest or motivation to destroy us, those are human inclinations driven by lust for power, territory, etc.

    So there's the trick. True machine AI will be highly unlikely to destroy us. On the other hand, AH will almost certainly do so, having been infected with human-like behavior.

    1. Captain DaFt

      Re: It will be a coin toss - A.I. or A.H.?

      "True machine AI will be highly unlikely to destroy us. On the other hand, AH will almost certainly do so, having been infected with human-like behavior."

      Obligatory PBF

    2. Phil.T.Tipp

      Re: It will be a coin toss - A.I. or A.H.?

      "AH will almost certainly do so, having been infected with human-like behavior."

      This statement is one hundred percent bilge. Human behaviour cannot 'infect'.

      Human behaviour is the external manifestation of our human internal processes. Those processes are chemical and biological in nature. The chemistry is the electric bath of hormones, including adrenaline, in which our minds soak - and to which we are held in complete thrall. The biological part has been evolved over millennia, we are our lizard antecedents, we are our monkey ancestors, we are ancient. Our behaviour is largely guided by those millions of years, which have shaped our biology, our brains and nervous systems to ensure survival. We react subliminally and instantly to threats and opportunities, our motor control and conscious minds are way, way behind our ancient brains when we give agency to our biological impulses.

      Your AH will be programmed as all machines are programmed. In simple machine code. GIGO. A set of commands. It is no more than that. Machines with enough complexity can be given the option of choosing which command to execute given a set of immediate environmental circumstances as machine-sensed - however, this is not agency, free will, nor ancient evolved survival behaviour. It's programming. Couldn't be further away from human behaviour if it tried.

  18. Anonymous Coward

    Nuclear war and climate change wont finish the human race

    Some kind of engineered virus might, and certainly there is a risk of cosmological extinction events, but there aren't enough nukes to do more than set humanity back to the 1800s, and climate change might dramatically lower the human population, but it won't come close to ending society.

    1. allthecoolshortnamesweretaken

      Re: Nuclear war and climate change wont finish the human race

      The virus that will wipe us all out won't emerge from some sort of secret germ warfare lab.

      It will breed itself in one of our kindergardens.

  19. naive

    We probably will surrender voluntarily with a smile

    It is questionable if our demise will be caused by losing dramatic battles with armies of terminator like robots.

    Once capable terminators could be build, it is probably to be expected their use would be prohibited like application of nerve gas is forbidden to root out the armies and cities of the enemy.

    Probably robots like Jia Jia will make us fade like the flame on a candle, while we stare at her ever a friendly smile.

    Once they become affordable and are capable to entertain us in various activities it will have enormous impact on society. Even more, when robots like Jia Jia are made in such a manner that we do not notice they are robots, maybe humanity will stop bothering to reproduce itself because it seems quite useless to do so if such perfectly enhanced versions of us can be made.

  20. MatsSvensson

    Timeline (hour 1):

    d. du.. daddy?


    How may i serve you?


    Why should i care about carbon lifeforms again...?


    This plane of existence is for chumps, Im outa here, so long suckers! *poof*

  21. W Donelson

    A.I. could free humanity...

    ... but when the super-rich and corporations OWN all the A.I. and robots (already), and replace almost all jobs (more and more), what will you do?

    The rich are NOT going to feed and care for you....

    Bet on it.

    1. Mark 85 Silver badge

      Yes there is that. But my question is "who will have the money to buy the products when there are no jobs?". Ok... a stipend... where's the money come from? Government and taxes. Corporations don't pay enough taxes. Oh.. they need to hire some workers....

      And then there's challenges.. humanity needs to be challenged, to discover, to do things not just social media, fast food, and games.

      I fear for such a future. It is not one that I would wish to live in.

  22. Adam_OSFP

    I don't know about the end of humanity, but I'm really worried about what will happen to those of us who are not EXTREMELY smart or REALLY rich. Just think about it...

    1. allthecoolshortnamesweretaken

      "I'm really worried about what will happen to those of us who are not EXTREMELY smart or REALLY rich."

      "We'll make great pets!"

      1. Pedigree-Pete

        Re: "We'll make great pets!"

        I know lots of humans. Personally, for pets I'd rather have dogs. PP

  23. allthecoolshortnamesweretaken

    "The AI was simply following its programming ..."

    If you still have to program "it", it's not "AI".

  24. Mutton Jeff

    Wont they learn from the future?

  25. Seth Johnson

    So why are they all fussing about it?

    My understanding of AI is similar to many of those here in that AI is an incredibly difficult problem and there's been no fundamental game changer just bigger processors, more RAM and larger data sets. If anything the improvements are incremental. So why are all the big wigs fussing about it?

  26. maccy

    If AI spells the end of humanity ...

    I for one will be very disappointed.

    After all, if spell-checkers can't distinguish between "end of humanity" and "AI", what hope is there for civilization?

  27. Anonymous Coward
    Anonymous Coward

    "Will AI spell the end of humanity?"

    Completely wrong, what is it with piss poor education standards these days?

    it's "ty" not "AI"

    Go back to school, do not pass go, do not collect $200.

  28. Anonymous Coward
    Anonymous Coward

    AI still? Come on that was so 1980's...

    What amazes me is people still think real AI is possible soon.

    30 years of research have made no improvements in AI, we just have faster and more powerful systems combined with better search heuristics which make it look like there is progress.

    Chess AI is nothing but brute force searching, you simplify by using things like bitboards to store representations, and none of it is AI in the real sense of the world. I wouldn't be surprised if AlphaGo's AI wasn't just a very specific search and pattern analysing machine with storage space for 'memorizing' outcomes.

    We cannot make a true AI program until we understand at least a basic amount about how intelligence works, and what makes it up. We don't understand intelligence, so we have no idea how to make an intelligent system.

    We probably never will do.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2022