back to article Ukrainian teen created in lab passes Turing Test – famous nutty prof

Some software has supposedly passed the Turing Test – a controversial benchmark of artificial intelligence – by fooling a handful of humans into thinking it's a talkative 13-year-old Ukrainian lad. Cyborg prof Kevin Warwick argues this is the first time a machine has ever passed the famous test. We're told the successful chat …

COMMENTS

This topic is closed for new posts.
  1. Robin

    Transcript

    Is there a transcript of this conversation available?

    1. diodesign (Written by Reg staff) Silver badge

      Re: Transcript

      I'm hunting for transcripts. In the meantime, this is the sort of level of conversation you can have with the public online version (DDoSed under popularity)

      C.

      1. Anonymous Coward
        Anonymous Coward

        Re: Transcript

        Complete drivel and that supposedly fooled celebrities and other humans. OK, celebrities fine but other humans? To paraphrase Bill Clinton, we need to define "other humans"

      2. GettinSadda

        Re: Transcript

        Judging by that transcript it is no more than Eliza with a larger list of stock phrases. It even tripped up on a common Eliza fault with "do you like to eat $food" where $food="at home"

        1. VinceH

          Re: Transcript

          Indeed. I didn't see much in that conversation that made it look significantly better than something I wrote at a YTS place over 30 years ago, other than benefiting from greater storage (and processor grunt) to provide a much bigger range of data.

          I despair at the judges.

          Edit: I've just spotted that the example transcript was from just after one of the previous attempts, when it achieved just under 30% - so it may have improved since then. But I still despair at the judges in that attempt.

        2. Lockwood

          Re: Transcript

          Oh, what a fruitful conversation;-)

          1. sisk

            Re: Transcript

            I could see it fooling some people if they believed they were talking to a human with less than stellar command of the English language (such as could be expected from a 13 year old Ukrainian), but 30% of humans? I think that's pushing it.

            Then again, people are stupid and will believe whatever they want to be true or what they fear is true.

            1. Blitterbug
              Meh

              Re: 30% of humans? I think that's pushing it.

              Perhaps we need an 'advanced' Turing Test, judged solely by IT types. Seriously. As has been said here several times, this was a really dreadful Eliza hash without anything to show the improvements that have been made in AI in the 30-odd years since that program was being bashed into every home computer on the planet.

              If it can fool > 30% of developers, I'll definitely sit up and take note.

              1. Anonymous Coward
                Anonymous Coward

                Re: 30% of humans? I think that's pushing it.

                I'm wondering if the real humans are doing their best to pretend they're not -- in which case it could be quite tricky to figure out who or what you're talking to.

              2. Pookietoo
                Trollface

                Re: If it can fool > 30% of developers, I'll definitely sit up and take note.

                Are you including VB devs in that?

      3. spatulasnout

        Re: Transcript

        In the transcript, it had begun to seem like a chat bot by its second reply:

        J: What did you do today so far?

        E: Since early this morning I’ve been involved in this funny contest. I also plan to visit some interesting places in Atlanta.

        The "funny contest" part was cute, but the Atlanta part already seemed like an embellishment that doesn't quite mesh with the question being asked--in a manner so typical of chat bot responses.

        I wonder what kind of briefing is given to the judges beforehand. The simplest bit of advice to the judges could thwart any chat bot I've yet encountered: try to teach it something.

        I usually start out by attempting to teach them how to draw even the simplest possible shapes as ASCII art. (The simplest way to draw a "star" might be an asterisk.) Unsurprisingly, the bot immediately begins dithering and misdirecting and trying to change the subject. Not one has yet even made a pretense of being willing to attempt learning something.

        Of course, if ASCII art shapes became a popular question, bot authors would code in some logic for that. The key is just finding something simple it doesn't yet know how to do (that any human could easily respond to and learn) and trying to teach it that.

        The day someone makes a bot that can navigate these simple interactive learning tasks and actually add new tricks and vocabulary to its repertoire in the process, I'll be genuinely impressed (regardless of how it might perform in a Turing test setting conversationally.)

    2. JeffyPoooh
      Pint

      Excuse the philosophical question...

      Can the Turing Test be automated?

      (The distant sound of heads exploding... LOL)

      1. Yes Me Silver badge
        FAIL

        Re: Excuse the philosophical question...

        It turns out to be quite amusing to ask Eugene that question over and over:

        Q. Can the Turing Test be automated?

        A. Oops! I don't have an answer... It might have been stolen by malicious android!

        Q. Can the Turing Test be automated?

        A. Don't repeat yourself, please:-) Could you tell me about your job, by the way?

        Q. Do you know about Spinoza?

        A. Dutch philosopher. It looks like you didn't go to school. So you don't know anything about Baruch Spinoza.

        Q. Can the Turing Test be automated?

        A. Ummm... Frankly, I didn't get your question:-(

        Q. Can the Turing Test be automated?

        A. Could you rephrase your question? I've been bewildered a bit!

        Q. Can the Turing Test be automated?

        A. Don't repeat yourself, please:-) Oooh. Anything else?

        How could anybody be misled for a moment by this?

    3. Gordan

      How long before...

      ... auto-troll feature is implemented in online forums and games? And how will we be able to tell it's not humans trolling?

      1. Anonymous Coward
        Anonymous Coward

        Re: How long before...

        I suspect that has already occurred. As for telling the bots from the trolls, what's the point?

      2. Euripides Pants
        Unhappy

        Re: How long before...

        Damn! I've been discovered!!

  2. Jonathan Richards 1

    Language skills?

    Was the conversation conducted in Eugene's native language, which I presume is Ukrainian, or in English? If the latter, I suggest that this is not the scenario that Alan Turing was envisaging. I'd make an uninformed guess that the best discriminators between AIs and humans are currently language-based jokes.

    1. Filippo Silver badge

      Re: Language skills?

      I'm pretty sure the conversation was in English. Choosing a character for which English is not the primary language is probably a trick to plausibly invalidate language-based jokes.

      1. Tony Haines

        Re: Language skills?

        //Choosing a character for which English is not the primary language//

        That together with pretending to be thirteen seems like cheating to me. Else, why not claim to have a three-year-old battering away at the keyboard?

        1. BlueGreen

          Re: Language skills?

          That's what I thought. If you can play it that way, here's my entry (inspired, unfortunately, from real life)

          10 print "Waah! Waah! Waaaa!"

          20 goto 10

          To be fair, the accompanying transcript of Eugene isn't totally unimpressive. It shouldn't fool anyone but in its limited way it's not bad.

          (edit: and Kevin Warwick is a tit)

          1. Tom 13

            Re: Language skills?

            I think you can easily improve this one to more effectively imitate the baby.

            10 Nu=Rnd(0)

            20 If Nu >= .5 then 10

            30 print "Waah! Waah! Waaaa!"

            40 Goto 10

            1. horse of a different color

              Re: Language skills?

              13 year old kid is even easier -

              10 print "u gay fag!"

              20 print "* rage quit *"

              30 end

              Admittedly, most of my experience stems from online gaming.

      2. beep54
        Meh

        Re: Language skills?

        "Ha. Ha. That is very logical." Firesign Theater (possibly paraphrased)

    2. Mage Silver badge
      Headmaster

      Re: Language skills?

      As well as that doubt was cast on the concept of Touring test YEARS ago when Eliza was written. Given the quality of Reality TV and Soaps you need an expert.

      IMO even when the "The Touring Test" can be passed well enough fool experts, it doesn't mean anything about progress on AI, just progress on simulation of conversation. Just like Chess was thought to need AI and Alan Turing himself proved it didn't.

      I think anyway Alan Turing's comment was an off the cuff statement rather than anything with any mathematical proof, unlike his paper about solvable & unsolvable problems illustrated with the infinite paper tape driven computer. ("Turing Machine").

      Of course Kevin Warwick involved makes one think it may be ill-informed hype.

      1. Tom 13

        Re: Language skills?

        I don't think it should be held against the Turing Test that there are people who can't pass it. I leave the question of what should be done with those who fail to others as a homework assignment.

  3. Anonymous Coward
    Anonymous Coward

    I wonder if Prof Cyborg will deign to publish his results in a peer-reviewed journal, or whether this will be another one of his media friendly press releases that never seems to get into the formal literature.

    1. diodesign (Written by Reg staff) Silver badge

      Re: Mike

      "I wonder if Prof Cyborg will deign to publish his results in a peer-reviewed journal"

      I hear the results from Saturday will be put into a paper of some sort. Waiting for the university to get back to me. I gather the uni denied the Telegraph access to the transcripts, so this could turn interesting.

      C.

      1. Shady

        Re: Mike

        "I hear the results from Saturday will be put into a paper of some sort."

        Such as the Daily Mail. EAST EUROPEAN ARTIFICIAL INTELLIGENCE ON ITS WAY TO CLAIM BENEFITS FROM UK TAXPAYERS!

      2. TheOtherHobbes

        Re: Mike

        >I hear the results from Saturday will be put into a paper of some sort.

        It's been in the Daily Mail already.

  4. ADJB

    Pot - Kettle

    Has Kevin Warwick ever managed to pass a Turing test?.

    1. Alan Bourke

      Re: Pot - Kettle

      Har!

    2. Mad Chaz

      Re: Pot - Kettle

      He still did better then any politician at pretending to be human.

  5. Jedit Silver badge

    What's the opposite of a Turing test?

    Is this really an advance in computing, or does it mark a degeneration in humans? The typical 13-year-old on the internet gets harder to distinguish from an Eliza every year.

    "What do you think about X?"

    "hurr lol your mom is X"

    1. Anonymous Coward
      Anonymous Coward

      Re: What's the opposite of a Turing test?

      I wonder what would happen if this "character" met up with his forebears PARRY or ELIZA.

      Those two did get chatting one day, it didn't go well.

      1. Ken Hagan Gold badge

        Re: What's the opposite of a Turing test?

        "I wonder what would happen if this "character" met up with his forebears PARRY or ELIZA."

        That's an interesting idea. One of the problems of the Turing test is that any sane human will give the machine the benefit of the doubt and "rescue" any conversation that is heading for the madhouse. So why not instead put two instances of <test-subject> into conversation with each other and ask your human to judge whether the resulting conversation is between two humans or two machines?

    2. CADmonkey

      Re: What's the opposite of a Turing test?

      @ Jedit: FAIL you spelt 'your' wrong

      1. Jedit Silver badge
        Headmaster

        "FAIL you spelt 'your' wrong"

        No, I didn't. I admit, though, that I did make a mistake in that I spelt it correctly.

        1. Anonymous Coward
          Anonymous Coward

          Re: "FAIL you spelt 'your' wrong"

          @Jedit "ur" hth

  6. Alan Bourke

    From reading the sample above linked by DioDesign

    all I can say is that if it fooled 33% of the people testing it then they're a pretty credulous bunch.

    1. Matt_payne666

      Re: From reading the sample above linked by DioDesign

      I'd hate to check the condition of the computers these judges own....... I'm guessing that they have reset their banking credentials, downloaded their parcel redirects and are currently waiting for the reimbursement of $Millions from some Nigerian bank account they have inherited...

      Yes, some replies are natural language replies... but a lot are the sort of guff you would expect Siri, Cortana or some clever internet chat page to come up with...

      Here is hoping that Eugene was an early beta version....

    2. hplasm
      Happy

      Re: From reading the sample above linked by DioDesign

      Slightly OT... Is it DioDesign or DiodeSign?

      That would thow an AI.

      1. Doctor_Wibble
        Coat

        Re: From reading the sample above linked by DioDesign

        > That would thow an AI.

        But how far and how well would it fly? Like an arrow or a banana?

      2. John Brown (no body) Silver badge

        Re: From reading the sample above linked by DioDesign

        "Slightly OT... Is it DioDesign or DiodeSign?"

        How many diodes could a diodesign sign if a diodesign could sign diodes?

  7. John Tserkezis

    "the bot managed to convince 33 per cent of the judges that it was a real boy. The pass mark is 30 percent."

    Not bad. Less than 30% of the people I talk to face-to-face seem like real humans to me.

  8. Pete 2 Silver badge

    Intelligence on the internet?

    The ability to imitate a 13 y/o boy is a good goal - if you're a 12 y/o boy.

    However, I feel that if Alan Turing was alive today, and looked at the traffic on the world's most popular social media sites, he would (quite naturally) assume that they were test-beds for AI's and that there was still a long, long way to go before any of them appeared even faintly human.

    If this result tells us anything, it's that a test devised right at the dawn of the IT era, before there was any experience of AI to draw on, is too limited to be useful. Just as we don't believe that aircraft imitate birds (even though they both fly), we shouldn't consider this anything like a computer imitating a person.

    1. Anonymous Coward
      Anonymous Coward

      Re: Intelligence on the internet?

      Unlike most chatrooms of 50+ year old men pretending to be 14 year old girls

    2. plrndl

      Re: Intelligence on the internet?

      It's a long time since I was 13, but I seem to recall that mumbling a lot and refusing to complete sentances and answer questions were par for the course.

      It should be easy for a computer to replicate that.

      1. I ain't Spartacus Gold badge

        Re: Intelligence on the internet?

        Whadda you mean I failed the Turing Test? It's not fair! I HATE YOU!!!!!

  9. Anonymous Coward
    Anonymous Coward

    Lowering the bar, surely?

    I don't think the point of the Turing test was really to reduce the scope of the conversation to the point where the AI could be convincing; if it was then someone could have claimed the "every bit as convincing as a 1 year-old" victory decades ago.

    1. Pascal Monett Silver badge

      Totally agree

      From the extract in the article itself, the Turing Test makes no mention that the subject tested is supposed to be anything but an adult.

      But hey, you gotta start somewhere. Personally, I would have failed it straightaway. 13-year-olds use way to much l33tsp35k for me to understand them. This one wrote complete sentences. That, in my book, is a dead giveaway that it can't be a human child.

  10. NomNomNom

    impossible

    this is bullshit of the highest order

    1. ian 22

      Re: impossible

      Highest order? We should try for excellence in all things!

  11. Anonymous Coward
    Anonymous Coward

    /* 13-year-old boy */

    main()

    char noise[256];

    {

    while(1)

    {

    fgets(*noise, 256, stdin);

    printf("Yeah, whatever.\n");

    }

    }

    1. Anonymous Coward
      Anonymous Coward

      laughing out loud @ 256 char buffer.

      1. SoaG
        Trollface

        256?

        Should be 140 characters, Shirley?

  12. Zog_but_not_the_first
    IT Angle

    Hmmm...

    This has me wondering how many commentards posting on El Reg are really bots.

    The groundwork for the programming algorithm has already been done:

    http://www.theregister.co.uk/2012/03/12/seven_kinds_of_commentard/

    And now I come to think of it, my mind is going. I can feel it slipping away Dave. Stop.

    1. Anonymous Coward
      Anonymous Coward

      Re: Hmmm...

      On any article about Dixons it sometimes appears to be close to 100% because the comments are always the same.

      1. Nick Ryan Silver badge

        Re: Hmmm...

        Pretty close to the "reviews" many goods, services, holiday locations or restaurants receive. It's almost uncanny how many always include the same key points.

    2. I ain't Spartacus Gold badge
      Happy

      Re: Hmmm...

      I'm a bot, and so is my wife.

      Not sure about that nice aManFromMars though. I've almost understood a couple of his recent posts. So there's probably been a software upgrade. Whether that's to him, or me, is another question entirely...

  13. itsallcrap

    Don't forget, 30% of people are idiots.

    I had a go at speaking with the web-accessible version before it got crushed under the weight of traffic. No one with at least half a brain would believe that was a human being. I didn't even think it was as convincing as Cleverbot.

    1. Anonymous Coward
      Anonymous Coward

      Re: Don't forget, 30% of people are idiots.

      And what are the other 80%?

  14. Anonymous Coward
    Anonymous Coward

    Only 35 years too late ...

    The late, great (and sadly missed) Dr. Christopher Evans wrote in his masterpiece (still worth reading today) "The Mighty Micro" about the Turing test in 1978.

    He mentioned checking into a hi tech conference where you entered your name into a hospitality system. Apparently the writers had included a little routine which "chatted" with the delegates, a lot of who assumed it was a real employee they were conversing with.

  15. lansalot

    hmmm

    How hard can it really be to impersonate a 13yo boy on the internet?

    From what I hear on Xbox Live, it's a very limited vocabulary, mainly consisting of assertions about everyone else's mothers and their sexuality...

  16. lansalot

    and...

    ..since when was talking a sign of thinking anyway?

    I come out with some right shite at times, which all but proves I don't have a fucking clue.

    1. Anonymous Coward
      Anonymous Coward

      Re: and...

      And that self-awareness sums up the difference, right there :-)

  17. Anonymous Coward 101

    Making the chat bot a 13 year boy seems to be giving oneself an unfair advantage. Why not just make the chat bot a developmentally disabled 8 year old child thereby giving an easy excuse for lack of knowledge or writing ability?

    1. Anonymous Coward
      Anonymous Coward

      >Why not just make the chat bot

      Any member of the royal family with a degree

      Tony Blair - Try getting any sense or a straight answer out of him

      A dailymail reporter

  18. M7S

    "Fooling celebrities" - insufficient data

    We need a list, if the rest of them are the sort of people who appear on Big Brother and it was carried by majority vote, then I don't think we need to get too worried for a while yet.

  19. Anonymous Coward
    Anonymous Coward

    o m g

    Someone on the INTERNET is wrong.

  20. Tim Starling

    AI is harder than Turing expected

    The original article by Alan Turing which El Reg linked to was interesting, and funny in places. Certainly it's clear that passing the test turned out to be a lot harder than Turing expected, in terms of hardware requirements:

    "As I have explained, the problem is mainly one of programming. Advances in engineering will have to be made too, but it seems unlikely that these will not be adequate for the requirements. Estimates of the storage capacity of the brain vary from 10^10 to 10^15 binary digits. I incline to the lower values and believe that only a very small fraction is used for the higher types of thinking. Most of it is probably used for the retention of visual impressions, I should be surprised if more than 10^9 was required for satisfactory playing of the imitation game, at any rate against a blind man. (Note: The capacity of the Encyclopaedia Britannica, 11th edition, is 2 X 10^9) A storage capacity of 10^7, would be a very practicable possibility even by present techniques. It is probably not necessary to increase the speed of operations of the machines at all. Parts of modern machines which can be regarded as analogs of nerve cells work about a thousand times faster than the latter. This should provide a "margin of safety" which could cover losses of speed arising in many ways."

    I don't know what kind of hardware Eugene runs on, but I suspect it is a lot better than 125MB of memory (including program and data) and a clock speed measured in kilohertz. And we are still a long way short of being able to construct a "learning machine" of the kind Turing describes in his final section.

    http://loebner.net/Prizef/TuringArticle.html

    1. Nick Ryan Silver badge

      Re: AI is harder than Turing expected

      That's the problem with, in this case, a historical lack of understanding. The brain isn't a binary device and while any individual component doesn't run especially fast, they do run in parallel. The concept of a machine fooling a human in a blind test is still a clever device, even if the understanding and predictions were out.

      This kind of historical take on something is often quite interesting, for example Asimov's robots could not speak but could understand. It was later advances in technology that lead to the "artificial voicebox" in his books. From a biological point of view it was correct - babies and toddlers can understand much more than they can speak, however from a technology point of view it's reversed as speech synthesis is simple compared to contextual comprehension.

      1. cracked

        Re: AI is harder than Turing expected

        I think Turing probably had a coder's understanding of what it means to be human ;-)

        Without the same frame of reference, there is no hope of comprehension, let alone caring enough (or not caring enough) to make an appropriate response.

        Really, it's unfair to compare these chat tests to AI; the former doesn't even really seem credible as a step along the way, to the later.

        And, if a machine did become self aware; wouldn't that mean the word "artificial" was no longer necessary?

      2. This post has been deleted by its author

    2. Anonymous Coward
      Anonymous Coward

      Re: AI is harder than Turing expected

      "Certainly it's clear that passing the test turned out to be a lot harder than Turing expected"

      AI turned out to be a lot harder than ANYONE expected. Even the supposedly (from a 1950s point of view) "trivial" AI problems of computer vision and voice recognition are only now really coming to fruition. One of the problems of course - other than hardware - is that the brain simply does not work on boolean logic except perhaps at the very highest level some of the time. Eg: "If its sunny I'll go for a bike ride else I'll watch a DVD". But the computer pioneers assumed it worked like this all the way down. You can't blame them , its an obvious inference , it just happens to be wrong.

      1. Anonymous Coward
        Anonymous Coward

        Re: AI is harder than Turing expected

        It makes me think to another of his hallmarks: the Halting Problem. He proved that a deterministic computer (his theoretical machine or the latest octo-core or whatever) cannot determine if a program will halt or not. But now what if we applied this to ourselves and asked if a human can examine the code and reliably make the same determination. Because if it can be proven we have the capability, then it may be possible to prove that humans can think in a way a computer cannot, thus making it mathematically impossible for a computer to ever completely imitate a human mind. If this determination has been made already, I'm a little out of touch, but it would certainly have some serious implications in AI research. I strongly suspect people realize or assume this, which is why I'm seeing AI research take different directions from before.

        1. Nick Ryan Silver badge

          Re: AI is harder than Turing expected

          Intriguing read about the "Halting Problem". But it goes to justify why my loosely held general belief that mathematicians should stay clear of programming still runs true.

          I've had countless arguments with mathematicians pretending to be programmers... from those that claimed that "5g" languages would make programmers obsolete to those that can't grasp that while small parts of a typical application can be represented in a mathematical manner, it quickly becomes pointless trying to apply such an unsuitable technique to wider applications or algorithms. While it is of course possible, the dataset rapidly becomes a ludicrous set of multi-dimensional possibilities and while the analysis can be streamlined the sheer processing power requirements to model and validate the entire thing renders any attempt pointless. In the end the algorithm effectively degenerates into a simulation. In many ways this is similar to computer chess.

          1. Kubla Cant

            Re: AI is harder than Turing expected

            @Nick Ryan: "5g" languages would make programmers obsolete

            I thought it was 4GL that made programmers obsolete. Does this mean I've been acting obsolete for no reason over the past 20 years?

            1. Nick Ryan Silver badge

              Re: AI is harder than Turing expected

              Yes. You're so obsolete that you don't even know it yourself.

        2. lotus49

          Re: AI is harder than Turing expected

          Your understanding of the entscheidungsproblem is flawed. It is perfectly possible to determine whether some classes of program will halt and both computers and people can do this.

          What Turing showed in his famous paper was that there is no generally applicable way of doing so. Since we do not know whether human beings can determine whether all programs will halt, we cannot judge whether humans can do something machines cannot. Maybe we can, maybe we can't. My money is on the latter.

          1. Anonymous Coward
            Anonymous Coward

            Re: AI is harder than Turing expected

            And mine's on the former. See, I don't think we completely understand how we ourselves think. It's quite possible we have some nondeterministic aspect to thought or some other method that is not black-and-white. Either way, it's something a computer cannot imitate (either at all or not without prohibitive amounts of resources).

  21. Phil W

    The 13 year old boy in me....

    ...sniggered at the headline saying "beats off".

    1. This post has been deleted by its author

    2. Anonymous Coward
      Anonymous Coward

      Re: The 13 year old boy in me....

      I think you could have phrased that better, unless you really did have a 13 year old boy in you.

  22. Steve Evans

    Impersonating a teenager...

    Randomly replying "Whatever", "bored" and "meh" should just about cover it.

  23. Elmer Phud

    13 yr old boy

    Well, that's pretty bloody easy.

    Ask it a load of Game of Thrones related questions and you won't shut the fucker up!

  24. Arachnoid

    I know, Iknow.............

    To get better than 33% just lower Eugenes age to 12,11 or even 10

    In all seriousness the "AI" isn't actually thinking though, it looks more like Eugene is reacting to the word associations that are fed into it with predefined responses without emotional content.

    Like "I hate you"........."But i want one!"........"You Shadup".........."It wasn't me"

  25. Caaaptaaaain kick arse

    Coming soon to a customer service experience near you...

    Punter: "I have a problem with my account"

    Botviser: "tell me about your mother"

    1. jonathan keith

      Re: Coming soon to a customer service experience near you...

      "My Mother? Let me tell you about my Mother."

      *kablam*

  26. Anonymous Coward
    Alert

    GroomBot

    Building an AI to sound like a 13 year old boy? Haven't people been arrested for doing what this bot does?

    "Honest m'lud, it wasn't me, it was my bot"

    1. Destroy All Monsters Silver badge

      Re: GroomBot

      Hmm.... "pedophilic manuals" come to hand. I mean, mind.

  27. David Pollard
    Coat

    This provides enlightenment as well as chat

    http://www.wisdomofchopra.com/

  28. fearnothing

    If anyone is interested in the topic, I can HIGHLY recommend the book "The Most Human Human". Fascinating book, well written and really enlightening. And Jonathan Richards 1 is absolutely right, language-based jokes are highly effective at identifying humans. Want to convince an expert on the other side of a computer screen that you're flesh and blood? Make up a pun based on something he said.

    Also, celebrity judges? Not exactly lifting my opinion of celebrities here if the example output is anything to go by.

    1. Charles 9

      OK, so perhaps we can agree that the Turing Test as we see it now is a little broad. Still it's an interesting step, and now that this step's been cleared, we can tighten the test: give it new conditions and call it the Revised Turing Test. Starting with the baseline of fooling at least 30% of the humans after a five-minute conversation, let's say we say the machine must simulate someone roughly analogous to the human (someone of the same age group and gender, so the program must be adaptable from person to person) with a comparable grasp of the human's language (this means the machine has to be able to understand language-based subtleties like puns). Perhaps in future revisions, we can include a requirement for using vocal communication and so on. So instead of dwelling on the past, we set ever-harder challenges for the future.

    2. Destroy All Monsters Silver badge

      language-based jokes are highly effective at identifying humans

      Just call MICROFTXXX, and get in touch.

  29. Anonymous Coward
    Anonymous Coward

    Real people

    It would be interesting to know how many real people in that test sample were judged to be bots.

  30. JeffyPoooh
    Pint

    It's actually quite simple...

    "Simple?"

    Yes, the bot simply repeats the last word (or two, if required), followed by a question mark, to emulate a common human response.

    "Human response?"

    Yes, many humans (if they're not really listening) will actually spend hours using this algorithm.

    "Algorithm?"

    An algorithm is a sequence of computer instructions.

    "Computer instructions?"

    Precisely, the input will be parsed to find the last word or two corresponding to a noun, or an adjective plus a noun.

    "Noun?"

    Yes, find the final noun and repeat it back in the form of a question, with rising inflection.

    "Rising inflection?"

    Yes, ....

    etc.

    1. Tom 13

      Re: It's actually quite simple...

      Yes, many humans (if they're not really listening) will actually spend hours using this algorithm.

      Uh-huh. And?

      1. Steven Raith

        Re: It's actually quite simple...

        But what is is?

        *gets Red Dwarf coat, leaves*

  31. Destroy All Monsters Silver badge
    Trollface

    Why is Turing trolling so intelligently?

    an average interrogator will not have more than 70 per cent chance of making the right identification after five minutes of questioning

    It took me a full two minutes to ascertain that this means the "pass" criterium is indeed that more than 30 percent of the people shall fail to identify the machine behind the terminal as being, indeed, a machine after five minutes.

    Maybe he thought the world is full of Alan Turings.

    1. Tom 13

      Re: Why is Turing trolling so intelligently?

      Maybe he thought the world is full of Alan Turings.

      Well, at least that would be better than billions of Carl Sagans.

  32. Anonymous IV

    Post-Turing test

    Determine the meaning of "However this event involved the most simultaneous comparison tests than ever before". Show your working. Do not pass Go.

  33. Clive Galway
    Trollface

    I feel like I am in a Turing Test....

    ... pretty much any time I join an online game.

    1. Steven Raith

      Re: I feel like I am in a Turing Test....

      I have been thinking that when playing ostensibly multiplayer console games (things like Titanfall) offline, the AI bots really need to tell me to go suck my mummys faggy cock more often, to make it more like a real XBox Live experience.

      Steven "doesn't play games online any more" Raith

  34. Anonymous C0ward

    This might be the first time a machine has passed it

    But how many times has a human failed it?

    1. Horridbloke

      Re: This might be the first time a machine has passed it

      Estate agents fail the Turing Test every working day of their lives. They also invoke the "uncanny valley". I was talking to an EA the other month and become convinced she was animatronic.

  35. Anonymous Coward
    Anonymous Coward

    Total bull

    I remember articles about chatbots (and that's what this is) fooling over half of people. That doesn't make them intelligent, it makes the people they fooled stupid, gullible or apathetic (do the testees get any sort of reward for guessing right?)

    It was rigged by claiming it is a boy from another country, which would allow people to excuse the sort of obvious mistakes a computer will make as being due to him not being a native English speaker, and not being an adult. Let's try it again with some Londoners thinking they're talking to someone who has lived his whole life in London, some Texans thinking they're talking to someone from Texas, etc. It is a lot harder to fool people if they use the kind of wording, slang and expressions they would expect, know places and landmarks, public figures, and so on.

    Even if you fooled 100% of people what you'd have would be a program that's good at carrying on conversations. It would be noteworthy, but hardly a measure of intelligence. Until the computer can truly understand what is being talked about, rather than simply coming up with a likely reply to what is being typed at it, the whole thing is silly.

    Turing developed his "test" when people didn't really have any idea what machine intelligence would consist of. He assumed that to carry on a conversation the computer would have to be intelligent. He didn't foresee the ability to have a database of gigabytes worth of facts available at the "fingertips" of the computer that will allow it to fake its way through a conversation well enough to fool people. Passing the Turing test is no more proof of intelligence than beating a person at chess by iterating through all the possible moves is. When computers start coming up with original ideas and inventions unprompted and unprogrammed, THEN they'll be ready to kill all the humans and run the world.

    1. DropBear
      Stop

      Re: Total bull

      No. By this test, people from _any_ other culture than your own fail to pass, and that makes your test invalid as a measure of being human. On the other hand, since it's a verbal / written communication, you would indeed be entitled to expect better-than-reasonable command of the common (likely English) language used in the test - so none of these "what you mean by that / sorry I'm Ukrainian" excuses should be admissible. Basically, language should not be used as an excuse impeding free communication of thought - and indeed what the testers should try to do is ascertain that their partner does actually have some of their own.

      You know what - here's a tip for free for future testers: "Do you think everybody should learn how to code? Either way, make your argument in no less than five sentences. Go!"

  36. DerekCurrie
    Thumb Down

    ELIZA Much?

    I've worked with ELIZA a lot over the years. Chatting with the online rendition of Eugene, I knocked it off its fake human pedestal with the first question. It's ELIZAesque fumbling only got worse as it persisted in trying to keep my interest. My only response is that, all these years after the invention of ELIZA, how far we have NOT come. Cry Dr. Kurzweil. Cry.

    And that's fine with me. We humans have trouble enough getting con-jobbed every day by real humans. We don't need machines adding to the demolition of our culture and trust.

  37. DryBones

    Article Title Incorrect

    It should be "Adults' Opinion of Teenage Writing Ability Now Equal to Chatbot". Seriously, that's all it is. The standard of perception is slipping.

  38. Anonymous Coward
    Anonymous Coward

    Re. Article Title Incorrect

    Lost of money is being lost every day by online phishing due to variants of this chatbot, normally on dating sites.

    The MO is convince victim that chatbot is real human then extract money via poisoned links to a phishing site.

    Of course, this only works if the victim is a complete dumba$$ of low intelligence and/or desperate, maybe the banks should use these to decide who is allowed access to online banking :-)

  39. Andyf

    //grumpy old fart mode

    I'd suggest that telling the difference between 'Eugene' and a real 13 year old would be pretty straight forward. Most real 13 year olds struggle to get their faces out of facebook or twitter long enough to utter more than a grunt.

    /off

  40. Alan Johnson

    Dod it really pass or is it just noise?

    Even if more than 30% of the judges in one test could not tell it was a machine this does not mean it actually passes because of uncertainty in the measurment. If for the sake of argument a judge has a 25% chance of thinking the machine is human (ie. the machine fails the turing criteria) and there are 20 judges then there is a 21% chance that 35% of judges will give a pass result. How many time si the turnig test run if you consider all teh entrants every year? Just probability/statistics will give a pass eventually. The turing test has been run many times so the fact that we get an outlier result should not be a surprise.

    The sample conversation was very poor and no noticeable improvement from early efforts.

  41. DropBear
    Joke

    If it's admissible to specialize as " a 13 year old foreigner"...

    ...I'm unveiling my brand new cutting-edge NSA chatbot:

    10 INPUT String

    20 PRINT "We most definitely did not and do not!"

    30 INPUT String

    40 PRINT "I'm sorry, that's classified."

    50 GOTO 10

  42. rekrab em
    Trollface

    We can pipe line noise through a teco editor and claim it to be a 1 year old.

    Voila! We've passed the Turing test.

  43. Ken Y-N

    I can't find the story again, but...

    A couple of years ago I read a story in the Telegraph (?) where the reporter was the human control for the Turing Test. The thing I noted as odd was that they were primed to pretend to be a computer, so it was really more a test of which was better at pretending to be what they weren't.

  44. JeffyPoooh
    Pint

    Turing Test 2.0

    The Turing Test is a failure. It has to be made much more serious, with major consequences; otherwise people will just guess.

    Here's the solution (additional rules):

    - If the judge is correct about it being just SW, then the software creation team are immediately executed.

    - If the judge is incorrect in the case where the SW fooled the judge, then the judge is immediately executed.

    - If the judge is incorrect in the case where an actual human pretended to be poor SW, then both are immediately executed.

    - If the judge is correct in the case where it is an actual human, then nobody gets hurt.

    - Everyone playing the Turing Test 2.0 must play 20 rounds minimum to reach the Finals.

    This should weed out the false claims.

    1. Charles 9
      Facepalm

      Re: Turing Test 2.0

      I hope you're joking, since you chose a Pint over a Joke Alert.

      They're SUPPOSED to guess (there are no false claims; just lousy guesses). That's the point of the test: the judge has to INFER (read: guess) whether or not the other end is human or not, based on the conversation. It's like playing To Tell The Truth: the imposters are trying to trick the panel into picking them instead of the real person. Why do we need to kill whoever's wrong? The judge doesn't know better (because he HAS to guess), and win or lose the software people get data for improving the AI.

This topic is closed for new posts.