back to article Dismayed by woeful AI chatbots, boffins hired real people – and went back to square one

Convinced that intelligent conversational assistants like Amazon Alexa, Microsoft Cortana, and Apple Siri are neither particularly intelligent nor capable of sophisticated conversation, computer boffins last year began testing a crowd-powered assistant embodied by Amazon Mechanical Turk workers. The chatbot, a people-powered …

  1. as2003

    > The abusive conversation lasted nearly three conversational sessions till (sic) the user realized it was humans.

    What does it mean when a human fails to pass the Turing test?

    1. Flocke Kroes Silver badge

      When a human fails a Turing test

      It often means the tester has unrealistic expectations for the abilities of AI.

      The fact that ELIZA has passed a Turing test shows that the results say at least as much about the tester as the subject.

    2. John Smith 19 Gold badge

      "What does it mean when a human fails to pass the Turing test?"

      This has been done in at least one short story.

      They are accepted for a job with the Tax office.

      1. TheElder

        Re: "What does it mean when a human fails to pass the Turing test?"

        Good question.

        Man with almost no brain

        But despite his minimal remaining brain tissue, the man wasn't mentally disabled - he had a low IQ of 75, but was working as a civil servant. He was also married with two children, and was relatively healthy.

        Not only did his case study cause scientists to question what it takes to survive, it also challenges our understanding of consciousness.

        Civil Servant with no brain. That explains a lot...

  2. Warm Braw Silver badge

    The number of suitcases a person can take on a plane

    This does raise the issue of whether you can trust the answer you are given, however deftly it is delievered.

    The problem with a lot of call centres is that the staff are little better than the participants in this project - they simply try to match questions against an FAQ and will spout any garbage that gets the caller off the phone before they get penalized for an excessive call duration.

    And, anyway, if organisations have large volumes of customer service calls, it's usually because they're doing something wrong. I was particularly irked recently when I tried to call a local council about a Countil Tax error (theirs) only to be informed there would be a 55 minute estimated wait time to speak to an agent, that if I in fact waited 60 minutes I would be cut off and have to redial and that I should resolve my query online - two months after I'd reported the error online and had no response. Fix your problems sooner (or stop making the errors in the first place) and your support load will drop accordingly and much more reliably.

    1. Peter2 Silver badge

      Re: The number of suitcases a person can take on a plane

      I spy with my little eye:-

      1) An SLA that all calls must be answered with a maximum of a one hour waiting time.

      2) A person working for the council negiotating with the provider who had never watched "Yes Minister", and had little imagination for ways of meeting the letter of a rule whilst comprehensively violating the spirit of that rule.

  3. Sureo


    It seems that the more people we have in the world, the more they want to automate stuff. Am I the only one who finds that paradoxical?

  4. Lee D Silver badge

    Gosh, so all that AI stuff was just nonsense? Whod've thunk?

    It was tosh from day one and it doesn't take much to realise it's just keyword hunting rather than actually interpreting what you're saying.

    Again: We don't have AI. It doesn't exist. It's not likely to (in any meaningful context) for decades. We have clever tricks made by advanced technology ("indistinguishable from magic") but we don't have AI and it falls apart the second you actually want to use it.

    P.S. this is the junk that's also running your magic-sensor car. Think about that for a moment.

    1. Andrew Orlowski (Written by Reg staff)

      "this is the junk that's also running your magic-sensor car "

      Not really, but then this is the tech industry's dirty little secret. Machine learning has its uses, but helping a self-driving car avoid hitting an old lady is not one of them. We should have a sweepstake on when the MSM will figure this out.

  5. Tom Paine Silver badge

    Well put

    "I don't see the path from what we have right now to a completely automated system that is as capable as me calling up a friend on the phone," he explained.

    40 years of AI research in a nutshell.

    1. stephanh

      Re: Well put

      AI research has definitely produced results;unfortunately it has been the victim of a constant hype/disappointment cycle. Recall the "AI winter" of the later 80's. Now we are again in the "hype" phase.

      AI research has produced stuff like A* path-finding and linear programming (LP) solvers. However, these things are now so well understood that we don't consider them "AI" anymore.

      AI has also produced a few "generic optimization" algorithms , such as neural networks, simulated annealing, and genetic algorithms. My prof used to joke that if you had no understanding of your problem and you had absolutely no clue what to do, you would choose one of those. They "work" but are slow and not guaranteed to give something approaching the global optimum.

      It seems every generation has to learn again that these "AI" methods are not a panacea and inferior to specialized approaches which leverage the structure of the underlying problem.

  6. Pascal Monett Silver badge

    "It was hoped by businesses the world over" . . .

    . . that they could finally kick out all those smelly, expensive meatbags and replace them by one single, shiny electronic server that they could then shut in a room and ignore for the rest of time while paying a minimal fee on power, air conditioning and replacement parts now and then.

    Sorry, Megabuck CEOs, seems like you're first going to have to invest another few hundred billion bucks in R&D before seeing that rosy future.

  7. vistisen

    So now we have people pretending to be machines pretending to be human! I think I can see a potential waste here. I have only used a chat once or twice, and once with Apple and I still am not sure whether I was talking to an AI or a person with no personality.

    1. TheElder

      I still am not sure whether I was talking to an AI

      Ask the chatbot this question:

      Will you inform me when you cease to exist?

      The only valid answer is NO.

      I have tested quite few. Answers have been Yes!

  8. Calimero

    Any AI weather prediction [next winter] ?

    Just kidding, it won't be another winter! Too expensive and disappointing for all the Singularity adepts... They can't be wrong! (or rather one cannot prove them wrong) But I bet the notion of AI-completeness will be re-phrased, re-thought, re-worked, re-defined, re-invented, and retracted.

  9. TheElder

    Hasn't managed to overcome poor human behavior.

    Undoubtedly why Mr. Musket is so worried. He has been shooting with his mouth a lot lately.

    So called AI Chat Bots are nothing better than the MIT Eliza. Just a few lines of code is all that is needed along with database access. Compare words with high probability of matching and that is it. The average American uses a vocabulary of only a few thousand words and use less than half of them 99% of the time. In many (most?) a lot of those words are right brain words such as Fuck, Shit, Asshole and so on. Those have nothing to do with critical thinking. They are strictly emotive responses to when shit happens not on the toilet (well, sometimes...)

  10. davenewman

    There are companies that have solved the natural language analysis of recognising what someone types in (e.g. - but not the problem of how to generate a situationally relevant answer, that gives you what you need to know rather than what you asked for.

  11. TheElder

    Recognising what someone types

    I doubt that would work very welkl with ny typing. I ten to make a loy of typos which ZI then must correct.

    No I did not to that on purpose. I was just typing fast and my fingers do not always obey my mind.

  12. Version 1.0 Silver badge

    Email versus voice?

    Which is the fastest way to resolve a problem? It's always a phone conversation, but can I get our young lads in sales and customer support to recognize that fact?

  13. Herby

    Simple: Artificial Intelligence...


    They have been trying for years, and haven't gotten much. I don't hold out much hope either. Of course, given the "Intelligence" of people these days, the standard they wish to obtain is getting lower and lower as time passes.

    Life goes on.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2020