Microsoft: Our AI speech recognition mangles your words the least Microsoft researchers working on AI computer speech recognition have reached a word error rate of 6.3 per cent, claiming to be the lowest in the industry. Hot on the heels of Google DeepMind announcing a “breakthrough” in AI speech recognition, Microsoft was quick to respond by saying it, too, has reached a “milestone” while …
"Computer speech recognition has come a long way"
It really, really has. I've been using Dragon for 10+ years now, and each new release gets better and better. As the effects of my CP become more noticeable, it's great that I can finally rely on my computer to understand 99% of what my thick Kiwi accent says. However, that's not my experience with the built in speech recognition in Windows 10. It's not awful, but it's not a patch on Dragon 15 for sure. If MS's new AI speech recognition i as good as claimed, I hope it does filter down to Win 10 sooner rather than later
More than 40 years of development and that is where we are? A comparison with human performance with the same material would be useful.
I keep comparing computer speech recognition with computer autonomous car driving and consider the former to be relatively a piece of piss.
Would you want to be a passenger in an autonomous car which incorrectly interprets what it 'sees' 6 out of 100 times?
Depends on your identification needs. With speech recognition you have a measurable goal which if you miss is very obvious, whereas with the 'seeing' for driving you only have a limited set of results. I don't care whether it's a person, another car or a tree. All I care about (from the car's perspective) is whether I'm going to hit it based on current trajectory. That would appear to be a lot simpler to get right than "Where can I find Sweden sour chicken?"
"I don't care whether it's a person, another car or a tree."
I care about not jamming on the brakes and having someone tail end me for the sake of a plastic bag blowing across the road or a tin can that looks much bigger on radar.
I care about it not even trying to interpret subtle clues about the future actions of pedestrians and other road users. Will it see a football as a threat? What about a kid that is about to run into the road after it?
Will it see horse shit on the road and take it as an indication there could be a couple of people on horses around the next corner? (an observation and interpretation I have correctly made on more than one occasion). Will it be even surer because it observed the horse shit was still steaming?
The amount of information a competent driver can collect and what can be interpreted from it is simply huge compared to recognising speech. Just look at the data rates 8kb/s for speech, how much all round stereo vision do you get in that?
"The amount of information a competent driver can collect and what can be interpreted from it is simply huge compared to recognising speech. Just look at the data rates 8kb/s for speech, how much all round stereo vision do you get in that?"
Absolutely true and I agree with the lot.
On the other hand, the larger problem is the typical driver will be able to say with certainty what the last tweet\facebookpost\sms that just came up was....
"More than 40 years of development and that is where we are? A comparison with human performance with the same material would be useful."
Agreed. I'd also like to know what sort of material was used. I could perhaps get correctly 6.3% of the words from a Glaswegian chav. (no offence, Scots!)
There's also the problem with words that sound alike but have different meaning. It's not just converting the sounds to words, the AI also needs to decipher the sentence - many people don't adhere to the correct syntax (and some don't always even make sense).
"I'd also like to know what sort of material was used"
If you try you can find examples. They are recorded telephone conversations. The one I found appeared to be a woman talking to her mother about an upcoming party or reunion. Fast two way conversation and pretty clear apart from some person names which I would be guessing at.
Just a comment. If you were to hire a secretary with a 6.3% error rate, she would be out the door the first day!
I have memories of some work done in the 60's that could (after training) understand the words "one, "two", "three" and "four". The training took a while, and the program used speaker specific samples. Pretty crafty for around 50 years ago.
So, yes we have come a long way, but more needs to be done. In my case, Siri works pretty well though.
Also at the end of that you have a string of text. Then a completely different AI problem to parse that and decide what is wanted.
To have really good speech recognition you need context and meaning unless it's very simple list of commands for a car radio or dialling a phone.
Since most people have difficulty using search well with a keyboard, I don't think adding speech on top is going to result in successful queries or searches except for expert users.
I've had 34 years of development, and I still have a double-digit error rate when dealing with accents outside of my home region, especially those from Deep South of the US, Newfoundland, and much of the non-London portion of England.
And I have better luck with people from India, Bangladesh, and Pakistan, talking to me through a cell phone in those locations, than whatever the Florida Man has to say.
I work with a lot of international teams, and some of them are just plain unintelligible. Having some sort of AI, even at 5% error rate, to translate their speech into text would help immensely when doing phone calls with them.
Well, Skype has a translate button, it presumably works for English to English.
I doubt it would manage the reliability seen here but Skype is usually much higher fidelity than an old-fashioned phone call, in fact it is often better than a modern phone call since they are all digital but Skype seems to throw a lot more bits at the problem.
Nobody talks to their computer.
Really, if you do anything meaningful on your PC, you don't talk to it. That's because if you do you'll seriously piss off the people you're working with or those sitting around you. And it's slow.
The technology was pretty much sorted in the 1960s. Yes, that's around 50 years ago. So why hasn't it taken off? Because 1) it's inefficient; 2) people don't want to do it; and 3; it pisses off anyone who can hear you.
That's all.
Well at least the alternative BBC did - this being the broadcaster that can be seen on the UK fly-on-the wall documentary series W1A that was broadcast on Monday this week.
The latest version of their in-house developed Syncopatico automatic live sub-titling software was achieving 93% accuracy - with just the slight problem that the 7% errors were all related to names.
So, our beloved PM became Tweezer May, the Russian president Vladimir Puking and the actress Maggie Smith was transformed into Dame Baggy Smith.
Go Microsoft!
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2025
