Self-driving truck boss: 'Supervised machine learning doesn’t live up to the hype. It isn’t C-3PO, it’s sophisticated pattern matching'

Let's get cracking with some machine-learning news. Starksy Robotics is no more: Self-driving truck startup Starsky Robotics has shut down after running out of money and failing to raise more funds. CEO Stefan Seltz-Axmacher bid a touching farewell to his upstart, founded in 2016, in a Medium post this month. He was upfront …

[Pascal Monett]

Finally, a proper description of what the media dubs "AI" actually is

Sophisticated pattern matching. Sounds about right. When I did the Google Beginners Course in AI, I followed a dozen lectures in statistics. There wasn't a hint of AI, it was just how to define a slice of dataset to get the desired result.

Now a head of company has finally called it. Good. I'm not expecting that to actually change the media's mind, but I'm glad that somebody is putting "AI" back into its place.

[Artem S Tashkinov]

Re: Finally, a proper description of what the media dubs "AI" actually is

"AI" is a whole lot better to flog to venture capitalists than "pattern matching" which sounds a lot like comparing integers. And the worst thing about pattern matching in AI currently is that the resulting models turn out to be black boxes we neither understand, nor can tune to avoid misidentification. In short I expect a new AI winter sooner rather than later unless we have a major breakthrough in the intelligence part of AI which doesn't seem likely.

[BillG]

Re: Finally, a proper description of what the media dubs "AI" actually is

Trying to model every scenario is not only impossible but expensive. “In fact, the better your model, the harder it is to find robust data sets of novel [new] edge cases. Additionally, the better your model, the more accurate the data you need to improve it,” Seltz-Axmacher said.

In other words, AI is really G.I.G.O., proving once again you are only as good as your data set.

[Anonymous Coward]

Re: Finally, a proper description of what the media dubs "AI" actually is

That was my big takeaway from this. The company failed not because it their pattern matching could not be developed further but because they were honest with venture capitalists about the limitations and time frame.

There's a place for pattern matching and it can improve with time (and money) but true AI is a pipedream.

Clippy will never become HAL.

[Alan Brown]

Re: Finally, a proper description of what the media dubs "AI" actually is

A lot of "human stuff" is pattern matching too - the difference being that we have fuzz filters and the tendency to press on regardless if we don't recognise the patterns or miss patterns we should have been trained on (like feet moving under parked cars at the side of the road) - frequently with.... unfortunate.... consequences)

What it comes down to is _ACTUARIES_

Robot drivers don't _need_ to be perfect to start getting an advantage in the insurance stakes.

They just need to be better than the average human

That's a stunningly mediocre bar to cross and when you couple that with the issue that a lot of humans actively dislike driving AND it only really takes a few poor drivers to really screw up a highway(*), there are some real incentives to get the angry+distracted monkeys out from behind the wheel anyway.

A normal driver will make a couple of mistakes a minute. Even Lewis Hamilton lets his attention slip sometimes. Our roads and road rules are setup so that it takes _AT LEAST_ 2, usually 3 serious errors to cause a crash - and frequently one of the most serious errors is poor road design, which is a major problem in the UK and yet noone's taking the people responsible (local authorities) through the courts.

(In the UK those responsible are frantically trying to avoid responsibility for by pointing everything as "driver error" even when egrarious road design flaws are pointed out - such as roads with 40mph speed limits that are designed for 60mph traffic - traffic speed is intrinsically set by road design not by signs, as the US DOT worked out decades ago and started clamping down on dangerously LOW speed limits set for revenue-raising purposes. The UK has a serious case of stats fiddling going on in many areas to mislead safety researchers about traffic speeds too, apparently for financial reasons - reporting lower speeds gives higher rewards and a pat on the back, etc)

(*) Just about ALL the traffic jams, nose-to-tails, phantom slowdowns, etc etc are caused by people breaching the rules about following distances or entering intersections when they can't exit them, etc. The assholes who are determined to engage in a Bitumen Waltz to get past everyone else in a jammed up mess are the CAUSE and continuation of those jammed up messes.

[c1ue]

Re: Finally, a proper description of what the media dubs "AI" actually is

Robot drivers are subject to all manner of attacks which human drivers aren't bothered by including

data poisoning and spoofing.

How will actuaries react to that?

[Brian Miller]

Re: Finally, a proper description of what the media dubs "AI" actually is

But why is this news? All along "AI" has been pattern matching, and has always been shown to be pattern matching. I remember early demonstrations of AI telling the difference between headshots of men and women, and then being confused when given headshots of the Beatles.

The real question is, can we use pattern matching to reliably navigate a multi-ton object without human intervention? Sure, if the object is on rails, and isn't subject to major random interference.

[Artem S Tashkinov]

IOW, the current crop of artificial "intelligence" does neither see, nor understand, nor reason about the world around us and it makes it extremely dangerous to rely on it.

We have yet to even begin to crack the intelligence which exists even in worms and ... plants.

[Francis Boyle]

Brain the size of a planet and you've got me controlling traffic

I don't wan't any sort of intelligence controlling the vehicles I travel in. Trains have been extremely safe for something like two centuries despite for most of that time relying on a complex system of interlocking metal bars to prevent things crashing into other things. I want safety systems to be built on simple robust logic. That's one of the reasons I think Musk's refusal to have anything to do with Lidar is a serious mistake.

[Trigonoceps occipitalis]

Re: Brain the size of a planet and you've got me controlling traffic

"I don't wan't any sort of intelligence controlling the vehicles I travel in."

Judging by what I see on the roads every day that is oh so true.

[Mage]

Such honesty

“Supervised machine learning doesn’t live up to the hype,” he declared. “It isn’t actual artificial intelligence akin to C-3PO, it’s a sophisticated pattern-matching tool.”

We have not advanced much, if at all, since 1980s "Expert Systems". I argued in the 1970s that if we knew how to do it we would simply have slow AI, if it was a question of needing more computing power. We have very much more CPU speed, RAM, storage and the Internet now. Still rubbish translation (hint it uses pattern matching and known "Rosetta translations", not real grammar and parsing as envisaged in the 1960s and 1970s). Hardly better spelling and grammar checkers than 35 years ago. Image Recognition that is really easily fooled pattern matching using big curated databases.

AI is more about hype and marketing than research or real products. A few big "tech" companies that didn't exist in the Mainframe era are more interested in selling overpriced gadgets or stealing usage data to help sell adverts. Not doing real AI research.

I learnt programming because I wanted to help develop AI. I think I read too much SF in the 1960s and 1970s. Modern SF with AI now seems very self-indulgent, egotistical and pointless. We are still no closer to defining what natural intelligence is, or why we have language but the smartest animals only seem to have vocabularies. Nor why it seems there is little correlation between brain size and tool using / problem solving / vocabulary in animals (compare corvids, dogs, horses, goats, dolphins, chimps, whales). There is none in humans.

Note IQ tests and Psychometric tests only compare people with the same background and training. Not at all intelligence, though we don't quite know what intelligence is. It's not purely about language, though that's important. A vocabulary IS NOT language. One is simply recall and matching and the other is still a bit mysterious.

[Anonymous Coward]

Tesla exists

You know Tesla exists right?

These things that you're claiming are not coming to pass and never will, they're like, in showrooms and on your phone and on the net now, right?!

Grab Google translate now and do real-time voice to voice translation like a 'Universal Translator' of 80's SciFi movies.

This idiot didn't know how to break a complex problem down into parts small enough to make the training set for each viable, but others do. His approach needed a ridiculously large training set he could never have delivered.

But a lot of these AI mini companies are really investor or patent troll plays not companies ever intended to make the thing they raised money for. So a lot of them will fail.

[Anonymous Coward]

Re: Tesla exists

Tesla's "self driving" works until it doesn't and you crash into something.

Google Translate works until it doesn't and the results get laughed at on social media.

There is no "Intelligence" in AI, just a lot of "Artificial".

[Anonymous Coward]

Re: Tesla exists

Human intelligence trying to use a foreign language gets laughed at too. And human intelligence works until human stupidity causes a crash.

There are good reasons to believe AI is overhyped, but using things that AI does that people also do is not a valid argument.

[Anonymous Coward]

Re: Tesla exists

Which is buggy but useful pattern recognition. Not AI. It works the same as it did in the 90s, but now fits in your hands (actually it does not, the dataset is often GBs large, and specific data pruned from a server and sent to your phone), instead of a desktop PC (they had speech to text in 1995!).

[Anonymous Coward]

@AC Re: Tesla exists

As someone who has worked in mapping and image processing/detection, I can tell you that its not as easy as you think and its now dawning on some of these idiots that for the past 2 years they only scratched the surface of the problem.

Object detection is only part of it. Try something 'simple' like lane detection on a well worn Chicago street where you don't always see your lane lines and then the issue of trying to guess where you should be.

So yeah, fully autonomous driving is still a ways off.

[c1ue]

Re: Tesla exists

The main benefit of Tesla's "AI" is that it is mostly killing Tesla drivers.

Not us innocent human pedestrians, bicyclists and other drivers.

[Alan Brown]

Re: Such honesty

"Note IQ tests and Psychometric tests only compare people with the same background and training. Not at all intelligence"

Yup, but then researchers turn around and say with confidence "someone with an IQ of 100 in the 1930s would score 80 in a contemporary IQ test - ergo we're getting smarter!" - when what you're comparing is apples to orange juice.

[Anonymous Coward]

Re: Such honesty

Not smarter. The comparison is valid but shows how the computer revolution and rise of white collar work has resulted in a generally higher level of abstract reasoning about patterns, which is what IQ measures. If IQ measured practical ability, a lot of people wouldn't do very well today.

I don't think my grandfather would score highly on a modern IQ test, but he could repair watches and clocks, use a lathe and milling machine, mend shoes, make things out of leather and wood and so on. I think today he would be in the top 1% for practical skills, but in the 1950s it would have passed without comment.

[Francis Boyle]

Re: Such honesty

We can't give people from the thirties contemporary IQ tests but we can give contemporary people thirties IQ test and they consistently and measurably better than thirties people did. Whatever you think of IQ tests (I don't think much) it's clear that people are getting better at abstract reasoning.

[jake]

Re: Such honesty

We have not advanced much, if at all, since the 1960s AI investigations lead into the so-called "AI Winter" in the 1970s.

[Mike 137]

Re: Such honesty

All good points, Mage. However, as most of our education system is now based on prompted regurgitation of rote memorised statements (multiple choice testing, which is essentially pattern matching), we're operating more and more like the machines, so there may be a point where artificial and real intelligence become indistinguishable.

[Anonymous Coward]

Re: Such honesty

Ah, the rose tinted spectacles.

The education of the past was mainly rote learning: amo, amas, amat, amamus, amatis, amant. The products of Argentina. The method of long division. The Kings and Queens of England, with dates.

A small minority went into fast streams where some thinking was in order and got to do A and S level or Oxbridge entrance where they asked you some open ended questions you had to think about. I think in my day about 1% got into Oxford or Cambridge, 8% went to university. And how many of us ever did any truly original thinking?

When Feynman went to Brazil after WW2 to take a look at physics education in their universities, he reported that it was all rote learning, no real thinking involved. It wasn't just us.

I suspect all you are seeing is that more people go to university but the proportion of thinkers in the population hasn't changed much.

[Anonymous Coward]

With 5g

His approach is a perfect hybrid to get self driving vehicles working. A remote driver simultaneously monitoring a few cars for the edge cases. As a more experienced driver they may even be able to handle emergencies better than a driver.

As the Software gradually improves the remote driver can handle increased number of simultaneous drives (each drive may be monitored by more than one remote driver providing redundancy)

A first use might be delivering fast food.

The service might only cost £5 an hour so popular with commuters who would rather play on the internet than drive

[Irongut]

Re: With 5g

That's not self driving, that's remote control.

[Anonymous Coward]

Re: With 5g

99% self driving and 1% remote control

[Anonymous Coward]

Re: With 5g

Prey tell, how does such a remote handler help when 2 incidents happen at the same time? We don't phone in aircraft pilots for a reason, and the reason is not the lag in video feed or the resolution of the cameras.

[biffed]

Re: With 5g

We, er the US does. But we use it to murder people from above.

[Anonymous Coward]

Re: With 5g

More that one remote person monitoring each stream

[John Brown (no body)]

Re: With 5g

"As the Software gradually improves the remote driver can handle increased number of simultaneous drives (each drive may be monitored by more than one remote driver providing redundancy)"

Come back and suggest again that after the military decide that a single operator can handle multiple drones in uncongested airspace.

[Anonymous Coward]

Re: With 5g

A couple of hundred drones requires far less manpower than millions of cars

[IGnatius T Foobar !]

The phrase "artificial intelligence" is the real problem.

The actual phrase "artificial intelligence" has caused more problems than anything else. People hear that phrase and they think "oh, we're right around the corner from having a computer equal to a human brain." This is prima facie stupid, of course. Machine learning, expert systems, neural networks, all of these things can be trained (by humans) to automate tasks in specific problem domains. The phrase "artificial intelligence" was stupid when it became popular in the 1980's, and it was stupid again when it became popular in the 2010's. The phrase needs to be retired. Anyone using it should automatically be flagged as a non-expert.

[jake]

Re: The phrase "artificial intelligence" is the real problem.

We thought it was pretty stupid in the 1960s at SAIL, too. But it got research grants. The more things change ...

[Henry Wertz 1]

"A remote driver simultaneously monitoring a few cars for the edge cases. As a more experienced driver they may even be able to handle emergencies better than a driver. As the Software gradually improves the remote driver can handle increased number of simultaneous drives"

No and no. A remote driver can't be monitoring "a few" cars, a vehicle emergency is an emergency, not something that can dealt with a second or two later when someone looking at several screens realizes yours is going wrong. Also, given there can and will be vehicles beside and behind these vehicles; is that remote driver really keeping track of what's beside and behind 3 or 4 (or more) cars? Oh, they aren't? Then their reaction may just cause the vehicle to sideswipe other vehicles or be violently rear ended. And no, as the software improves the remove driver absolutely cannot handle an increased number of drives, that would just make the reaction time and awareness problems even worse.

[Il'Geller]

NIST TREC QA wanted to find the right patterns into a gigantic number of texts. The resulting technology can be called AI because the patterns should be found by their meaning.

[John Brown (no body)]

It doesn't "find" patterns. It matches patterns it's been told about to patterns that exist in the text. That's exact definition of patterna matching and not even close to AI.

Unless you mean the pretty white "rivers" you sometimes accidentally get in pages of text?

[jake]

Rivers.

Artificial Malevolence, perhaps ... if you're a typesetter, you'll understand.

[Il'Geller]

Re: Rivers.

A matching is possible if both a question and its answer are known. The developed for NIST TREC QA technology is for the unknown questions and answers.

After the unknown answer is found it’s tried. If the feedback good, the AI memorizes the pair. If not, a new text is searched for, which is Machine Learning, and the new attempt follows.

Starksy Robotics has shut down after running out of money because they tried to discover all possible questions and answers. However my technology for NIST TREC QA doesn’t do this and is only a tiny fraction what Starsky paid.

[Anonymous Coward]

I have always preferred "supervised NN's are glorified curve fitting"

It covers a broader swath of what these half-bright SV types give themselves so much credit for.

[ocratato]

Pattern matching is not intelligence

Not too many years ago it was thought that finding patterns was something that was too difficult for software and would be something that humans would always have to do.

We now have software that is often better than humans at finding patterns, though it is often still in specific domains.

What this company has discovered is that complex activities like driving require more than just the ability to recognise patterns. It can be enough in a well regulated environment to use this as part of a control system, but it cannot provide a human like response on real world roads.

The missing ingredient is the ability to "understand" the world around it. We use our pattern matching ability to recognise things, but we then use that information to build up a model of our surroundings. That model has all sorts of data that we have learnt in a life time of experience attached to the objects in that model.

It does not seem to be an impossible task to build a software system that could build such a model and thus have something approaching an understanding of the things it can see. Then we might have autonomous vehicles that won't be a danger to everyone.

[USER100]

Re: Pattern matching is not intelligence

> The missing ingredient is the ability to "understand" the world around it

Exactly. No amount of 0s and 1s will ever be sentient. That is the truth which adherents of 'strong AI' fail to grasp.

At some basic level, 'intelligence', however you measure it, requires life.

[Anonymous Coward]

Re: Pattern matching is not intelligence

[proof required].

An alternative and just as unverifiable statement might be "anything showing intelligence is technically alive."

[T. F. M. Reader]

Basic stuff before AI

While the Commentariat discusses AI vs pattern matching as applied to autonomous vehicles, I am struck by something else in the article that looks much more basic to me. The stuff in Waymo's Open Dataset Challenge: frankly, and possibly naively, I'd expect those problems to be solved "beyond reasonable doubt" before any talk of autonomous driving can be taken seriously.

[John Brown (no body)]

Re: Basic stuff before AI

Yes, that took me aback too. Not only are the challenges for things that should already be done before getting out on the road, but the "prizes" are a joke. They should already be paying experts in the field the big bucks to do this, not offering a pittance for something potentially worth a lot of money. I have no doubt the T&Cs will give Waymo total ownership of any submitted works.

[Anonymous Coward]

AI - Accident Investigation

In 50 years of driving I've had two auto accidents, each time I hit a car driving towards me that suddenly turned in front of me.

Last week I was using Google "Driving" to give me directions in a remote area of the USA. I approached an empty road that appeared to be a crossroads and it told me to take the next left ... about half a mile down the road the oncoming truck started flashing its lights at me and I realized that I was driving the wrong way down a dual carriageway with trees on both sides so there was no way to see the other carriageway traffic at all - luckily there was very little traffic. I managed to find a cop crossover and got back to the right side very quickly but those AI driving instructions scared the hell out of me. Google maps knows which side of the road you are on and it didn't bother to say anything, just "take the next left".

[jake]

Re: AI - Accident Investigation

You used the phrase "dual carriageway", which tells me you were a visitor to these fair United States (we call it a divided highway). I rather suspect you were looking for the road signs in the wrong places when you made your turn. One of those places was Google, to be sure, but I'm fairly certain you missed the obvious local highway markings.

Can you please provide us with the exact intersection, so we can eyeball it for ourselves? More curiosity than anything else.

[Anonymous Coward]

Re: AI - Accident Investigation

There are things in the US road system that amaze the visitor, and you quickly understand why such excessively powerful cars are needed to deal with the strange intersections and merges. Roundabouts seem more complicated to the visiting American over here, but satnavs and Google Maps are really rather good at telling you what lane you should be in.

Tl;dr the US highway system seems very simple with its frequent 4 way junctions, but they often seem designed to confuse the navigation system and the unwary visitor.

I'm the idiot that drove around Louisville for half an hour once looking for the exit onto the freeway because I couldn't believe that what looked like an entrance to an underground car park, with concrete sidewalls blocking all visibility, was in fact a merge onto a 4 lane road.