Cruise emits software fix after self-driving car slams into bus Cruise pushed a software update to its fleet of 300 self-driving cars operating in San Francisco after one vehicle crashed into the back of a bendy bus. On March 23, one of the upstart's driverless vehicles hit a Municipal Transit Authority (Muni) bus when the car failed to detect the bus had decelerated and didn't brake in …
The speed with which Cruise analysed and rectified the problem is impressive.
But I wonder how many other 'unique' situations these vehicles are going to encounter. I reckon that pretty much every time I go for a drive, I see some weird event that I've never encountered before. Some random recent examples:
(1) When joining a large motorway-junction roundabout, met a vehicle that was reversing back round the roundabout.
(2) Had to slow down to let a peacock cross in front of me.
(3) Had to go the wrong way round a traffic island because someone had parked in such a stupid place that I couldn't get through on my side.
(4) Had to disobey a red light at some roadworks, as it became obvious that the lights were stuck on red in both directions.
Really not sure how a self-driving vehicle would have handled any of those. And they're just a few examples.
Still seems easy concept that there's this large back end of a bus in front of you and you need to avoid running into the back of it.
The fact that there's some middle section which bends 30' further up the road seems neither here nor there.
Sounds like something in the basic AI model is missing. Rule 1 don't run into things. The fact that bendy buses might act a little different to other buses is a minor point.
Exactly this - If your algorithms can't keep the vehicle further away from the thing in front than the distance it takes to stop, then it shouldn't be allowed on the road; this is the standard expected of human drivers [1].
What if someone was crossing the road behind the bus? Would that be another unique situation. I would like to see autonomous vehicle operators held to a standard where this kind of fuck-up is subject to some weight of criminal responsibility if they can not show reasonably practicable measures undertaken by competent persons to assure others' road safety. Yes I am grumpy this morning.
[1]: At least when they pass their test. Many will drift out of cal over time.
It reminds me of the stories from a couple of months back of a human beating Go computers which surprisingly wasn't covered here on the register. They exploited the fact that the systems don't understand Go they have just been trained to react in certain ways to certain moves. Play a game the computer isn't expecting and it doesn't know what to do.
Same thing here, it seems this AV has been trained to handle certain situation on the road but no one has trained to to avoid hitting things.
If the "back" is facing off to the right (or left if you drive on the other side of the road) while making a turn a person might think "I better keep my distance since that back section is coming around" while an AI measuring distance via LIDAR will think it can come up close enough that it gets whacked by the bendy backside.
Except it is clearly stated that it "rear-ended" the bus, rather than being sideswiped by it.
My understanding is that it "saw" the front bit, which it saw as a bus". It also "saw" the back bit, which it (probably correctly) identified as a trailer attached to said bus. It then proceeded under the, correct, assumption that the trailer would follow the front end at the same speed as the front. The missing bit is that, with the front end completely obscured by the trailer section, it couldn't "see" the front part slow to stop and thus, in the absence of any evidence to the contrary in its limited world view, assumed that the trailer was going to keep on going at the same speed.
This is where ML and AI diverge. ML models may look smart, but completely lack any ability to think outside the box or even perceive that there is a box to think outside of.
> Had to disobey a red light ...
It would definitely be annoying to be stuck in an autonomous car with no where to legally go. On the other hand it could inform the fleet of the problem so all other autonomous vehicles route around the situation. Public transit has similar issues when buses break down or subways are temporarily stuck between stations, you just have to wait for help to arrive or the issue is resolved. Then again you could just abandon the car and catch a taxi, after all the car can find its own way home.
"Then again you could just abandon the car and catch a taxi, after all the car can find its own way home."
Will the Johnny Cab allow you to end the ride and get out or will it tell you to wait so it can move to a safe place for you to get out. Since it's stuck, it can't and if you find the door will still open for you, you might get fined or banned from the service. If you can't end the ride before your intended destination, you may not be given the option to pay and bailing out might leave the meter running or show you as not having paid. I have to wonder if the programmers have thought of these situations and what the system will do if a person just gets out while the car is stopped in traffic.
As IT people using systems every day, we already know the answers to the questions you pose. We're screwed. System only ever cover the majority cases. Edge case are ignored. If the "system" doesn't work the way you want it to, then YOU have to change what you do to match the "system".
The passengers of the first autonomous vehicle to approach this point after the problem has been fixed are in for a nasty surprise.
===
No matter how much thought is applied to the various scenarios there will always be exceptions.
L.T.C. Rolt's "Red For Danger" should be mandatory reading for anyone in the autonomous vehicle industry. Yes, it is about railway accidents, but if after over 100 years of passenger railways there are still accidents (where vehicles are constrained to rails), what hope is there for road users?
I wonder how these self-driving cars would handle driving in Malta, I understand there they drive 'on the shady side of the street'.
This ML (Machine Learning) and AI (Artificial Intelligence) is all very well and good, but it is not AW (Artificial Wisdom), yet, it just seems to think it is.
My experience in Malta is that the Maltese drive on whichever side of the road has the fewest potholes. Two Malts approaching each other on the road from opposite directions will only appear to be playing Chicken; their natural pleasant nature* means that everything just works harmoniously.
*Footnote here. This "natural pleasant nature" does not apply to migratory birds, or (innocent) German tourists. In the former case, hunting guns blaze; in the latter, memories of 1940-42 remain raw...
In the later '90s, 'Budget Travel" ran some adverts that took up the whole of the rear of a double-decker bus in Dublin - no idea if it was just the 1 bus or were there several - the vinyl wrap was of a tanned model wearing a (yellow?) bikini - the slogan was "Look like the back of a bus". What would an autonomous vehicle make of that. Granted, the figure was about 3 to 4 times life-size. Whichever, it shouldn't hit either.
That also reminds me of an incident when I was in college - someone I knew (honest, it wasn't me), ended up crashing into the back of a car - the car was fine, he was fine, the front wheel was relatively fine, but the forks were bent. The reason for the momentary loss of concentration? The two young ladies who happened to be walking by
> could make the whole vehicle appear completely invisible to a computer.
True if it relies solely on cameras. If lidar or radar is used graphics would have no effect. However, non-metallic vehicles or trailers could be a problem for radar, light absorbing paints could stymie lidar.
Best to have a mix of sensors. I don't think I would trust a system that relies only on cameras, it is more reassuring to know the car can also "see" in ways that I can't.
"Since the AV had previously seen the front section and recognized that the bus could bend, it predicted that the bus would move as connected sections with the rear section following the predicted path of the front section. This caused an error where the AV reacted based on the predicted actions of the front end of the bus (which it could no longer see), rather than the actual actions of the rear section of the bus. That is why the AV was slow to brake."
What? Guessing the actions of the front half of a bus when you have lost sight of it is a bit dodgy in itself, but not being able to detect the position/range/closing speed of the back half of a bus to the extent that you collide at 10mph is absolutely ludicrous. What sort of sensors is it using? A ouiji board?
This is an ideal accident from Cruise's point of view.
One, it's trivial. No one was hurt, or much scared, and damage was tiny.
Two, it's an easy fix. They whacked out a patch within days, and "voluntarily recalled" before any relevant regulator could even get their A into G. (Of course the code change won't affect any other situations, what a strange suggestion. I'm sure it's been comprehensively regression tested.)
Three, it's a sufficiently unusual scenario that they can spin this whole story about how it happened and how they've now altered their code so it doesn't happen again. Thus painting a picture of exactly the sort of gradual improvement and learning from experience that we'd all like to believe in.
Four, and perhaps most importantly, it's relatable. Who hasn't had at least a near miss caused by a moment of inattention in traffic? Makes the car seem more human.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
