back to article Cruise emits software fix after self-driving car slams into bus

Cruise pushed a software update to its fleet of 300 self-driving cars operating in San Francisco after one vehicle crashed into the back of a bendy bus. On March 23, one of the upstart's driverless vehicles hit a Municipal Transit Authority (Muni) bus when the car failed to detect the bus had decelerated and didn't brake in …

  1. David M

    Unique error

    The speed with which Cruise analysed and rectified the problem is impressive.

    But I wonder how many other 'unique' situations these vehicles are going to encounter. I reckon that pretty much every time I go for a drive, I see some weird event that I've never encountered before. Some random recent examples:

    (1) When joining a large motorway-junction roundabout, met a vehicle that was reversing back round the roundabout.

    (2) Had to slow down to let a peacock cross in front of me.

    (3) Had to go the wrong way round a traffic island because someone had parked in such a stupid place that I couldn't get through on my side.

    (4) Had to disobey a red light at some roadworks, as it became obvious that the lights were stuck on red in both directions.

    Really not sure how a self-driving vehicle would have handled any of those. And they're just a few examples.

    1. ChoHag Silver badge

      Re: Unique error

      Which part of a bus being on a road in a busy city is the unique part?

      1. DS999 Silver badge

        Re: Unique error

        The part where the bus is bendy.

        1. Anonymous Coward
          Anonymous Coward

          Re: Unique error

          Still seems easy concept that there's this large back end of a bus in front of you and you need to avoid running into the back of it.

          The fact that there's some middle section which bends 30' further up the road seems neither here nor there.

          Sounds like something in the basic AI model is missing. Rule 1 don't run into things. The fact that bendy buses might act a little different to other buses is a minor point.

          1. drand

            Re: Unique error

            Exactly this - If your algorithms can't keep the vehicle further away from the thing in front than the distance it takes to stop, then it shouldn't be allowed on the road; this is the standard expected of human drivers [1].

            What if someone was crossing the road behind the bus? Would that be another unique situation. I would like to see autonomous vehicle operators held to a standard where this kind of fuck-up is subject to some weight of criminal responsibility if they can not show reasonably practicable measures undertaken by competent persons to assure others' road safety. Yes I am grumpy this morning.

            [1]: At least when they pass their test. Many will drift out of cal over time.

            1. Anonymous Coward
              Anonymous Coward

              Re: Unique error

              It reminds me of the stories from a couple of months back of a human beating Go computers which surprisingly wasn't covered here on the register. They exploited the fact that the systems don't understand Go they have just been trained to react in certain ways to certain moves. Play a game the computer isn't expecting and it doesn't know what to do.

              Same thing here, it seems this AV has been trained to handle certain situation on the road but no one has trained to to avoid hitting things.

          2. DS999 Silver badge

            Think about it around a curve

            If the "back" is facing off to the right (or left if you drive on the other side of the road) while making a turn a person might think "I better keep my distance since that back section is coming around" while an AI measuring distance via LIDAR will think it can come up close enough that it gets whacked by the bendy backside.

            1. TeeCee Gold badge

              Re: Think about it around a curve

              Except it is clearly stated that it "rear-ended" the bus, rather than being sideswiped by it.

              My understanding is that it "saw" the front bit, which it saw as a bus". It also "saw" the back bit, which it (probably correctly) identified as a trailer attached to said bus. It then proceeded under the, correct, assumption that the trailer would follow the front end at the same speed as the front. The missing bit is that, with the front end completely obscured by the trailer section, it couldn't "see" the front part slow to stop and thus, in the absence of any evidence to the contrary in its limited world view, assumed that the trailer was going to keep on going at the same speed.

              This is where ML and AI diverge. ML models may look smart, but completely lack any ability to think outside the box or even perceive that there is a box to think outside of.

          3. Sorry that handle is already taken. Silver badge

            Re: Unique error

            It seems to me that the issue might be that self-driving models that are purely reactive are very slow, so they need to be predictive too. And in this case the predictive part took precedence over the reactive part and made a mistake.

      2. An_Old_Dog Silver badge

        Re: Unique error

        The "unique" part is that Cruise hadn't yet thought of that specific event, in those specific circumstances yet ... almost as if it weren't usiing "AI/ML" at all, but instead was using a huge if( ... ) ... elseif( ... ) ... elseif( ... ) ... endif-like statement.

        1. David Hicklin Bronze badge

          Re: Unique error

          > if( ... ) ... elseif( ... ) ... elseif( ... ) ... endif-like statement.

          Isn't that what AI/ML is after all?

    2. zuckzuckgo Silver badge

      Re: Unique error

      > Had to disobey a red light ...

      It would definitely be annoying to be stuck in an autonomous car with no where to legally go. On the other hand it could inform the fleet of the problem so all other autonomous vehicles route around the situation. Public transit has similar issues when buses break down or subways are temporarily stuck between stations, you just have to wait for help to arrive or the issue is resolved. Then again you could just abandon the car and catch a taxi, after all the car can find its own way home.

      1. MachDiamond Silver badge

        Re: Unique error

        "Then again you could just abandon the car and catch a taxi, after all the car can find its own way home."

        Will the Johnny Cab allow you to end the ride and get out or will it tell you to wait so it can move to a safe place for you to get out. Since it's stuck, it can't and if you find the door will still open for you, you might get fined or banned from the service. If you can't end the ride before your intended destination, you may not be given the option to pay and bailing out might leave the meter running or show you as not having paid. I have to wonder if the programmers have thought of these situations and what the system will do if a person just gets out while the car is stopped in traffic.

        1. John Brown (no body) Silver badge

          Re: Unique error

          As IT people using systems every day, we already know the answers to the questions you pose. We're screwed. System only ever cover the majority cases. Edge case are ignored. If the "system" doesn't work the way you want it to, then YOU have to change what you do to match the "system".

      2. Ken Moorhouse Silver badge

        Re: it could inform the fleet ... so all other autonomous vehicles route around the situation

        The passengers of the first autonomous vehicle to approach this point after the problem has been fixed are in for a nasty surprise.

        ===

        No matter how much thought is applied to the various scenarios there will always be exceptions.

        L.T.C. Rolt's "Red For Danger" should be mandatory reading for anyone in the autonomous vehicle industry. Yes, it is about railway accidents, but if after over 100 years of passenger railways there are still accidents (where vehicles are constrained to rails), what hope is there for road users?

    3. Roopee Bronze badge
      Stop

      Re: Unique error

      Pretty much the comment that I was going to make, complete with my own sample of unique situations that would have stymied any predictive algorithm. There are millions of unpredictable idiots with driving licences out there on our badly-designed roads...

    4. Anonymous Coward
      Anonymous Coward

      Re: Unique error

      Do you live in Florida?

    5. Eclectic Man Silver badge

      Re: Unique error

      I wonder how these self-driving cars would handle driving in Malta, I understand there they drive 'on the shady side of the street'.

      This ML (Machine Learning) and AI (Artificial Intelligence) is all very well and good, but it is not AW (Artificial Wisdom), yet, it just seems to think it is.

      1. David 132 Silver badge

        Re: Unique error

        My experience in Malta is that the Maltese drive on whichever side of the road has the fewest potholes. Two Malts approaching each other on the road from opposite directions will only appear to be playing Chicken; their natural pleasant nature* means that everything just works harmoniously.

        *Footnote here. This "natural pleasant nature" does not apply to migratory birds, or (innocent) German tourists. In the former case, hunting guns blaze; in the latter, memories of 1940-42 remain raw...

  2. Anonymous Coward
    Anonymous Coward

    Look like the back of a bus

    In the later '90s, 'Budget Travel" ran some adverts that took up the whole of the rear of a double-decker bus in Dublin - no idea if it was just the 1 bus or were there several - the vinyl wrap was of a tanned model wearing a (yellow?) bikini - the slogan was "Look like the back of a bus". What would an autonomous vehicle make of that. Granted, the figure was about 3 to 4 times life-size. Whichever, it shouldn't hit either.

    That also reminds me of an incident when I was in college - someone I knew (honest, it wasn't me), ended up crashing into the back of a car - the car was fine, he was fine, the front wheel was relatively fine, but the forks were bent. The reason for the momentary loss of concentration? The two young ladies who happened to be walking by

    1. David 132 Silver badge
      Happy

      Re: Look like the back of a bus

      Well, there are other distracting things that can be on the back of a bus, as reported a few years ago by this very, ahem, organ.

      1. John Brown (no body) Silver badge
        Windows

        Re: Look like the back of a bus

        Over 15 years is more than "a few years". Yer gerrin' old man, it's sneaking up on you!

    2. ravenviz Silver badge

      Re: Look like the back of a bus

      I recall RV hire companies posting vast prairie scenes on the backs of their motorhomes, I found it off putting as it was, but could make the whole vehicle appear completely invisible to a computer.

      1. zuckzuckgo Silver badge

        Re: Look like the back of a bus

        > could make the whole vehicle appear completely invisible to a computer.

        True if it relies solely on cameras. If lidar or radar is used graphics would have no effect. However, non-metallic vehicles or trailers could be a problem for radar, light absorbing paints could stymie lidar.

        Best to have a mix of sensors. I don't think I would trust a system that relies only on cameras, it is more reassuring to know the car can also "see" in ways that I can't.

        1. MachDiamond Silver badge

          Re: Look like the back of a bus

          "True if it relies solely on cameras. If lidar or radar is used graphics would have no effect."

          Elon Musk doesn't believe in RADAR or LiDAR and has insisted that the Tesla FSD operate entirely with cameras.

  3. Anonymous Coward
    Anonymous Coward

    Bend me shape me

    Bendy bus got bent.

  4. Anonymous Coward
    Anonymous Coward

    Extensible code?

    If Cruise decides to start operating in Geneva, the cars will have to learn to cope with bendy trolley busses with two bends, not just one (and also not run into trams,,,,)

  5. Zack Mollusc

    Artifical stupidity

    "Since the AV had previously seen the front section and recognized that the bus could bend, it predicted that the bus would move as connected sections with the rear section following the predicted path of the front section. This caused an error where the AV reacted based on the predicted actions of the front end of the bus (which it could no longer see), rather than the actual actions of the rear section of the bus. That is why the AV was slow to brake."

    What? Guessing the actions of the front half of a bus when you have lost sight of it is a bit dodgy in itself, but not being able to detect the position/range/closing speed of the back half of a bus to the extent that you collide at 10mph is absolutely ludicrous. What sort of sensors is it using? A ouiji board?

  6. veti Silver badge

    Relatable

    This is an ideal accident from Cruise's point of view.

    One, it's trivial. No one was hurt, or much scared, and damage was tiny.

    Two, it's an easy fix. They whacked out a patch within days, and "voluntarily recalled" before any relevant regulator could even get their A into G. (Of course the code change won't affect any other situations, what a strange suggestion. I'm sure it's been comprehensively regression tested.)

    Three, it's a sufficiently unusual scenario that they can spin this whole story about how it happened and how they've now altered their code so it doesn't happen again. Thus painting a picture of exactly the sort of gradual improvement and learning from experience that we'd all like to believe in.

    Four, and perhaps most importantly, it's relatable. Who hasn't had at least a near miss caused by a moment of inattention in traffic? Makes the car seem more human.

  7. EricB123 Bronze badge

    Average Schmo?

    I resent that! But I'm impressed that El Reg knows Yiddish.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like