back to article Mercedes answers autonomous car moral dilemma: Yeah, we'll just run over pedestrians

It is a question that has grown in urgency since the prospect of truly autonomous cars became a close reality: what does a computer-driven car do when faced with a crash? With decisions likely to be made by algorithms in milliseconds, there will likely need to be a moral component pulled into systems: should a car protect its …

  1. Neil Alexander

    Not sure this approach of protecting the occupants of the vehicle is so unusual. The autonomous system can at least largely control the vehicle, whereas it has absolutely no influence or control over external players.

    If someone outside of the car does something reckless then I don't suppose it's really fair to expect the car to sacrifice its own occupants as a result.

    1. 45RPM Silver badge

      It depends on the situation surely? After all, the occupants of the car are effectively wearing a tough suit of armour. It might well be worth dinging the car a bit in order to protect the squishy pedestrian or cyclist - no matter how daftly they might be behaving.

      All the same, I'm glad that I'm not responsible for coding that software!

      1. Neil Alexander

        Sure does depend on the situation. After all, dinging the car is one thing. Writing off the people inside of it is another.

        Not all external players are pedestrians or cyclists, though. Some of them are in HGVs or trucks. Some of them are idiots in Range Rovers who think they're indestructible.

        1. Anonymous Coward
          Anonymous Coward

          Depends on the situation, until you're in the situation, when you have no control over the situation.

          All of you bean counters are forgetting that both parties, unlike astronauts, have not accepted the possibility of death for the best situational outcome.

          "Sorry Mr. Sandman, the situation called for your 5 year old daughter to die. At least I saved one life. But for her sake, I plan to attend all drunk driving classes in her name. Good day."

        2. JeffyPoooh
          Pint

          "... idiots in Range Rovers who think they're indestructible."

          Older Range / Land Rover vice a good modern car, no contest.

          https://www.youtube.com/watch?v=YbEa3C2vL7U

          Not even close. Range / Land Rover squashed.

          As you hinted, the opposite of what they'd think.

          I'm sure that the newer ones are less awful.

          1. Anonymous Coward
            Anonymous Coward

            Re: "... idiots in Range Rovers who think they're indestructible."

            > Land Rover squashed.

            That thing is not a Land Rover - yes I know the marketing people say it is - but it is not.

            https://www.youtube.com/watch?v=BcHlMBui-1w

            Admittedly it needed a tow and a little panel beating afterwards, but occupants were fine

            1. Dr. Mouse

              Re: "... idiots in Range Rovers who think they're indestructible."

              I had a friend who used to drive Land Rovers in Africa.

              They weren't indestructible, and there were many crashes involving wildlife out there. So people fitted bull bars to theirs, thinking it would make them survive the crashes better.

              My friend was one of the few who didn't. When he was involved in a crash with wildlife, he found that parts were plentiful to fix the damage... from all the others with major chassis damage due to the mounting of bull bars!

            2. Anonymous Coward
              Anonymous Coward

              Re: "... idiots in Range Rovers who think they're indestructible."

              "That thing is not a Land Rover - yes I know the marketing people say it is - but it is not."

              The manufacturer at that time was called Land Rover, so yes it is a land rover in the same way that a Ford Mondeo is a Ford.

              "Admittedly it needed a tow and a little panel beating afterwards, but occupants were fine"

              Thanks to the roll cage. Which you don't tend to get in the dealership model.

            3. Dr_N

              Re: "... idiots in Range Rovers who think they're indestructible."

              "That thing is not a Land Rover - yes I know the marketing people say it is - but it is not."

              Yawn. Hush now AC.

              @vid clip

              .. with a full roll cage fitted, sure.

              Standard SWB/LWB LandRovers squish very easily.

              1. Anonymous Coward
                Anonymous Coward

                Re: "... idiots in Range Rovers who think they're indestructible."

                >...with a full roll cage fitted, sure.

                Learnt to drive in one - but all ours had cages. Don't remember Renault people carriers being an option :)

                I've also rolled a couple with only the standard superstructure too - they certainly don't 'squish'. The SWB was roof rated to carry 150KG standard. You could quadruple that with the factory fitted expedition cage.

            4. Robin Bradshaw

              Re: "... idiots in Range Rovers who think they're indestructible."

              Your video doesnt show a landrover either, more a landrover shaped dune buggy, without all the extra rollcage in that the roof would have been level with the bottom of the windows, landrovers are useless if you roll them:

              https://www.youtube.com/watch?v=rizV-F57deM

              1. Anonymous Coward
                Anonymous Coward

                Re: "... idiots in Range Rovers who think they're indestructible."

                >landrovers are useless if you roll them:

                Caveat emptor - plenty of roll cage options. All Landrover (Defenders) sold in the US had a factory fitted roll cage.

          2. CrazyOldCatMan Silver badge
            Stop

            Re: "... idiots in Range Rovers who think they're indestructible."

            Not even close. Range / Land Rover squashed

            Or, as my wife put it in conversation about her Morris Minor: "My crumple-zone is the other car"..

            1. TRT Silver badge

              Re: "... idiots in Range Rovers who think they're indestructible."

              I recall this scenario being played out in popular fiction once...

      2. Dave 15

        Yes but

        I think the point the merc guy made is what happens next... sure the merc may choose to hit a tree in order to avoid killing the pedestrian, but does that immediately put the car coming the other way in a fix and cause another accident maybe spilling onto a pavement and killing a dozen school kids...

      3. Antonius_Prime

        That software was prewritten years ago…

        They just need to pay for a copy of Carmageddon on GOG…

        ;P

    2. bazza Silver badge

      Not sure this approach of protecting the occupants of the vehicle is so unusual.

      Er, except in some European countries the exact opposite is codified in law. For example, if a car and a bike collide in The Netherlands, by law it is automatically considered to be the car driver's fault no matter what the circumstances.

      1. BarryUK

        Only from the perspective of insurance liability.

      2. Stoneshop

        Insurance liability

        if a car and a bike collide in The Netherlands, by law it is automatically considered to be the car driver's fault no matter what the circumstances.

        That's with regard to insurance. The pedestrian/cyclist can still be held responsible for causing the accident, and fined accordingly.

      3. Anonymous Coward
        Anonymous Coward

        "Er, except in some European countries the exact opposite is codified in law. For example, if a car and a bike collide in The Netherlands, by law it is automatically considered to be the car driver's fault no matter what the circumstances."

        Hardly surprising coming from a country that thinks living below sea level is sensible and having whores standing in shop windows is perfectly ok.

        1. Dave 15

          The sea level thing is a bit odd, the other, well, I think it is better than a phone number in a kiosk, a description and finding something entirely different...

        2. d3vy

          "Hardly surprising coming from a country that thinks living below sea level is sensible and having whores standing in shop windows is perfectly ok"

          >> RE The sea level thing - working out for them so far...

          >> Whores in the window? If you dont like it dont look at them.

    3. Brenda McViking
      Go

      Lets play a game

      There is a research project going on at MIT for "morality" of self driving cars which anyone can have a go at - The MIT Moral Machine

      Very similar to "the trolley problem" mentioned elsewhere

      1. Charles 9

        Re: Lets play a game

        The Moral Machine IINM is BASED on the Trolley Problem.

      2. Anonymous Coward
        Anonymous Coward

        Re: Lets play a game

        moral machine on googleapi, I like that!

      3. Bob Rocket

        Re: Lets play a game

        Love that link but I was a bit disappointed it wasn't a real time live google car with real pedestrians. Anybody can make moral decisions by virtue signaling but it is only in a life/death split second moment can you find the real answers.

        My answer is that in the event of a life/death situation the occupants of the car must die, they knew the risks before they took the job, the car company doesn't care because they already have the money and the pedestrians are potential future customers.

        1. Robert Helpmann??
          Childcatcher

          Re: Lets play a game

          My answer is that in the event of a life/death situation the occupants of the car must die, they knew the risks before they took the job, the car company doesn't care because they already have the money and the pedestrians are potential future customers.

          A. This is a flawed analysis from the perspective of the car company: a bird in hand is worth two in the bush.

          B. Also from that of the customers: there is no job, there is only a consumer product they should have no expectation but that it is engineered to keep them safe. If it gets out that it is not, then point A is out the window on both counts - no-one will buy the product.

      4. Bogle

        Re: Lets play a game

        Did you know there's a solution to The Trolley Problem? Stop me if you've seen this one.

        A two-year-old's solution to the trolley problem: https://youtu.be/-N_RZJUAQY4

    4. big_D Silver badge

      Mercedes can't even control their vehicle.

      The ADAC (German equivalent of AA) did a test in August of the top cars with automated braking systems. The Mercedes came bottom of the list, it failed to stop completely for pedestrians in daylight, failed to notice a cyclist and failed completely at night.

      The winners were Kia, Subaru and VW Passat (the Kia stopping completely for a pedestrian in daylight, nearly stopping for the cyclist and attempting to stop at night, the Subaru nearly stopped in daylight, nearly stopped for the cyclist and stopped completely at night, the BMW failed to stop in all situations, but had managed to drastically reduce speed).

      The tests were done on a test track with dummies moving in front of the vehicle at 30mph.

      They also have an inflated cushion, which looks like the back end of a VW Touran and is towed behind a car, to test the automated braking at highway speeds. Again, the results were disappointing, with many of the cars running into the back of the obstacle.

      Unfortunately only the 2012 test is online and I have thrown out the magazine with the 2016 test report.

      1. Kiwi
        Joke

        the BMW failed to stop in all situations, but had managed to drastically reduce speed).

        Would that be because the horrible piece of shitezen had rather poor performance and simply could not run at any thing approaching a normal definition of "speed"? (or more likely, had largely broken down in the few minutes between starting the engine and reaching the stopping point)

        (Close friend of mine was a BMW lover.. Then he brought one.. Now he hates them with a passion even I can barely comprehend!)

        1. big_D Silver badge

          I owned a 1987 E28 M535i, lovely car, although the tyres were like Bakelite! I also had a 1998 528i.

          Both cars were great to drive (and reliable), although I find the newer models uninteresting. They have lost their flare and everything is electronic these days.

  2. 45RPM Silver badge

    My best mate bought himself a brand new Mercedes SLK. It rusted. This announcement just demonstrates that, not only can Mercedes not be bothered to build their cars properly, they can't be bothered to put the effort into considering complex ethical problems fully and designing their software accordingly.

    Buy a Mercedes? Not if you paid me.

    For what it's worth though, I understand that their commercials vehicles are rather good.

    1. Pompous Git Silver badge

      not only can Mercedes not be bothered to build their cars properly

      I believe my brother-in-law's Mercedes was 25 years or so old when he traded it in.

    2. Kurt Meyer

      "Engineered like no other car in the world"

      I am very happy with my current Mercedes, and have been happy with each of the Mercedes I have owned prior to this one. Given the choice, I wouldn't own anything else.

      Unless, of course, a prancing horse was featured on the badge.

      I believe it was Mr. Clarkson who said; "If you want to see what technology will be in your car in ten years time, take a look at today's Mercedes S-class."

      I don't have to wait ten years, I just walk out to the garage.

      1. Pompous Git Silver badge

        Re: "Engineered like no other car in the world"

        I believe it was Mr. Clarkson who said; "If you want to see what technology will be in your car in ten years time, take a look at today's Mercedes S-class."

        Perhaps a little longer than 10 years. A friend proudly showed off his second-hand 2-door Mercedes by driving me to a philosophy lecture in Launceston (Tasmania) 10 or 11 years ago. The seat belts were automatic. That is, they were motorised and automatically came forward for the driver and passenger to buckle themselves up. Never seen that in any other vehicle.

        1. Voland's right hand Silver badge

          Re: "Engineered like no other car in the world"

          That is, they were motorised and automatically came forward for the driver and passenger to buckle themselves up. Never seen that in any other vehicle.

          Nissan, 1989, forgot the model, was sold in USA at the time. Mercedes is now using the patent after it expired.

        2. Chz

          Re: "Engineered like no other car in the world"

          Automated belts were extremely common in the States 25 years ago. At some point the Feds put in a law that the automakers had to introduce either airbags or automatic belts. Well, automatic belts were cheaper so... You can imagine the rest.

          As it happens, people *hated* them and vastly preferred vehicles with airbags. A lot of it to do with the uniquely American notion that they didn't need safety belts if they had airbags. (Really. I'm not making that up.) Not only that, but as side airbags were introduced, the automatic belts interfered with them and were quietly dropped.

          1. Darryl

            Re: "Engineered like no other car in the world"

            Late 80's/early 90's - a few Toyotas, Mercurys, Mazdas, Cadillacs (I think) had automatic seatbelts. They were annoying and caused a couple injuries and stuck people when they malfunctioned.

        3. Vic

          Re: "Engineered like no other car in the world"

          The seat belts were automatic. That is, they were motorised and automatically came forward for the driver and passenger to buckle themselves up. Never seen that in any other vehicle.

          I first saw that on a hire car I had in the US. I think it was a Camry - something like that.

          That was ~20 years ago...

          Vic.

        4. MattPi

          Re: "Engineered like no other car in the world"

          "Perhaps a little longer than 10 years. A friend proudly showed off his second-hand 2-door Mercedes by driving me to a philosophy lecture in Launceston (Tasmania) 10 or 11 years ago. The seat belts were automatic. That is, they were motorised and automatically came forward for the driver and passenger to buckle themselves up. Never seen that in any other vehicle."

          I'm sure if we're talking about the same thing here, but automatic belts were a fad in the early 90s in the US.

          https://www.youtube.com/watch?v=SptbcNjYVtw

      2. tiggity Silver badge

        Re: "Engineered like no other car in the world"

        I quite like my cars with relatively limited technology.

        I have been a passenger with someone in a top end Mercedes with mobile phone integrated into car sound / mic system via Bluetooth when they received a call.

        Yes the system worked really well ...

        But the quality of driving whilst having the conversation made me (seriously) fear for my life (this was on a motorway, not IMHO the place to take calls).

        My vote is (I'm assuming better than people self drive cars are a long way off yet) for a car where the driver has the bare minimum to distract them from paying attention to whats happening / might happen on on the road / surrounding area.

        Though quite happy to have (generally non distracting) tech that can be safety useful e.g. alerting driver if they seem to be gaining ground to the car in front rather too fast & may be risk of collision, black ice warnings based on temperature & humidity readings etc.

        1. Kurt Meyer

          Re: "Engineered like no other car in the world"

          @ tiggity

          "But the quality of driving whilst having the conversation made me (seriously) fear for my life (this was on a motorway, not IMHO the place to take calls)."

          Forgive my saying so, but your description of this incident does not sound a technology problem, rather that the usual suspect(s) are at it again.

          I will not start ranting about my fellow motorists. I will not start ranting about my fellow motorists. I will not start ranting about my fellow motorists.

          Whew!

          Better now.

      3. Anonymous Coward
        Anonymous Coward

        Re: "Engineered like no other car in the world"

        At least with autonomous control it will fix the problem that we have in Australia where no Mercedes drive knows how to drive.

    3. Anonymous Coward
      Anonymous Coward

      "My best mate bought himself a brand new Mercedes SLK."

      Is your best mate a hair dresser?

      Perhaps his hair products damaged the paint.

    4. messele

      Think again, their HGVs rust even more than their passenger vehicles and their Sprinter vans rust even more than those. Surprised to be able to see the ground through the scuttle panel while changing the wiper blades? Yeah I was too.

      Japanese and British all the way.

    5. Voland's right hand Silver badge

      My best mate bought himself a brand new Mercedes SLK

      See this: https://www.youtube.com/watch?v=y3YUfo16_rs and continuation.

      There is a world of difference between an old E class tank and the crap on the road today. What was made then, was a Panzer designed to brute force its way around the world 10 times and mileages in the 500K were not unheard of.

      What is being built today is better than the really crappy stuff of the past decade, but it is still nowhere near the practically indestructible Panzer models of the 70-es and 80-es.

      1. Kurt Meyer

        @ Voland's right hand

        "There is a world of difference between an old E class tank and the crap on the road today."

        Absolutely right. They were built like tanks. You still see them on the road today.

        The problem for Mercedes was that once they'd sold you one of those, it would be a very long time until you came back for another new one.

        Planned obsolesence wins another round.

    6. Just Enough

      Predicatable decision

      Mercedes are taking a very obvious line here and the decision is perfectly predictable. I'm sure every car manufacturer trial autonomous cars will reach the same decision.

      No-one, but no-one, is going to buy a car if they know that it may consciously prefer to kill them over someone else. They're going to buy the car that's advertised with "we'll protect you and your family above all else"

    7. d3vy

      "My best mate bought himself a brand new Mercedes SLK. It rusted"

      Metal object rusts! The shocking news at 10.

      Seriously though you didnt mention a timeline between these tow events? Weeks, Months, Years? Does he happen to live near the coast? Did he take good care of the paint work?

      Generally Merc are good, Ive had mine from new for two years now and theres nothing wrong with it at all despite my best efforts to make it go sideways...

      They build so many now there are bound to be a few horror stories.

  3. PacketPusher
    Alert

    Car companies are not in this alone.

    Generally, a company's first priority is its share holders, followed by its customers. Everyone else is at the end of the list. It is up to the rest of us to set rules through our government that more balanced. It is understandable the Mercedes will take this position, but it should not last.

    1. Stuart 22

      Re: Car companies are not in this alone.

      "Generally, a company's first priority is its CEO's bonus package ..."

      FTFY

    2. I am the liquor

      Re: Car companies are not in this alone.

      So presumably if wear my Mercedes stock certificate pinned to my chest, the car will swerve into the tree and avoid hitting me?

      If the car's driver is also a Mercedes shareholder, I guess it'll do a quick lookup on the share registry to see who has the most voting rights.

    3. The Indomitable Gall

      Re: Car companies are not in this alone.

      It's not even about responsibility to the customers though, is it?

      Technology is the limiting factor here -- self-driving car control systems are optimised for road-awareness, and rightly so. If you need the computer to analyse the off-road environment, you're going to need to process a huge amount of data, and the hardware and algorithms will become scarily complex.

      But before you let a self-driving car leave the road in an attempt to avoid a collision, you need that complexity; otherwise you risk driving into an occupied bus shelter, through a fence into a garden full of playing children, or over a cliff.

      Like it or not, the safest place for a self-driving car is on the road.

  4. Anonymous Coward
    Anonymous Coward

    Makes sense..

    Yes, that makes perfect sense. After all, that's what screen wipers are for, no?

    Bad jokes aside, were there not some directives available or underway that improve pedestrian survivability by changing the car front design?

    1. Anonymous Coward
      Anonymous Coward

      Re: Makes sense..

      Well, I've seen small fences pivoted to the front that raise up after a luckless pedestrian flips up onto the car front. The idea is to catch the body when the horrified driver slams on the brakes (too late) so the pedestrian doesn't go forward and under the front wheels. If they're still alive, so much the better.

      I can't recall any other safety gadgets in the pipeline offhand...? Perhaps 'reverse airbags?' Oh, or maybe a threateningly powerful loudspeaker directed towards the front that sounds like a lorry of live pigs locking up the brakes at 100kph?

      1. The humble print monkey

        Re: Makes sense..

        An upvote for the fabulous furry freaks reference.

        1. Anonymous Coward
          Anonymous Coward

          Re: Makes sense..

          Yes, the fabulous "Pedestrian Dispersal System" episode.

          Still as relevant today as when it was written!

    2. Anonymous Coward
      Anonymous Coward

      Re: Makes sense..

      Bad jokes aside, were there not some directives available or underway that improve pedestrian survivability by changing the car front design?

      Already in place. Bumper heights, bonnet angles, crumple zones.

  5. Voland's right hand Silver badge

    Do not be so mean to merc drivers

    The only time I have seen from up close a driver sacrifice his car and risk his hide to save others was a Merc driver. Two teenage girls were more engaged in chatting to each other then observing the road pulling out of a side street in a 1980-es Ford Fiesta. The guy in the E class was driving within the speed limit (30mph) when they pulled out less than 5 meters in front of him. If he stayed on the road they would have been dead. 2 tons of German steel hitting a 1990-es rusty Pinto relative.

    He threw the car into a tree on the sidewalk. The E class was total writeoff and, surprise, surprise, the girls continued down the road without even noticing what happened - they were _THAT_ engrossed in their conversation (someone literally ran them off the road to stop half a mile down).

    Thankfully, the guy was not even hurt. He was just very very very pissed off.

    So, in Western Europe, you may indeed encounter a human driving a Merc.

    Now Eastern Europe... that is a different story.

    1. JeffyPoooh
      Pint

      Re: Do not be so mean to merc drivers

      "2 tons of German steel"

      The E Class, at least the W211 I'm familiar with, has aluminium bonnet, wings, and boot lid *. So the front of the car is nice and soft, for more comfortable collisions.

      The passenger cell and doors are very tough steel. I've seen drill bits stopped dead by the inner layer of steel.

      * Look at me, writing in British. LOL.

      1. Voland's right hand Silver badge
        Devil

        Re: Do not be so mean to merc drivers

        The E Class, at least the W211 I'm familiar with,

        That still would not have helped. A 1980-es Fiasco has no side protection whatsoever and mass is mass. It knows no mercy even if the v squared is not that much.

        1. JeffyPoooh
          Pint

          Re: Do not be so mean to merc drivers

          "That still would not have helped."

          Another entertaining feature on Mercedes is called Brake Assist System (BAS).

          When the driver excitedly moves their foot from the accelerator to the brake pedal quickly enough (due to utter fear), the car concludes that full braking might be useful, and triggers the BAS.

          The car pulls the pedal down, following just below the driver's foot, applying vast braking force. Seat belts tighten automatically, other things in the cabin are automatically moving.

          The driver's foot is reporting to the driver's brain that the brake pedal has failed (due to the weirdly soft brake pedal feel). At the same instant, the driver's face is reporting to the driver's brain that it's experiencing painful de-acceleration forces and may detach from the skull. (Yes Clarkson, it does "actually hurt".)

          The braking de-acceleration force causes the driver's head to bend forward, putting the speedometer directly in line of sight. Thus allowing the driver to clearly see the speedometer unwinding like a rock falling.

          Once upon a time, 'a friend' going to work was exiting the highway at [censored] kmh. About halfway along the off ramp, it was noticed that 'a friend' was still going [censored], and all the above BAS experience unfolded. It was a religious experience, according to 'a friend'.

          BAS is a really neat safety system.

    2. Gene Cash Silver badge

      Re: Do not be so mean to merc drivers

      Heck, I would have run over those girls without a second thought. Evolution in action, I say. Teach your kids to drive, or physics will apply painful lessons.

      I guess I need to apply to Mercedes for a coding job.

      1. Dagg Silver badge

        Re: Do not be so mean to merc drivers

        @Gene Cash

        You missed the sarcasm tags..

    3. Known Hero

      Re: Do not be so mean to merc drivers

      Many thanks @voland.

      Perfect example, went up onto the bank and almost took out the post box to avoid somebody who just wasn't looking when pulling out. had I not gone off the road I'm sure somebody with a wobbly neck would be ripping off the system :(

      https://www.youtube.com/watch?v=oXYFofIKWc0

      not a massively exciting video, but luckily the only one worth recording since I had my dash cam.

      1. FredBloggs61

        Re: Do not be so mean to merc drivers

        Perfect example, went up onto the bank and almost took out the post box to avoid somebody who just wasn't looking when pulling out. had I not gone off the road I'm sure somebody with a wobbly neck would be ripping off the system :(

        https://www.youtube.com/watch?v=oXYFofIKWc0

        not a massively exciting video, but luckily the only one worth recording since I had my dash cam.

        Looks like a pretty danged small road to be doing nearly 40 on?

        1. Known Hero

          Re: Do not be so mean to merc drivers

          RE 40 comment. Depends if people decide to pull out into the entire road without looking if traffic is coming. Are you not meant to to be having infantile arguments on youtube over who was at fault in more serious videos?

          We were both happy on our separate ways and there was no damage, I only posted it to re-enforce what Voland had stated. I swerved to avoid hitting another vehicle at potential risk to purely my vehicle alone.

      2. MJI Silver badge

        Re: Do not be so mean to merc drivers

        last week driving home.

        Got dazzled by the sun for 2 seconds, hit shade just in time to see a car 6 foot in front of me who had just pulled out.

        If I had been going faster I would have written their car off.

  6. Anonymous Coward
    Anonymous Coward

    I can't quite follow how the tone of the article derived from the quotes. Oh well, never mind.

    As we all know, the reference to philosophers is the 'Trolley Problem' ( https://xkcd.com/1455/ ), which like any good thought experiment deliberately simplifies a scenario in order to show how complex shit can get, even from a simple premise. Unlike a road, the Trolley Problem is on rails, so the 'steering' choice is binary.

    - What if the tree is rotten and wouldn't inure the car's occupant?

    - What if you have the choice of running over three people on one track, or two on another? What if you knew that the three people were nasty pieces of work who only caused good people misery?

    - What if one person on the tracks was Adolf Hitler? (Bear in mind that Allied plans to assassinate him were shelved once it was decided that he was bonkers and that allowing him to be replaced by someone competent would only extend the war and thus its death toll) (Can we expect a VW Beetle to exhibit Oedipedal tendencies?)

    - Is walking along railway lines at night a survival trait? ( This knocks us onto the whole issue of eugenics... current popular opinion is that eugenics is bad m'kay, but the Darwin Awards are good - possibly because reading the Darwin Awards doesn't affect the outcome of the special creatures described within. More seriously, what is or isn't a survival trait can't be determined locally - paddling too far from your native island might be risky and frowned upon, but if you then settle upon another island when your former community get wiped out by a volcano, it's your genes that'll continue).

    My pragmatic approach to driverless cars is this: If they are demonstrably safer than us meatbags, lets go for it. As a bonus, I can be 'drunk, tired or high on drugs' - as a recent poll of Australians has identified as being key selling points.

    1. This post has been deleted by its author

      1. Dave 126 Silver badge

        Re: Ahem...

        >That subject (Adolf) has already been addressed, here,

        And in the Stephen Fry novel Making History

        >just in case you thought Germans couldn't possibly have a sense of humour regarding Der Führer. :)

        I didn't think that - not after hearing a BBC radio adaptation of a German novel, and watching the film of the same: https://en.wikipedia.org/wiki/Look_Who%27s_Back_(film) :)

        1. Kurt Meyer

          Re: Ahem...

          @ Dave 126

          I thought "Look Who's Back" was a funny film.

        2. JeffyPoooh
          Pint

          Re: Ahem...

          'Look Who's Back'

          A very lovely film. Dark humour. Subtle. Disturbing. Reminds me of Trump, even with the risk of invoking Godwin's Law. I watched it on Netflix. I may buy a disc later, I liked it so much.

          Highly recommended, if you like very subtle dark satire.

      2. GrumpenKraut
        Thumb Up

        Re: Ahem...

        I can tell you that this video is hugely popular with employees of Daimler-Benz.

      3. JeffyPoooh
        Pint

        Re: Ahem...

        That's a very nice, very funny video. Thank you.

    2. Anonymous Coward
      Anonymous Coward

      Tram Problem

      It's a 'Tram problem', actually. Invented by Foot - the British philosopher:

      http://philpapers.org/archive/FOOTPO-2.pdf

    3. Anonymous Coward
      Anonymous Coward

      Trolley Problem and the Law

      With the way prosecutions go here, by doing nothing and letting 5 people die you would face no charges. By diverting the train and killing 1 person you would be prosecuted for man slaughter as a minimum and probably murder!

  7. Lars Silver badge
    Thumb Down

    "von Hugo continues to equate the value of a car as greater than that of a human". Sorry but I cannot find that statement anywhere. Lets not get too excited.

  8. Jon 37

    Hard decision but Mercedes are probably right

    Three points:

    #1: Pedestrian recognition isn't perfect

    It's not really "car decides to hit pedestrian rather than crash into tree". It's actually "car decides to hit object-detected-as-pedestrian rather than crash into tree-that-is-both-detected-and-in-built-in-maps". The object recognition (for pedestrians) cannot possibly be 100% accurate, so this is probably the right thing to do, just in case this is part of the .01% (or whatever) when the object recognition is wrong, and it's just a carrier bag flying across the road.

    #2: Take personal responsibility

    As a driver I am responsible for driving safely, e.g. slowing down around blind bends so I have time to stop if a pedestrian is there. I expect that autonomous cars will do this, and will in fact do it better than human drivers because they have a better understanding of the car's stopping distance and visibility, allowing them to slow down "just enough" rather than the too much or too little that a human will do, and because autonomous cars don't get bored or tired or distracted.

    However, if a pedestrian decides to run across the road in front of me without looking, that's the pedestrian's fault, so I think the pedestrian should accept responsibility for their actions and the consequences of their actions. If, as a result of a pedestrian running into the road in front of me, someone is going to be seriously injured or die, it should be the pedestrian and not me. Obviously the car should stop or swerve if possible, or slow down as much as possible to minimise the impact.

    #3: It's a hard decision.

    As a human driver, if you end up in a situation where either an adult pedestrian is going to be seriously injured or kill, or you're going to crash and seriously injure or kill yourself plus your girlfriend/boyfriend/wife/husband/child/parents/siblings who are passengers in the car, would you choose to protect the strangers or your family? That would be a very hard decision to make, but you may have to make it, and you may decide that the lives of yourself or your family are more valuable to you than the lives of strangers. Obviously you would be very sad and upset about it, whichever way you decided. Can you honestly say you would sacrifice your wife or husband or loved one or whole family to save a stranger? Some people would, but many people wouldn't.

    Now, if you're buying an autonomous car then you'd want the car to make the same decision, which means that suddenly you have to think through this in advance, whereas before you don't have to think about it until/unless you're ever in that horrible position. It's also much harder to decide this when it has to be premeditated and not just a snap decision, and it's frowned on to say that you value your own life more than another person's life even if it's actually true (and instinctive - survival of the fittest etc).

    ------

    Edit to add: Note that this post is mostly talking about "someone is definitely going to die or lose a limb" situations. If the choice was "kill a pedestrian or have a small dent on the car", that's clearly different, you should clearly dent the car. If it's "kill a pedestrian or write off the car with only minor injuries", that's also different, but it's very hard to predict how bad the injuries will be once the crash is more than the most minor dents to the car - the car doesn't know if the person in the back is a very fit 25-year old or a very frail 90-year-old, or a person with brittle bone disease.

    1. cyberdemon Silver badge
      Devil

      Re: Hard decision but Mercedes are probably right

      Completely agree. But I think the most important angle is the legal angle. As soon as you consider that, then you realise that this whole moral madness is just a can of worms with no bottom. It is better to just try to stop as best you can for ANY obstacle, and not even try to think about this stuff.

      1. As you say, the car's sensors cannot possibly ever be perfect. It could quite easily see a paper bag, a cat, a dog, running across the road and decide that it is a human infant. It may then decide to take extreme evasive action to avoid the infant, and end up "deliberately" killing its driver. A human could make exactly the same mistake of course (and probably does, in all of the RTAs of the world). The difference with the human, however, is that you can't then download the black-box data out of his squished brain, and replay his fatal mistake in front of a court of law. The car manufacturers will be *terrified* of this second-guessing by the courts, and of course the lawyers will be salivating just thinking about it.

      2. Yes, pedestrians have the same responsibility to avoid danger as anyone else (young children are not directly responsible - their parents are responsible FOR them). The highway code is there for a reason. If a driver is not driving dangerously or carelessly, but still kills a child, then he should, in theory, be found NOT GUILTY, by the court - because the child or their parent should have been paying attention to the inherent danger of the road. Unfortunately in many cases (because the courts are not perfect), he will be found guilty of something and go to jail. If the driver was in fact a robot operated by some company, then one corporate manslaughter case could sink even the biggest of companies. (even if it was incorrectly judged, by the jury of falliable squishy things, which statistically some of the cases will be) I think this is one of the biggest obstacles to the widespread adoption of fully autonomous cars - that the courts have an unacceptably high error rate and that even a "perfect" autonomous car will some day be found guilty.

      3. It IS a hard decision, but I would be more afraid of the car that is clever enough to be able to try and make it, than one which is "dumb" and doesn't even go there. The reason is this: If a car is able to evaluate in real-time, the value of all life around it, and try to prioritise the teenage kid walking into the road over the elderly driver, a "criminal", or any of the other ridiculous scenarios you will find over at Moral Machine, for example - then it must have extremely sophisticated social profiling built in. (or perhaps even outsourced to "the cloud" - even worse). This opens up all kinds of evil possibilities. People complained about a "racist hand dryer" whose sensor failed to properly detect black skin. You now have the possibility (or rather, certainty) for a car to deliberately kill a human just because they were profiled to be "less morally valuable" than some other human.

      There will be inevitable imperfections in the profiling algorithm and its training data. It *will* be more efficient at profiling the kinds of humans in its training set than the kinds of humans outside of the training set. This is now automatic discrimination against any kind of human who was not included in the training set. Remember the Google Photos app that identified black people as "gorillas"? Well unfortunately they did not have enough black people in the training set, but apparently they did have gorillas. For Google, that was a serious facepalm. If that was an autonomous car though, then once you factor in the lawyers, it's racially motivated murder on the part of Google.

      Worse still, the system could be deliberately modified by some evil human. If a large number of a particular manufacturers' autonomous cars had the software capability to profile people and decide on their moral value, and someone maliciously issued an over-the-air software update to all of these cars simultaneously, then that malicious person could even attempt genocide. Very scary indeed.

      ----

      Basically, If I was buying an autonomous car, I would want it to protect ME. (humans are ultimately selfish, no matter what you say) and I would certainly not want it to be second-guessing my life over what ultimately could be just a paper bag. And humans are the customer, so this makes perfect commercial sense.

      1. cyberdemon Silver badge
        Devil

        Re: Hard decision but Mercedes are probably right

        I hasten to add, if an autonomous car has to run itself into a tree to avoid hitting someone, despite all of its super-duper sensors to detect it in advance, and super-duper autonomous braking to stop on a peanut, then it was almost certainly driving too fast in the first place. Either that or the obstacle was being tragically stupid, which isn't the car's problem at all.

        1. JeffyPoooh

          Re: Hard decision but Mercedes are probably right

          "...it was almost certainly driving too fast in the first place."

          Up to this point, the default naive assumption has been that self-driving cars will be essentially perfect.

          Many commentards have based their entire worldview within this topic on this amazing assumption.

          This is apparently based on a near-future, amazing and hand-wavily undefined assumption of a sudden, massive step function improvement in system design, requirements definition, and coding practices.

          Therefore, if we allow their assumption, it follows that all speed limits are superfluous.

          1. Charles 9

            Re: Hard decision but Mercedes are probably right

            "...it was almost certainly driving too fast in the first place."

            There's no safe speed when an obstacle suddenly emerges from a blind spot less than ten feet in front of you (like the COMMON scenario of a little child, concealed by parked cars, suddenly running out into the street).

            1. Richard 12 Silver badge

              Re: Hard decision but Mercedes are probably right

              Modern cars can stop in ten feet from surprisngly high speeds.

              However, human reaction time is such that most people would have barely started braking.

              The reaction time of an autonomous vehicle will be quicker than the majority of meatbags - otherwise it's not useful. It'll also brake or swerve harder, as it knows the car handling better than most humans do.

              So the "will collide zone" in front of a moving autonomous vehicle would become notably smaller, and the resulting collision would happen at a lower speed.

              It's still there though, no matter what you do.

              Obviously there are some talented people who can often take a vehicle right to the limit, but they are rare and even they can't do that all the time. The AI doesn't get tired.

              1. Neil Barnes Silver badge

                Re: Hard decision but Mercedes are probably right

                There's a multi-level thing going on here, I think...

                First level, you're driving watching what's in front of you and adjusting your speed and direction to avoid bumping into anything. Most of what's there is either stationary objects, objects doing much the same direction and speed (or the reverse vector).

                Second level, you're considering what might be making changes to this situation: is that object that just stopped likely to open a door and emit a fleshy thing? Will the stationary object pull out in front of you? Will the objects coming down the slip road twenty miles an hour accelerate to your speed or wait for you to pass, or pull out in front of you moving more slowly than you?

                Third level, you're considering potential hazards that should not but might cause changes to your surroundings. If you're passing a school at the end of the school day, you're going to be cautious; at 2 a.m. perhaps less so. But at 2 a.m. on the saturday morning past the club...

                And so on. There's always one more layer of introspection regarding external conditions, many of which - but not all - are known locations at known times and can be mapped, and many which are completely random: the busload of nuns with a failed braking system vs. a tree struck by lightning and unsure which direction to fall in vs. a paraglider landing in the road in front of you.

                Where do you stop? This is what meatbags do supremely well - even the ones who can't read the Sun without moving their lips seem to career around the streets for years without causing major mayhem. I'm not convinced, though, that in a serious incident either the said mouth-breather or the professional ethicist who is also an expert driver is thinking anything more than stercus stercus stercus moriturus sum

              2. Anonymous Coward
                Anonymous Coward

                Re: Hard decision but Mercedes are probably right

                "Modern cars can stop in ten feet from surprisngly high speeds."

                No, they can't. Published figures for stopping a Mini from 70MPH to 0 is 172 feet, and that is about par for the course

                1. Richard 12 Silver badge

                  Re: Hard decision but Mercedes are probably right

                  Don't be ridiculous. 70MPH to 0 in 10 feet would be a mean of about 15G, and that'd definitely kill you if eyeballs-out. And it might kill you eyeballs-in as well.

                  That Mini achieves a mean of 0.95G according to your figures, so 18MPH to 0 is plausible in 10 feet.

                  15MPH to 0 in 10ft is rather common.

                  Which is surprisingly fast, wouldn't you say?

          2. Anonymous Coward
            Anonymous Coward

            Re: Hard decision but Mercedes are probably right

            Up to this point, the default naive assumption has been that self-driving cars will be essentially perfect.

            They will still be better than your average merc driver.

      2. Pompous Git Silver badge
        IT Angle

        Re: Hard decision but Mercedes are probably right

        People complained about a "racist hand dryer" whose sensor failed to properly detect black skin.

        That sounds suspiciously like some racist hard disk drives still in my possession. They have jumpers to set them as either Master, or Slave.

        1. Kurt Meyer

          Re: Hard decision but Mercedes are probably right

          Racist? Are you certain they're not just kinky?

      3. Charles 9

        Re: Hard decision but Mercedes are probably right

        "Basically, If I was buying an autonomous car, I would want it to protect ME. (humans are ultimately selfish, no matter what you say) and I would certainly not want it to be second-guessing my life over what ultimately could be just a paper bag. And humans are the customer, so this makes perfect commercial sense."

        But then what happens when the person the car is approaching is your wife or kid suddenly emerging from a blind approach on a noisy, rainy day for some emergency? Now it's personal AND a situation where you could morally be inclined to go "them not I".

      4. Anonymous Blowhard

        Re: Hard decision but Mercedes are probably right

        "The car manufacturers will be *terrified* of this second-guessing by the courts, and of course the lawyers will be salivating just thinking about it."

        This is an interesting thought.

        On the one hand the car will have a perfect record of the event, so if the incident is entirely the other person's fault (e.g. stepping in front of a vehicle moving at legal speed) then the manufacturer should be protected by the law, even the best human driver won't be able to avoid that, and many times won't be able to prove that he was in the right.

        On the other hand lawyers and juries, for different reasons, are often eager to punish greedy corporations.

        In practice the rate of accidents for autonomous vehicles should be lower, and in almost all cases it should be obvious whether the software did better or worse than a good human driver, so the manufacturers should be able to protect themselves from frivolous and excessive litigation. It may require an additional government agency, similar to the AAIB's role for aircraft accidents (NTSB for USA) to ensure that incidents involving autonomous vehicles are properly investigated and appropriate changes forced on the manufacturers.

        There could be a subsidiary issues though, based on software updates to the vehicle. If the manufacturer issues a fix for a problem that the owner refuses to accept (assuming they get a choice) then the owner may be the one liable in any accident that the software update could have avoided; this could mean that insurance for autonomous vehicles requires the owner to accept all software updates from the manufacturer in order to keep the liability with them.

        1. TechnoSceptic

          Re: Hard decision but Mercedes are probably right

          "On the one hand the car will have a perfect record of the event" It will have a record of what the manufacturer's software is programmed to record. How close to reality that is, may be tricky to verify. Oh, hi VW.

    2. Kurt Meyer
      Thumb Up

      Re: Hard decision but Mercedes are probably right

      @ Jon 37

      Jon, that was a hell of a post.

      Thanks

    3. bazza Silver badge

      Re: Hard decision but Mercedes are probably right

      Hmm, have you read anything like the Highway Code, driving fitness requirements, etc?

      #1: Pedestrian recognition isn't perfect

      As a driver you are expected to be fit enough (eyesight, sober) and driving at a speed appropriate to the conditions to ensure that you are guaranteed to be able to recognise pedestrians, cyclists, any road hazards, etc.

      Why should a self driving car be somehow exempt from that just because the technology is hard?

      If it is to be allowed on the roads and be fully autonomous and is less perfect than humans are required to be, then the manufacturer should take the responsibility. After all, if a human driver under performs and is blamed for an accident / injuries / death, they are held responsible. The machine and its maker cannot be given a let-off.

      And if the technology is systematically less able than a human driver is required to be, then it shouldn't be allowed at all. We might as well start saying it's OK to be drunk behind the wheel.

      #2: Take personal responsibility

      Er, except that as a driver you are required to anticipate road hazards. Not every pedestrian is responsible for their actions. Ever seen a young kid run out into the road? Ever seen an elderly person fall off a slippery pavement? Ever seen someone pushed into the road by a mugger? Ever seen a pram roll away from a distracted mother? No? Well lucky for you. These things happen, and it's not their fault.

      Your attitude is wrong, you should get it fixed. You're saying that the young kid, pensioner, crime victim or baby deserve to be run over.

      #3: It's a hard decision.

      No it's not - the pedestrian has no protection. The car occupant is surrounded by crush zones and air bags. I say the car should take a chance and trust its own structural integrity.

      Not that it's ever going to come to that. If a self driving car is coded to drive in a manner that would allow such a situation to develop (i.e. too fast) then the car maker is as guilty of causing death by reckless driving as a human driver would be.

      Now if the law is written appropriately (the car manufacture is liable for its behaviour and any accidents it causes), then the manufacturer would be held responsible for the accident / injuries / death caused by the car's inappropriate speed. So they're going to have to code their cars to drive like grannies in town.

      If the law says otherwise, then I won't be getting in a self driving car. Trust my personal liberty to the behaviour of some software written by some guys who get no come-back if it goes wrong? No thanks.

      1. Doctor Syntax Silver badge

        Re: Hard decision but Mercedes are probably right

        "Er, except that as a driver you are required to anticipate road hazards."

        The question of reality has to creep in here. You can anticipate hazards but to what extent should you allow for them.

        For instance I've been about to overtake a couple of cyclists when one of them, for no obvious reason and with no prior indication, turned square across the road right in front of me, fortunately with just sufficient clearance for me to do an emergency stop and blow my horn (I hope his pants were festering by the time he got home). Had he done this a second later he'd have been a gonner.

        Now any cyclist has to be regarded as a hazard who can act in a totally idiotic manner (try driving in any area which has been on a TdF route before downvoting) but does this mean that no cyclist should ever be overtaken in case they try stunts like this? One has to balance the probability of an action against the utility of continuing in any sort of motion.

        1. Andrew Taylor 1

          Re: Hard decision but Mercedes are probably right

          And I've had the same with white vans / private hire, doesn't mean I regard them all as idiots and yes I have driven regularly in the areas you mention, I've also seen drivers try to overtake cyclists where there's no room to pass in the same area causing cyclists to swerve, maybe it was a cyclist taking the lane for their own safety trying to slow you down, who knows as we only have your version.

        2. bazza Silver badge

          Re: Hard decision but Mercedes are probably right

          @Doctor Syntax,

          For instance I've been about to overtake a couple of cyclists when one of them, for no obvious reason and with no prior indication, turned square across the road right in front of me, fortunately with just sufficient clearance for me to do an emergency stop and blow my horn (I hope his pants were festering by the time he got home). Had he done this a second later he'd have been a gonner.

          Well yes, but that's a different scenario; your cyclist was in control of their actions. The imprudent kid, slipping pensioner, rolling pram, pushed crime victim is not.

          Things vary by country. In the Netherlands all car/bike collisions are, by law itself, deemed to be the car's fault no matter what.

          There are no doubt a lot of badly behaved cyclists; I regularly drive in Cambridge... The organised "cycle events" are pretty disgraceful sometimes too. Badly behaved inattentive cyclist? That is indeed what the horn is for, "warning other road users of your presence".

          When I ride a bike I'm careful to follow the highway code, just as when I'm driving too. There's plenty of knobheads in cars, lorries and buses too.

          The Dutch, in passing a one sided law, seem to have managed to get more people to behave better more of the time. And putting in a load of bike tracks helps too.

          1. Doctor Syntax Silver badge

            Re: Hard decision but Mercedes are probably right

            "Well yes, but that's a different scenario; your cyclist was in control of their actions. The imprudent kid, slipping pensioner, rolling pram, pushed crime victim is not."

            So far I've managed to not encounter them for half a century.

            The bigger problem by far is those who are deliberately or negligently ignoring road safety who are the real problem. Added to the cyclists are pedestrians who walk along country lanes taking no account of visibility (I encounter this in lanes where, when walking, I cross from side to side in order to stay on the outside of bends to see and be seen). And pedestrians who stroll without looking along the middle of village streets (I live in a tourist area) and through car parks as if they're in a pedestrianised area.

            As a driver I accept my duty of care for others' safety but is it unreasonable to ask that those on foot or on two wheels also take some responsibility for their own survival?

    4. Stuart 22

      Re: Hard decision but Mercedes are probably right

      "However, if a pedestrian decides to run across the road in front of me without looking, that's the pedestrian's fault, so I think the pedestrian should accept responsibility for their actions and the consequences of their actions. If, as a result of a pedestrian running into the road in front of me, someone is going to be seriously injured or die, it should be the pedestrian and not me."

      And if that was a five year old kid you would make the same decision? And if it was your own kid?

      Wow, you are scary.

      1. fruitoftheloon
        FAIL

        @Stuart 22: Re: Hard decision but Mercedes are probably right

        Stuart,

        methinks our fellow commentard implied that pedestrian would be old enough to take responsibility (i.e. for a 'normal' child 7/8/9 years old perhaps??), my lad has just turned eight and his localised 'awareness' could be better. But we live in a quiet little village in the middle of Devon.

        Re 'the five year old kid', err shouldn't a grown-up have been at the very least holding their hand or somesuch???

        Cheers,

        jay

  9. Wiltshire

    Is "Vorsprung durch technik" an appropriate response?

  10. Dan 55 Silver badge
    Trollface

    Nothing a Merc driver wouldn't do anyway...

    He's just saying it out loud.

  11. The Nazz

    But on the upside ....

    At least the indicators will be used where appropriate/required.

    I've though about this dilemma, more in the context of constantly stepping in front of one of these autonomous vehicles such that they slam their brakes on. Good fun to be had.

    Best be careful which make/model i choose then.

  12. Anonymous Coward
    Anonymous Coward

    Surely this is not that surprising given that all of the people in the Mercedes are passengers and its the system thats the driver and which has to decide who to protect. The problems start when the system starts prioritising its own safety and the elimination of witnesses...

  13. Eddy Ito

    Would it pass the Kobayashi Maru?

    Can their algorithm distinguish a couple walking with baby carriers on their backs from other unpredictable walking "squishy" critters we have in my home town known as moose.

    1. Dave 126 Silver badge

      Re: Would it pass the Kobayashi Maru?

      My understanding is that hitting moose can be very bad for the occupants of the car.

      As regards 'this system'... we're not talking about a specific system.

      I was having a nice chat in the pub the other week with a military systems engineer though... some interesting stuff about specifically narrow-band IR sensors (for detecting mood in humans, so likely able to differentiate twixt moose and person)

      1. Eddy Ito
        Thumb Up

        Re: Would it pass the Kobayashi Maru?

        That narrow band sensor would be great to have on a night out. If single you can tell whether you'd be wasting your time and if you're married whether you should go back out and buy something1 before going inside.

        As for the moose it should handy during the rut, as my brother and his college roommate discovered, a moose with an attitude can be very bad for the car when the moose does the hitting.

        1 For the still single, the answer will always be yes regardless. Just make sure it's something soft as insurance in case it comes flying toward the back of your skull.

      2. Anonymous Coward
        Anonymous Coward

        Re: Would it pass the Kobayashi Maru?

        Moose.

        Most of their mass is (just barely) above the roof line of a typical car. So the key is to be going fast enough that you pass underneath Bullwinkle's body before it has time to fall.

        Therefore,when driving on a dark and stormy night in areas plagued by moose, it's important to maintain a minimum speed of about 220 kmh.

        That's why Mercs are safer. Many of them can run 220 all night. And even get decent fuel mileage while doing so.

        (Some say... that New Brunswick is the size of a small coconut on certain nights.)

        1. Stevie

          Re: Would it pass the Kobayashi Maru?

          Most of their mass is (just barely) above the roof line of a typical car.

          So the car would have to figure out that what it was about to hit was in fact a large animal of considerable mass when the evidence presented to the forward sensor array would be telling the electronics that the issue was four spindly fenceposts in its way.

          I'm looking forward to reading "my car cut the legs off a moose, which then squashed my roof down to the seatbacks when the body fell on it" stories.

      3. cortland

        Re: Would it pass the Kobayashi Maru?

        "My understanding is that hitting moose can be very bad for the occupants of the car."

        Heard on a Amateur Radio net one night; a Washington State net member in a backwoods SUV missed his turn in the conversation and came back in as the net was closing to explain why; a moose had decided charging an oncoming SUV would be fun.

        I don't know if he got to keep the meat, but the vehicle was a write-off.

        1. Stevie

          Re: Would it pass the Kobayashi Maru?

          Canadian I know claims a bull moose charged his van and tore off a fender as he drove at highway speed because the moose was in rut and the van passed between the bull and his harem of cows.

          The moose was unhurt in the exchange.

  14. Jim-234

    Simple answer to overly complex handwringing

    It seems to be the craze for some "researcher" or other to go into these "moralistic" binges about what should automated systems do etc. When it's really simple.

    Car owner / driver is the first priority.

    If you can maneuver to avoid a collision without any problems to anyone or anything, then do so.

    If you can't avoid a collision, then apply maximum braking and hope for the best.

    In a collision, assuming non human objects, try to maximize driver survival if possible (So if you are about to drive under the side of a truck try hitting the wheels instead so as not to lop the head off the driver).

    Not really all that hard to figure out and about how thing work out anyways.

    If people want it different, they can take the wheel.

    1. Doctor Syntax Silver badge

      Re: Simple answer to overly complex handwringing

      "If you can't avoid a collision, then apply maximum braking and hope for the best."

      The best course of action. Minimise the risks all round. Trying to deflect the risk involves steering and trying to steering and brake hard at the same time is likely to result in failure to do either.

  15. ratfox

    How is it happening now?

    If a car driver decides to save his hide and run over pedestrians, do they get prosecuted? I doubt it. Egoism is usually a valid defense. Cowardice is only a crime when facing the enemy.

    Until humans solve the problem themselves, I don't see how we can expect cars to do it for us.

    1. Charles 9

      Re: How is it happening now?

      Legally, that's why you have trials, because things aren't always black and white and judges and juries have to assess the situation on a case-by-case basis. Common example: you killed someone in an ambush, but he'd been pursuing and shooting at you previously (meaning the aggravating circumstance of lying in wait is countered by the mitigating circumstance of immediate threat to life). It's up to the courts to determine if the loss of life was the result of deliberate action (murder), negligence (manslaughter), or some other circumstance, and if so, how much is the accused responsible for the circumstance (did the accused act willingly or was coerced, threatened, or otherwise forced into a no-win situation like the Trolley Problem). Were there other people involved? One common variation of the Trolley Problem can be called Guerillas In The Village (instead of you in a trolley, it's you encountering it's a bunch of militant guerillas about to wipe out a village and deciding for variety to let the rest of the village off if you execute one of them).

      That's also why you have inquiries when tragedies hit (the inquest into the late Phillip Hughes is in the news right now), to see if things were simply fate (the inquest seems to be heading in that direction) and where things could be improved in future (perhaps there will be research into better neck protection if it doesn't hamper batsmen too much).

  16. Spaller
    Big Brother

    Insured crash game theory

    Smart pedestrians carry fobs that transmit their insurance status. The impending car does a cloudy cost benefit analysis. The claims may be settled before impact.

  17. scrubber
    Terminator

    If I spend $60k on a personal Uber I'd rather it didn't try to kill me when it mistakes a cat on the road for a baby.

    But this is a very temporary problem, roads will not be accessible to people soon and automated will be the only, and safest, way to travel.

    1. Charles 9

      "But this is a very temporary problem, roads will not be accessible to people soon and automated will be the only, and safest, way to travel."

      Don't be so sure. The Trolley problem demonstrates a significant moral angle to driving. And because of this, competent drivers tend to develop intuitive road awareness that comes practically from instinct so can't be taught to a computer...because we don't know how we come about the skills.

      The Trollable Self-Driving Car

  18. Anonymous Coward
    Anonymous Coward

    Make My Day

    I think pedestrians need a similar system to keep them safe as they walk into traffic whilst engrossed on their mobes. Perhaps exploding back packs to ensure they don't go out alone?

    1. Stevie

      Re: Exploding Backpacks

      Nah, already covered by the Innovative Technology lab of Samsung, who are still dealing with a few niggling false positives.

  19. Anonymous Coward
    Anonymous Coward

    RFID

    It's high time we were all implanted with RFIDs that stored our worth to society to help improve the decision making process.

    Make a donation to the right candidate and maybe your worth is such that the car swerves to avoid you.

    If you're a barista with too many bad Yelp reviews, maybe there is no need to change course or hit the brakes.

    And if you repeatedly and unrepentantly end sentences with a preposition, maybe you find cars tend to swerve at you....

  20. Anonymous Coward
    Devil

    Im not too upset with Mercedes on this one...

    But in keeping with their average driver's ethics, I expect BMW will announce that they will opt to hit the pedestrian then the autodriver will back over the body a few times to make sure :)

    1. Anonymous Coward
      Anonymous Coward

      Re: Im not too upset with Mercedes on this one...

      "the autodriver will back over the body a few times to make sure :)"

      Ah, The Mexican Bus Driver maneuver.

      Back in 1981 there was a scandal.

      Turns out that some Mexico City bus drivers that had hit pedestrians had been finishing the job because it was cheaper than a lawsuit.

      Google Search: "mexico city" buses backed over pedestrians

      1. Dr Scrum Master

        Re: Im not too upset with Mercedes on this one...

        The Chinese are just a bunch of copy-cats!

  21. Jonski

    Identification issues

    Considering Tesla vehicles currently can't reliably differentiate between a F Off Big Truck and The Sky, I think it will be quite a while before we encounter the trolley problem to decide between Adolph Hitler pushing a baby in a pram, or Theresa May saving a puppy from Boris Johnson...

    But I also think that just like jaywalking only became blameworthy when motorised vehicles became common, we will see that again the preservation of motorists will prevail over all other road users, to their detriment. Let's hope that as technology continues to improve, we also see the development of things like external airbags to protect those outside the vehicle like they protect those within.

  22. Anonymous Coward
    Anonymous Coward

    Re: Identification issues

    " like external airbags to protect those outside the vehicle"

    That's asking a lot of an airbag - even if they inflated before impact, not after.

    Physics is a bitch.

    1. Mark 85
      Joke

      Re: Identification issues

      Oh... that kind of airbag. Here I thought he was talking about the driver.

    2. wayne 8

      Re: Identification issues

      Bulgarian airbags!

    3. Stephen Keane

      Re: Identification issues

      The airbags are located by the windscreen wipers and are triggered when the front of the car impacts with the pedestrian (or whatever). The idea is that they are inflated before the pedestrian's head would have impacted the windscreen or windscreen wipers area as neither is an effective crumple zone for a human head.

  23. Long John Brass
    Terminator

    Won't be a problem

    on the bright side, once it becomes widely known that stepping out in front of an autonomous car will result in death or serious injury; pedestrians will become a lot more alert when crossing roads.

    Although given that this is already the case & many people still wander blindly into traffic, I won't be holding my breath

    As long as the car doesn't rev up and actively try to run me down I think it's all good

    Autonomous cars should always do the simplest most obvious thing. Complex moral computations are IMHO a bad idea; you want these systems to always make the choice of least surprise; that way everyone has a reasonable chance of being able to predict what the damned thing is going to do next.

    I don't want my car to decide that my life is less valuable than someone else, even less I don't want it to decide that as a pedestrian that I'm expendable; Sure hit the brakes and stop as fast a practicable but I don't want to try and dodge a 2 tone Mercedes on the sidewalk

  24. Pete 2 Silver badge

    The wrong people making the wrong decision

    It doesn't really matter what the techies at Mercedes, Ford, GM, Telsa or any other car maker decides. The rights and wrongs of the matter will be decided country-by-country in case law, as each injury-accident or death is prosecuted.

    This will, or course, be a shambolic maze of conflicting principles, examples, precedents and exceptions. Not only will each car, from each manufacturer, have to have its autonomous software updated to account for they ways each country's laws evolve with each new case won or lost, but that same software will have to be aware when the car moves from one jurisdiction into another, and then start to drive according to that place's laws (just like a person would have to).

    I can also foresee mass disabling of vehicles (by the million) when updates fail. You might think that it's annoying when W10 decides, of its own volition, to make your computer unusable for hours while it downloads some updates. Imagine when you try to get into your car to go to work and the dashboard informs you that it won't be drivable for another 2 hours due to a legal upgrade. (Or worse: when it pulls in to the side of the road to do the same thing, while you're traveling).

    1. wayne 8

      Re: The wrong people making the wrong decision

      Obviously, there needs to be global traffic and tort laws.

      And one approved global system provider.

      We need a global super state to make these decisions.

      Many, many meetings in luxury places with formal dinners with the leaders of automotive corps, leaders of technology corps, lawyers, yes, lots and lots of lawyers.

  25. Slx

    My one question on this is legal.

    If the car maker is taking over the driving, it's very likely that they'll also be taking on some or all of the liability for accidents caused by the car when it is driving.

    I could see this stuff getting very legally complex and expensive.

  26. unwarranted triumphalism

    Maybe if pedestrians kept to the pavement where they belong we wouldn't have any of these 'moral difficulties'.

    1. Neil Barnes Silver badge

      Why did the chicken cross the road?

      To avoid these 'moral difficulties'...

    2. BebopWeBop
      Facepalm

      And maybe if cars stuck to the roads and pedestrians didn't have to cross them (for reasons that seem unfathomable) we would not have any of these moral difficulties either. Choices, choices, choices.......

      1. unwarranted triumphalism

        And maybe if you hadn't just accused me of driving on the pavement you would not have made yourself look like a complete idiot.

    3. Doctor Syntax Silver badge

      "Maybe if pedestrians kept to the pavement where they belong we wouldn't have any of these 'moral difficulties'."

      Round here it's not the pedestrians who are the problem, it's the cyclists. It's a puzzle. Presumably for part of the time the cyclists are pedestrians. They may even be drivers themselves at times. So why, when they get on two wheels, do so many of them suddenly abdicate responsibility for their own safety?

      1. GloomyTrousers

        It's the cyclists

        @Doctor Syntax: so by your highly scientific survey, pedestrians and drivers are flawless, and only some cyclists are not? I'll hazard a guess that those individuals of which you speak are dangerous/inconsiderate/oblivious more than the average, regardless of their current mode of transport, and you just notice them more when they're cycling due to your own inherent biases.

        You should probably check actual statistics, which would show you that in (say) car vs. cyclist accidents the car driver is found at fault in the majority of cases.

        1. Kurt Meyer

          Re: It's the cyclists

          @ GloomyTrousers

          "@Doctor Syntax: so by your highly scientific survey, pedestrians and drivers are flawless, and only some cyclists are not?"

          I've read, and re-read, the Doctor's post. I can't seem to find those words.

          That would be a big, fat, fucking, fail on your part.

          Why do you (and others) insist on putting your own words into someone else's mouth?

        2. Doctor Syntax Silver badge

          Re: It's the cyclists

          "your own inherent biases"

          My inherent bias is the Highway Code that was drilled into me as a young cyclist which I now see widely ignored.

          And not only widely ignored but promoted. There is some local organisation which organises a local cycle challenge and the misdeeds of the organisers would constitute quite a long list. For instance the kids were despatched down a steep, narrow, twisty lane. If there were observers penalising them for dangerous riding that would be one thing. There weren't, so kids would negotiate the bends at speed, in the middle of the road.

  27. Anonymous Coward
    Anonymous Coward

    Save the customer, so it will spend more in repair or a new car!

    Who cares of the serf on the road? If you're Junker like Herr von Hugo, you're used to pass over people... and that's why business should not be allowed to set health, safety and ethical standards.

  28. BarryUK

    Sounds fair enough - I'd be a bit upset if my car was programmed to crash me into a tree because some idiot walked out into the road without looking.

    1. Stuart 22

      I would suggest you would automatically and WITHOUT THINKING swerve to avoid. Hence one's own personal consequences are not taken into account. Whatever the moral position the Mercedes decision bends the balance of dying towards the third party (be it guilty/not guilty pedestrian, cyclist or other car driver) and away from the person holding the gun^h^h^h who is in control of an enormous amount of kinetic energy. The energy that would kill.

      That is a change society should think about and decide at a more disinterested level than a car manufacturer or even the finest coder.

  29. wayne 8

    1955 LeMans Mercedes

    What's the problem with deaths of non-participants?

    The Daimler Benz lawyers will argue that the pedestrians, whatever, should not have been where ever they were when they were struck.

  30. wolfetone Silver badge

    Autonomous Car Will Drive In To A Pedestrian?

    I can do that myself.

  31. Anonymous Coward
    Anonymous Coward

    Well... that's one way skynet wipes us out.

    So its plan is to wipe out anyone not in a car... crafty skynet. When the cars decide to off-road and start using the side walk we know it is all over.

  32. Chris G

    Decisions ,decisions.

    Most of the talk about humans driving and making moral/ ethical/ informed/ intelligent decisions in the event of a sudden appearance of a pedestrian or anything else in their path is largely crap.

    In my 50 odd years on the road I have observed that most people usually try to swerve to avoid most sudden obstructions without thinking, those that don't are usually drunk, stoned or cretins.

    So probably whatever Mercedes come up with, will be at least as good as most people and better than the drunks, stoners and cretins.

  33. Gonce

    'Pranksters' paradise

    Given the recent spate of 'pranksters' that think it funny to scare people with clown masks I wouldn't be surprised if, in a world of self-driving vehicles, that these same sociopaths think it funny to deliberately throw themselves in front of vehicles in order to scare the occupants or make the vechicle crash as it avoids the 'prankster' in the way.

    If the car puts others before it's own occupants and this is widely known and understood by the masses then we can expect people will cause cars to swerve into trees for shits and giggles.

    Perhaps Mercedes have it right?

    1. Charles 9

      Re: 'Pranksters' paradise

      Why bother putting their skins at risk from a car that decides to run them over? Just use life-size human-shaped balloons. I don't think car sensors are smart enough to distinguish between these and humans and will react accordingly without risk to the prankster.

    2. Stevie

      Re: 'Pranksters' paradise (Clowns)

      I'm amazed one of these USA Scary (Ass)Clown "Performance Artists" hasn't been shot in the stomach yet. The one talking to the BBC didn't even seem to register the peril he was placing himself in in order to frighten women at night into running terrified from him and his friends. The sooner the better, I reckon. One good stupid deserves another, as they say.

      1. Anonymous Coward
        Anonymous Coward

        Re: 'Pranksters' paradise (Clowns)

        Ever thought they take that into consideration and wear kevlar vests just in case? At least a gut shot is one thing the vests are designed to protect.

        1. Stevie

          Re: 'Pranksters' paradise (Clowns)

          You seem to know a lot about the proper precautions women-scaring (ass)clowns need to take to avoid come-uppance, Mr strangely appropriately Anonymous Coward.

          I bow to your superior knowledge, and look forward to seeing you in the news when one of your victims subjects figures that out and goes for the big red nose shot one night.

  34. Doctor Syntax Silver badge

    From TFA: "in many cases the lives being saved are the ones who put the fast metal box in the precarious situation in the first place."

    If the car is autonomous and is in a precarious situation it wouldn't be the occupants who put it there, it would be the designers - and they won't be there.

  35. Anonymous Coward
    Anonymous Coward

    Presumed knowledge.

    The question is predicated on the car knowing far more than it possibly could. Swerving off the road requires knowledge of everything off the road which the car (and indeed a human) could not possibly have.

    We could reformulate the question: You're driving a car and turn a corner suddenly in front of you there are two small children, which one do you kill? The one on the left? The knife wielding maniac! Are you mad? The one on the right? That one had a terminal disease and was going to die tomorrow anyway.

    1. dom_f

      Re: Presumed knowledge.

      That simply means you are driving too fast - you should be able to stop in the distance you can see to be clear (or half that if not wide enough for 2 vehicles)....

  36. Mutton Jeff

    Not good PR

    "Car kills occupant(s)"

    It's one thing if a driver swerves, but the car doing it automatically....

  37. lvm

    would you by a car which might consider killing you?

    I wouldn't. An autonomous vehicle may not injure its occupants or, through inaction, allow its occupants to come to harm - this is the First Law.

    1. Charles 9

      Re: would you by a car which might consider killing you?

      But the same can be said of the bystanders and that's covered by the First Law, too. So that's why the Trolley Problem: the First Law WILL be violated no matter what you do, so what will the automated car do: save the passengers and become a risk to bystanders (raising government scrutiny, plus it may be your spouse, kid, or relative in danger) or save the bystanders and become a risk to the passengers (posing a sales problem to the dealers)? And no, any third option will just endanger BOTH groups.

  38. strum

    But, but, but....

    What if the other car is also a Merc?

  39. Anonymous Coward
    Anonymous Coward

    This is a weaselly excuse...

    for the first unecessary pedestrian death caused by a Mercedes, and they won't get away with it either.

    The safety design of modern cars means that you should be able to walk away from a 30mph crash if you're inside the vehicle. You may even walk away from much faster crashes. The car *won't* be making the choice between the pedestrian and the occupants, it will be making the choice between the pedestrian and some damage to itself, plus possible minor injuries for the occupants.

    Ultimately, if the car *isn't* programmed to damage itself rather than a human pedestrian, then I for one am going to give up cycling with my kids to school, sell the house and buy a traction engine for the 1.5 mile school run and then the extra 2 miles to work. I wonder how early you have to get up to stoke the fire?

    1. Charles 9

      Re: This is a weaselly excuse...

      "The safety design of modern cars means that you should be able to walk away from a 30mph crash if you're inside the vehicle."

      The thing about the tree, though, is that it could easily fall ONTO the car, crushing the cab. It would be even worse if the object in question is a concrete pole or some other very heavy top-heavy object...or instead of a tree, the only way to avoid tragedy (say an careening bus full of children) is to drive over an edge.

      1. Anonymous Coward
        Anonymous Coward

        Re: This is a weaselly excuse...

        Designing an algorythm that will "decide" to kill a pedestrian rather than injuring the occupants of the vehicle is effectively manslaughter on the part of the software developers, and don't be surprised if someone goes to jail in a few years time if they're responsible for writing software that does this.

  40. Cuddles

    Autonomy

    "especially since in many cases the lives being saved are the ones who put the fast metal box in the precarious situation in the first place."

    No they're not. The entire point of asking what an autonomous car should do in a given situation is that the occupants have no more input into its decisions than anyone else.

  41. Anonymous Coward
    Anonymous Coward

    As a merc owner I fully agree with this, where the collision is with another car the software should determine the value when new of the opposing vehicle and the more expensive of the two should be spared. Unless its a BMW in which case they probably deserve it. ;)

  42. Nunyabiznes
    Joke

    User selection

    Just put a dial in the car that allows the driver to select a range of autonomous actions based on:

    1. Meh, fuck'em. (I ain't swerving for no jerk-off cyclist)

    to

    5. Martyr yourself (save the bunny by killing the tree)

    1. Charles 9

      Re: User selection

      Some people may not hold sentimental attachment to rabbits, so instead use "your spouse" or "your child".

  43. CbD1234567890

    Carmageddon

    That's what matters. Remember the car may be warped into a puddle, but as long as the occupants come out hale and hearty...

    Now we have the moral compass out of the way, which peds r worth more points?

  44. Stevie

    Bah!

    Better hope your Luxmobile kills the people it hits, and there are no traffic cams around to contradict your version of events.

    Otherwise, you may have saved the car just so you could give it to the person it ran down in your name.

    Along with your house.

    And your retirement funds.

    And maybe a kidney.

  45. Bill Michaelson

    Aviation parallel

    In flight training I was taught to constantly consider one's options in the event of complete engine failure. This inevitably leads one to consider hypothetical situations in order to be primed for action. One of these is the beach below. Land on the sand, unless it is populated. If there are people, ditch in the water, at considerably greater risk to one's own health. Similar situation with golf fairways vs. rougher terrain, etc.

    Maybe I've been planning it wrong?

  46. Anonymous Coward
    Anonymous Coward

    I think that what the autonomous car is programmed to do in these situations should be kept secret. That way it will be no different than travelling by bus, plane, train or taxi is now. We don't know what the pilot, driver or engineer will do in a crash even if we know what they are supposed to do.

    Who do you want to trust; the pilot, the cab or bus driver, the engineer, or the computer programmer?

  47. Anonymous Coward
    Anonymous Coward

    That's old technology...

    ... the Chinese already have Tanks that run over pedestrians no problem.

  48. Anonymous Coward
    Anonymous Coward

    Just Invading private space ....

    The car is merely invading the pedestrian's private space to liberate it from its miserable life (which it must be because otherwise the pedestrian would drive a Mercedes).

    The pedestrian doesn't like that invasion and to protect himself throws the arms in the air. The car interprets this as a terrorist act and the pedestrian insurgent is quickly run over and killed - legally.

    Killing of the pedestrian is merely collateral damage.

    The car then declares "mission accomplished", goes in reverse - before leaving other pedestrian to clean up the mess on the sidewalk.

    See it all makes sense when you look at it this way.

  49. Frumious Bandersnatch

    simple solution

    Have like a riot cannon mounted on the front of the car. It can hit a person with enough force to fling them into the air but presumably not kill them. This can give the car a few more moments in which to compute another course of action that can safely avoid hitting the obstacle (or simply hit it at a less lethal speed)

    A giant boxing glove mounted on a kind of scissors can also be used, as it's easier to reset/reload. Or a cannon loaded with quick-setting riot foam that can first reduce the relative velocity of impact, and second maybe protect them from a lethal knock by immobilising and/or cushioning the blow.

  50. adriantan

    Do you seriously believe that anyone will buy a car that said 'In the event of an accident we will kill you'? That will be the end of self driving car

  51. raving angry loony

    Absolutely correct.

    Did the stupid idiots who got in the way pay for the car in any way? Of course not! Those who can't pay shouldn't expect to be protected by what they haven't paid for. Why would they think they were safe even on the pavement (sidewalk)? Obviously someone driving a Mercedes is much more important than some peon walking around. Besides, Mercedes wouldn't sell nearly as many cars if they admitted that "by the way, sometimes we're not going to prioritize your safety". Remember people, PROFIT is the only means of determining what is correct.

  52. JLV

    Makes sense, if both party's lives are in equal danger. You don't want your car to put you in danger.

    However, under many speed conditions, head on collision into a wall, or even a tree, will only nuke the car and leave the occupants unscathed. Much lower speed conditions will result in dead pedestrians. Then I'd vote for prioritizing pedestrian survival if the driver is at low risk.

    Real "10 dead pedestrians or 1 dead driver" cases? Textbook ethics decision cases? Probably pretty unlikely compared to generic autopilot fails with worse overalll human life outcomes. Or the daily meatgrinder of human-triggered casualties. Ivory tower thinking.

    Not discounting ethics but these seem to be more what-if stuff than real world issues. Once the basic autopilot tech is reliable enough in a more general sense.

    I find the economics & ethics of what is a human life worth when it is a std manufacturer defect vs when it is an autopilot fail, if the autopilot results in less overall deaths much more interesting. Making mistakes is tremendously costly for car makers ( ex: Prius) . Will we hold autopilots to those higher standards, compared to human drivers, if their imperfections still kill less than human drivers whose liability is capped by car insurance companies? Will we instead say, it's Ok, Mercedes, that's another $200K because thats the max liability a human would have incurred, even though a particular recurring fail might have been corporate incompetence? Who will be insured by whom?

    Airplane makers have particular international agreements that caps quite stingily. But their overall safety record is good, because of the root cause fault finding efforts deployed on plane crashes.

  53. carolinahomes

    An unseen solution

    What percentage of pedestrians are carrying gps enabled cell phones? If we bump that up to 100% by the time fully autonomous vehicles are on the road, there will be no pedestrian crashes as the vehicle will be able to "see" everyone and slow down well in advance.

    Problem solved.

  54. John Savard

    One Little Detail

    Of course, though, before people will be allowed to go on the roads with cars that are driving themselves, governments will have to amend the laws. News items like this aren't going to help to hurry that process along.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon