Surely there are a lot of situations that it won't handle properly.
If faced with a queue of cars, how will it know if they are waiting for traffic lights to change round the bend or if it a line of parked cars at the side of the road?
Temporary restrictions. Some times it is difficult enough for a human to follow these signs with partially hidden signs for diversions. An "Access to Tesco only" in the middle of your route might cause problems if you are driving from one city to another and stopping at your local Tesco on the way home. Some road work signs are even hand scrawled with spray paint.
Can it tell the difference between a policeman waving a car round an accident and a thug waving your car into a dead end?
Can it make the right judgement on a partially flooded road?
And as for idiot that pull out in front of you on the motorway just as you are about to pass them. As a driver of many years I often have a sense about another driver that is going to do something stupid. Not so easy to build that intuition into a car.
I assume there may be circumstances where it will drop to human control (eg parking in a muddy field for a festival) but since these occasions will be few and far between, the human may have forgotten how to drive the car.