The day the first automated car is used to crash through the barriers of Downing Street and has been instructed to do so by someone that's almost untraceable, all that will end.
And we just don't have the tech in place. In the cars, the roads, or the supporting infrastructure.
When you can set up an entirely automated tram company (hard to go wrong here, really), then a bus company (special lanes, priority, slow and stop often), then a taxi firm, then a car hire company, then you can start thinking about just general personal use.
Fact is most tube lines are still human-driven and the only one that isn't has human safeguards on board. Even the "newer" tube lines are also human-driven. And that's in an enclosed, safe, highly controlled, highly automated (dead man's switches etc.), highly-monitored track with almost no hazards in front of it and no lane-changes etc. to worry about.
Solve that and then you can think about cars and other things.