Avionics is not a good comparison.
A prior comment stated that autonomous vehicle software should meet the same standards as avionics. This opinion is wrong on at least two separate levels.
First, in the US, the FAA has two certification paths for avionics hardware and software. 1. Prove it is accurate and reliable (typically via formal methods), then test enough to validate that proof. 2. Test the hell out of it, at a level 5x to 10x that done for the more formal path.
Small companies are pretty much forced to use the second path more than the first. Where I worked, we relied on the second path and aspired to the first. We had awesome lab and flight test regimens that the FAA frequently referred to as "best practices" for the second path.
Second, the risk of death due to an avionics failure (per incident) is massively higher than it is in cars, especially given the ever-increasing level of passenger safety measures present in modern vehicles. The fact that aviation death counts are so low is due more to the relatively tiny number of vehicles involved compared to cars (on the order of ~100K cars to each plane).
Autopilots are fundamentally simpler than autonomous driving: Fully functional autopilots have existed for well over half a century (the L-1011 was the first regular commercial aircraft to do an entire flight autonomously, including takeoff and landing). The primary reason for this achievement is the large distances between planes. In-air collisions simply don't happen outside of air shows.
The massively greater complexity of the driving environment (separate from the vehicle itself) forces the use of statistical methods (machine learning), rather than relying solely on formal, provable rules. If it isn't clear already, this means that autonomous driving systems will be forced to use the second path to certification: Exhaustive testing.
Most of that testing must occur in the real world, not in a simulator, because we simply don't yet know how to construct a good enough simulator. The simulator will always miss things that exist in the real world. One goal of ALL real-world self-driving tests MUST be to gather data for use by future simulators! Just because simulators are hard is no excuse to avoid building them. We just can't rely on them alone.
That said, all such on-road tests must be done with a highly trained technician behind the wheel. It is VERY tough to remain vigilant while monitoring an autonomous system. Been there, done that, got the T-shirt, hated every minute. In my case it was operating a military nuclear reactor. Boring as hell. Terribly unforgiving of mistakes. Yet it is done every minute of the day with extreme safely.
I'd focus on the test drivers more than the vehicles or their technology. Get that right, and we'll earn the trust needed to improve the technology under real world conditions.