The middle ground is the most dangerous.
If people aren't actively engaged in driving the car, there is no way they can take over if the car gets in trouble. Either you are driving or are not (there is no "try"). The time it takes to recognize the car is not keeping up, analyzing the situation and taking correct action is longer than most problems will allow before a crash. There are plenty of videos online of beta testers (tesla owners) where the car drives itself on the wrong side of the road or doesn't seem to notice a car in the way when making a mid-block turn (not at a controlled intersection). The cars also have a hard time figuring out intersections where only one direction of traffic has a stop sign. They'll edge out into cross traffic and stop when it would have made more sense to "gun it". Human drivers can also see a car briefly through obstructions and understand that it will pass through the intersection momentarily even when it becomes hidden. That's a really tough problem for automated vehicles. If they don't see it, it doesn't exist <insert bugblatter beast reference here>.
Another problem with car automation is the level of accuracy. Aircraft are often miles apart and have all sorts of wiggle room. A car off by half a meter may be up on the pavement or jutting out into a busy intersection. Or, buried into the rear of an emergency vehicle which Tesla's seem to have a weird sort of affinity for.
I expect that for vehicle automation to work properly, it will take a combination of the car and the roadway. First steps could be city centers and special lanes on a motorway. There needs to be some way to handle breakdowns that doesn't back up traffic for hours. To mark roads and install signage compatible with automated cars will be time consuming and costly. There will need to be a mechanism that routes cars away from road works, outages, hazards, with a define path so all of a sudden all of the traffic on a high street isn't going down small residential lane or blocking up a road in front of a school or hospital.
Anybody buying a car with the hardware fitted is paying for technology that may never get used. Elon keeps saying that all that is needed is the software, but I suspect that they will find some hardware deficiency that will prevent a later version of software from working and the owner will have to take the car in for an expensive refit. Like the new MCU's losing over the air radio, there will likely be some changes that owners will not like. I see all of the gadgets as one more thing to go wrong that borks the whole car. If you don't opt for FSD and one of the components goes "bing" and lets out it's magic smoke, the whole car could grind to a halt. In the very least, it will have to be fixed to clear the code so the car will pass its next inspection. A friend has that sort of thing happen. The interface to an uninstalled option failed or reported an error and they wouldn't pass it until the fault was corrected. Nice tidy 1,000 down the crapper to fix something that really didn't exist in the first place. Another work mate had an old BMW with an OEM fitted cell phone. The service for the mobile was discontinued years ago. The car looked and ran great, but the phone was nothing more than an ornament.
My current car is 14 years old with over 200k miles. A gently used low mileage replacement engine is much less expensive than even a comparable used car. There's no SatNav without support, no media device that doesn't function and no need for it to have any software updates. Besides the mileage, it's in great condition. All of that means it has good resale value. I do plan on dropping a newer engine in it and keeping it for many more years even if I add an EV to the fleet. That new EV (non-Tesla) will likely have a shorter run and far less resale value down the road. All of the modern technology will be unsupportable at some time after the end of the warranty. The displays probably won't be made in 7-8 years so finding one to replace a defective unit will be expensive. The Teslas, Ford MachE and the VW ID4 seem to share the same design folly by basing the operation of the car on one device that is interacted with by a the display. Ford has an even bigger issue with the knob embedded in the display. That will be a fun bodge some years down the line. In aerospace, we call this a single point of failure and work hard to not have them. One thing stops working and it compromises the whole. My old truck had one HVAC fan setting that didn't work. A simple fix, but I didn't care that much to invest the time. With a new car, one thing wrong could mean everything doesn't work or several things stop working at once. The fan stays on high and can't be turned off and the heat is set to max at the same time. One step forward, two steps back.