Re: Probably similar to the experiences of some Tesla drivers.
There's a large class of sudden unexpected problems that can't be detected until they happen, whether you're in automatic or manual mode. One of the classic examples is landing gear fails to deploy or lock. Though I do agree that this halfsies functionality of self-driving cars and self-flying planes causes a second or two longer reaction time at a critical moment, and all the ringing a human to take over in an emergency is never going to work long-term.