Re: FFS
'it will actually get better and better and learn'
Yes it will, but only for as long as humans are the ones doing the learning and expressing their knowledge and experience in the code. Even then, unless there is continuous transfer of knowledge and experience from the outgoing 'old guard' maintainers to the incoming 'new guys' us humans will forget *why* that bit of code does what it does in the wider system.
Given the propensity for people to just have a go despite not understanding what they are about to do, IMO there has to be an assurance that the overall system and code-base are continuously, and thoroughly, understood despite the inevitable staff/dev turnover.
AI is NOT the answer. AI is not, and never will be, capable of dealing with the 'edge cases' as they arise - you've only got to watch a few Mentour videos (Other providers are available, including CAA/FAA/NTSB 'wash ups') to realise that issues generally get solved by humans, not machines.
If something has gone wrong, that's where I want a properly qualified pilot with some experience in charge. Also, two real pilots instead of one mean there are two heads involved, and there is the chance to talk something through with a peer, or at least someone different, first. The only thing AI has to converse with before taking action is...itself..or more correctly the programmer who created it, or the crap data it was trained on.
If the stories about 'Olympic Airways 411' are true, the only thing that stopped a disaster was a bloke with experience - something that AI, and it's coders, cannot have (Especially as the human through the rules book out of the window..which AI would never consider doing!)
When it comes to flying lots of people about, the best things to oversee that activity are other humans... assisted by automation, yes...replaced by it, no.
OK, I guess supply/demand/costs will win and either single-or-zero pilot air travel will happen. When it does, I hope the CEOs of the airline and/or of the company doing the flight control software are forced to be on the certification flights and 'eat their own dog food'.