Much like "the self-driving car crashed and killed someone".
Until THAT THING ITSELF is a legal entity that's solely taking responsibility, then I'm not going to let it have control of things I'm responsible for.
Whether that's "driving" or "data security".
And guess what happens when you try to get self-driving cars or AI models to be legally responsible for their output? Nobody's interested in giving you that guarantee and prices skyrocket (much like prices of aviation components, etc.).
Sorry, but I can have the power and the responsibility - or neither. It's not a mix-and-match situation.