"Assistants are always listening, this is bad. I want them to listen more to fix this"
You do realise that you contradict yourself throughout this article?
"Voice assistants are always listening. So why won't they call Police if they hear a crime?"
Because that device listens for something that resembles a button being pressed. It does not continually upload the stream of audio to the cloud to be processed. Except you appear to not want this...
"These devices, or the cloud services that power them, can easily understand when someone is angry, or terrified or in pain. It should almost be trivial to detect when something is way out of range, and flag that."
It isn't, unless you want them to continually stream your home audio to the cloud for processing, which you appear to be against.
"Listening means being responsible for whatever you hear."
Which is why they don't listen. They wait for a predictable nudge; a vocal button-press. And if you don't press the button (say the wake-word), they don't hear anything.
"We're listening as never before, and we have to do something about it."
No, we aren't. This has always been the same; saying "Alexa" to wake up the hockey-puck is no different to pressing the "voice command" button on the side of your BlackBerrry 9700, except instead of pressing a button with your fingers, you press it with your voice.
The one exception would be Samsung's smart TV; I don't see that brand name in your diatribe.