Just cancel the payment method
Why does it not occur to people to cancel the direct debit/standing order or dispute the charge with a credit card company
Companies that make it trivially easy to sign up but impossible to leave (looking at you PlusNet / Amazon / McQueens Dairy etc) deserve everything they have to pay chasing bad debts that will never, ever be paid. Write them a short email cancelling the service, kill the payment method and (eventually they'll have sold the debt on to a bunch of shysters like Lowells) block the number of the inevitable begging calls.
Or is it so different in the US that doing so makes you a criminal?