"I'm as hypocritical as the next person (unless I'm standing next to a politician) and I buy from Amazon."
It's not really hypocrisy, it's choice. You have decided that your life is better with Amazon's prices, guarantees and convenience than with any of the alternatives. If you sacrifice Amazon on the altar of disapproval, you lose those benefits - you make yourself worse off.
Of course, if you chase Amazon out of the UK, you might get the benefit of tax paid by the alternatives that arise. Yep - tax off cluttered bookshops, failed startups, failing legacy companies, scam artists and cowboy outfits. Don't count on that adding up to very much, though.