I always find the most satisfying answer is to let them get started then say "Hang on, there's someone at the door - I think it's the parcel I'm expecting. Back in a sec" then just leave them hanging until they give up (often 4-5 minutes)
A variation if they ask for to speak to Mr Penygors is to say "He's just seeing a client out, he'll be with you in a moment" - same result
I believe that this sort of thing is a very socially aware and kind thing to do, as it prevents someone else being irritated by them for a whole 5 minutes. And those minutes of positive karma add up. And more satisfying than asking them if their mother knows they work in organised crime.