Re: Yes, I know. I'm a wimp.
Just withdraw their Implied Right of Access, tell them that you are well aware of the licensing regulations and that you will also take action under the Malicious Communications Act 1988 if they issue any further threats or send the boys round.
They won't pester you again.
BTW, there's no obligation to tell them whether you have a TV, nor do you have to let them in to check (except in the unlikely event that they obtain a valid search warrant).