Actually they can - at least that's my reading of the law in the USA and UK - based on supreme court judgements in the respective countries.
I see your slippery slope and i raise you 'letting bigoted opponents of free speech for anyone except themselves call for armed insurrection is the thin end of the wedge'