dont they use a custom ua to request those urls?
ok so i made a page that returns the ua of the request in the title and shortened it with bit.ly
i got the title back as "bitlybot" (bitly bot is the ua that bit.ly uses) then i made it so it would "cloak" the page when bit.ly was requesting it <a href="http://bit.ly/cloakua">http://bit.ly/cloakua</a> now you get different page if bit.ly requests it rather than a regular user. this is even better if you know all bit.ly ip then you can do ip based cloaking which is even better
i know this is a simple attack but i think it demonstrates my point