They do that anyway. It's called robots.txt.
Not a member of The Register? Create a new account here.
Remember me on this computer? Post anonymously?