I read in this forum that this .htaccess code should block access to the robot.txt file:

RewriteCond %{HTTP_USER_AGENT} ^Mozilla
RewriteCond %{HTTP_USER_AGENT} !(Slurp|surfsafely)
RewriteRule ^robots\.txt$ /someotherfile [L]

But I just get a error 500: "Internal Server Error"

What do I do wrong?


Make sure you have a file named "someotherfile" in your site's root (and also obviously a robots.txt file). Also the .htaccess should have RewriteEngine On somewhere above the block you posted.

  • Thank you soo much!! - I had the RewriteEngine On, but the problem was as you said that I haven't created a file named: "someotherfile" THANK YOU! :D
    – user1325927
    Jun 10 '12 at 19:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy