Webmasters Stack Exchange is a question and answer site for pro webmasters. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I read in this forum that this .htaccess code should block access to the robot.txt file:

RewriteCond %{HTTP_USER_AGENT} ^Mozilla
RewriteCond %{HTTP_USER_AGENT} !(Slurp|surfsafely)
RewriteRule ^robots\.txt$ /someotherfile [L]

But I just get a error 500: "Internal Server Error"

What do I do wrong?

share|improve this question

migrated from stackoverflow.com Jun 29 '12 at 12:54

This question came from our site for professional and enthusiast programmers.

up vote 1 down vote accepted

Make sure you have a file named "someotherfile" in your site's root (and also obviously a robots.txt file). Also the .htaccess should have RewriteEngine On somewhere above the block you posted.

share|improve this answer
Thank you soo much!! - I had the RewriteEngine On, but the problem was as you said that I haven't created a file named: "someotherfile" THANK YOU! :D – user1325927 Jun 10 '12 at 19:43

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.