Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

I read in this forum that this .htaccess code should block access to the robot.txt file:

RewriteCond %{HTTP_USER_AGENT} ^Mozilla
RewriteCond %{HTTP_USER_AGENT} !(Slurp|surfsafely)
RewriteRule ^robots\.txt$ /someotherfile [L]

But I just get a error 500: "Internal Server Error"

What do I do wrong?

share|improve this question

migrated from stackoverflow.com Jun 29 '12 at 12:54

This question came from our site for professional and enthusiast programmers.

1 Answer 1

up vote 1 down vote accepted

Make sure you have a file named "someotherfile" in your site's root (and also obviously a robots.txt file). Also the .htaccess should have RewriteEngine On somewhere above the block you posted.

share|improve this answer
    
Thank you soo much!! - I had the RewriteEngine On, but the problem was as you said that I haven't created a file named: "someotherfile" THANK YOU! :D –  user1325927 Jun 10 '12 at 19:43

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.