Webmasters Stack Exchange is a question and answer site for pro webmasters. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I know robots.txt is supposed to go in the root directory. But, could it be overridden by another robots.txt inside a sub directory?

share|improve this question

migrated from stackoverflow.com Mar 19 '14 at 1:42

This question came from our site for professional and enthusiast programmers.

This question appears to be off-topic because it is about SEO – John Conde Mar 12 '14 at 18:04
@JohnConde Yea, I feel not sure to post the question here, but I found many other questions related to seo and robots.txt, so decided to post it anyway. Where should it go if this is not the right place? Thanks. – Marco F. Mar 12 '14 at 18:36
Thanks @JohnConde, seems as I can't move it with my current reputation, and you can move any question, do you mind if you do it for me? Thanks! > meta.stackexchange.com/questions/85017/… – Marco F. Mar 12 '14 at 18:44
I can't move it but if it is flagged a mod will do it for you. – John Conde Mar 12 '14 at 18:52
up vote 3 down vote accepted

No. It is only valid to have one robots.txt file which is paced inside the root web directory of the website. Other robots.txt files are invalid and will be ignored (and never even looked for as crawlers only look in the root directory and never elsewhere. You can't even tell them to look elsewhere).

share|improve this answer
Thanks for reviewing my question. Yes, I suspected it and I wanted to be sure it was this way. I believe you are right. Isn't there any link to a source where to verify/confirm this? – Marco F. Mar 12 '14 at 18:41
The official robots.txt website is about as good as it gets. – John Conde Mar 12 '14 at 18:52
If you have subdomains, each of the must have a robots.txt if not, the bots can crawl them. – mouhammed Mar 16 '14 at 0:23

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.