Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

robots.txt is supposed to go in the root directory, I know. But, could it be overridden by another robots.txt inside a sub directory?

share|improve this question

migrated from stackoverflow.com Mar 19 at 1:42

This question came from our site for professional and enthusiast programmers.

    
This question appears to be off-topic because it is about SEO –  John Conde Mar 12 at 18:04
    
@JohnConde Yea, I feel not sure to post the question here, but I found many other questions related to seo and robots.txt, so decided to post it anyway. Where should it go if this is not the right place? Thanks. –  Marco F. Mar 12 at 18:36
    
Thanks @JohnConde, seems as I can't move it with my current reputation, and you can move any question, do you mind if you do it for me? Thanks! > meta.stackexchange.com/questions/85017/… –  Marco F. Mar 12 at 18:44
    
I can't move it but if it is flagged a mod will do it for you. –  John Conde Mar 12 at 18:52
add comment

1 Answer 1

up vote 3 down vote accepted

No. It is only valid to have one robots.txt file which is paced inside the root web directory of the website. Other robots.txt files are invalid and will be ignored (and never even looked for as crawlers only look in the root directory and never elsewhere. You can't even tell them to look elsewhere).

share|improve this answer
    
Thanks for reviewing my question. Yes, I suspected it and I wanted to be sure it was this way. I believe you are right. Isn't there any link to a source where to verify/confirm this? –  Marco F. Mar 12 at 18:41
    
The official robots.txt website is about as good as it gets. –  John Conde Mar 12 at 18:52
    
If you have subdomains, each of the must have a robots.txt if not, the bots can crawl them. –  mouhammed Mar 16 at 0:23
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.