Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

I have a hosting which has several websites in www folder. I set up a WordPress website in root directory for main domain.

Do I need disallow other websites directory in robots.txt? I think these directories are not relate to the main domain. But is this will totally disallow other website?

share|improve this question
    
Adding disallow tells robots to ignore the specified disallowed directories. However if there are other links to them (such as external links), they may get crawled anyway. If you want to "totally" disallow them from appearing in search engines, then prevent them from being crawled (e.g., password protect them). –  dan Aug 1 '13 at 8:04
add comment

1 Answer

You could use robots.txt to block the directories but there is an easier way by using canonical pages on your websites.

This will ensure sites that are hosted within containers within 1 hosting account for example:

  • /public_html/ = domain1.com
  • /public_html/domain2.com = domain2.com would only be indexed on the domain that the canonical is setup on.
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.