Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a hosting which has several websites in www folder. I set up a WordPress website in root directory for main domain.

Do I need disallow other websites directory in robots.txt? I think these directories are not relate to the main domain. But is this will totally disallow other website?

share|improve this question
Adding disallow tells robots to ignore the specified disallowed directories. However if there are other links to them (such as external links), they may get crawled anyway. If you want to "totally" disallow them from appearing in search engines, then prevent them from being crawled (e.g., password protect them). – dan Aug 1 '13 at 8:04

You could use robots.txt to block the directories but there is an easier way by using canonical pages on your websites.

This will ensure sites that are hosted within containers within 1 hosting account for example:

  • /public_html/ = domain1.com
  • /public_html/domain2.com = domain2.com would only be indexed on the domain that the canonical is setup on.
share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.