0

I have a hosting which has several websites in www folder. I set up a WordPress website in root directory for main domain.

Do I need disallow other websites directory in robots.txt? I think these directories are not relate to the main domain. But is this will totally disallow other website?

  • Adding disallow tells robots to ignore the specified disallowed directories. However if there are other links to them (such as external links), they may get crawled anyway. If you want to "totally" disallow them from appearing in search engines, then prevent them from being crawled (e.g., password protect them). – dan Aug 1 '13 at 8:04
1

You could use robots.txt to block the directories but there is an easier way by using canonical pages on your websites.

This will ensure sites that are hosted within containers within 1 hosting account for example:

  • /public_html/ = domain1.com
  • /public_html/domain2.com = domain2.com would only be indexed on the domain that the canonical is setup on.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.