Prevent Google Indexing SubDomains

Question

I have cPanel Hosting with only 1 IP. I'm in the process of pointing all my TLDs to this server.

I have created Addon Domains and all the folders from which it creates are located in sub directories example1.com, example2.com in my root website /home/user/public_html/client/ but they can also be accessed from example1.maindomain.com. The thing that is worrying me is if Google indexes these subdomains.

What can I do to prevent this, would a rewrite rule like this be sufficient?

<IfModule mod_rewrite.c>
RewriteEngine on
RewriteCond %{HTTP_HOST} !^www\.example1\.com$ [NC]
RewriteRule ^(.*)$ http://www.example1.com/$1 [L,R=301]
</IfModule>

Or is there a better way?

Answer 1

If you don't use cPanel, the correct way to do this is to put each of domain into is very own directory. Apache is not really designed to handle overlapping domains the way that cPanel sets things up. This seems to be a limitation of cPanel.

One technique that should work is to treat all of your domains as "add on domains." Set your main domain name in cPanel to a subdomain that doesn't actually have a DNS entry. Something like allmysites.example.com. Then Google can't actually crawl the main directory at all. Your directory structure would then look like:

• allmysites (serves allmysites.example.com which isn't accessible through DNS)
  • site1 (serves site1.example.com)
  • site2 (serves site2.example.com)
  • site3 (serves site3.example.com)

Answer 2

Google, like all decent web crawlers, honors Robots Exclusion Standard (robots.txt). If there really is a limitation not to use separate folders for every domain and you don't want to treat all domains as cPanel "Add on domains", you could prevent the indexing of these folders (but not access to the folders) by adding /home/user/public_html/client/robots.txt:

User-agent: *
Disallow: /site1-directory/
Disallow: /site2-directory/

As the additional domains have different roots, this robots.txt will only affect the main site.