1

I want all my subdomains to not be indexed under my domain, that means when I search in Google for example.com to not appear in index test.example.com, but when I search test.example.com to be indexed.

Is it possible to add a Code in domain.com robots.txt to not index the subdomains when search on google site:example.com?

Now if I search on Google site:example.com Google shows results for all my sub-domains, but I want results only for example.com and not test.example.com.

Only when I search site:test.example.com I want the results for the sub-domain.

Improve this question
2
  • 1
    This is confusing. Can you clarify this some??
    – closetnoc
    Commented Feb 26, 2016 at 22:18
  • Aren't you referring to the Google search results, rather than the actual "index" status of your subdomains? This would also make any reference to "robots.txt" irrelevant?
    – MrWhite
    Commented Feb 26, 2016 at 23:18

2 Answers 2

Reset to default
2

In Google, to do a site: search on the main domain (eg. example.com), but exclude the subdomain test from the search results you would use the search:

site:example.com -site:test.example.com

You would need to do this for each subdomain you wish to exclude from the results.

Is it possible to add a Code in domain.com robots.txt to not index the subdomains when search on google site:example.com?

However, you can't influence this behaviour in your server-side code, it is simply how Google's site: operator works. If your subdomains are indexed at all then they could be returned by a site:example.com search.

The only way to prevent your subdomain from appearing in the search results is to not have it indexed at all, which is not your intention.

Improve this answer
3
  • Oh! Now I get it!!
    – closetnoc
    Commented Feb 26, 2016 at 23:37
  • 1
    I think he want to enforce it from the server (either on example.com or on test.example.com). A user who only knows about example.com and does a search for site:example.com should not even learn about the existence of test.example.com. At least that's how I read the question.
    – kasperd
    Commented Feb 27, 2016 at 7:45
  • 1
    @kasperd Yes, I think you're right - I've updated my answer. Thanks.
    – MrWhite
    Commented Feb 27, 2016 at 10:26
0

I usually handle this with an X-Robots-Tag HTTP header on any subdomains I create. Setting something like Header set X-Robots-Tag "noindex" in the Apache VHOST or .htaccess file, or add_header X-Robots-Tag noindex; in Nginx, will tell search engines not to index anything on that subdomain.

This will have to be done for each subdomain you create, but it's not so much of an extra effort.

Improve this answer
1
  • 1
    Although they do appear to want the subdomain indexed, but only to appear in the SERPs for a site:subdomain.example.com search, but not included in a site:example.com search.
    – MrWhite
    Commented Feb 27, 2016 at 10:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.