I have a subomain which I do not want to be listed in any search engine result. Let say I have:

  • http://www.example.com
  • http://child.example.com

How can I hide all URLs of child.example.com domain which are currently showing in the search engine results?

4 Answers 4


In the root of the directory of the subdomain website, add a file called robots.txt containing:

User-agent: *
Disallow: /

This will tell web crawlers not to index the site at all. They do not have to obey, but the main ones will.

  • Hi, i believe this is the best thing. And one more thing let me know, if i do that, how long will it take to effect on the Web Results? (e.g. Google Search) Do i need time or will it take suddenly? Jan 10, 2013 at 9:58
  • 2
    It will be acknowledged the next time the indexers crawl the site. This can be anything from minutes to weeks, depending on how important your site is considered. You can move things along more quickly by adding the site to webmasters at Google and Bing and getting it to reread robots.txt - you can also remove pages from the index there.
    – Paul
    Jan 10, 2013 at 12:03
  • 1
    If you want to speed up the process you can submit a removal request at google.com/webmasters/tools/removals for Google and submit the URL at bing.com/toolbox/submit-site-url for Bing (yes, that's the recommended method for removing pages).
    – Kat
    Jan 10, 2013 at 12:22
  • You should also add noindex to the index and other pages on the sub-domain
    – Anagio
    Feb 23, 2013 at 18:11
  • How do I do when there is common root folder and subdomain names are dynamic? stackoverflow.com/questions/21229235/…
    – Gopi
    Jan 22, 2014 at 6:21

Using a robots.txt file in your subdomain will help (and Google will obey this), but another step you can take is to specify with a Google Webmasters account that you don't want this subdomain to be indexed. You can also use a meta tag on all pages in the subdomain:

    <meta name="robots" content="noindex">

If this happens to be a site that you are only using for internal testing, limiting visibility to a specifies set of IP addresses in your virtual hosts file would further hide the site.

  • In all pages????? oh no.. Jan 10, 2013 at 9:05
  • @4lvin It's done dynamically from the server side. You only need to code it once. Please don't tell me you're coding individual HTML pages.
    – Kenzo
    Jan 10, 2013 at 10:36
  • LoL, actually "yes". Because this is very old/legacy web service system. (thats why it needs to be bring down) Many isolated pages inside. :D Jan 10, 2013 at 10:37
  • @4lvin Ouch...good luck with that!
    – Kenzo
    Jan 10, 2013 at 10:39
  • 1
    Instead of embedding a noindex meta tag in every page, you can instead return an X-Robots-Tag HTTP response header from your server for the specified subdomain - this you only have to do once.
    – MrWhite
    Jan 10, 2013 at 11:27

Solutions from Kenzo and Paul are good, you can put meta tags noindex on your web pages and add robots.txt to disallow robots.

But in my opinion, the best solution is to use password authentification on your sub domain. This is the only solution you're sure robots can access and index your web site. If you use Apache, you can implement htpasswd.


...currently showing in the SEO results?

The other answers are more about proactively preventing the indexing of a (sub)domain (which is primarily what you are asking in the question) rather than actively removing your pages from the search results, which might be more what you are after, judging by your other question.

You still need to block your site in robots.txt and serve a noindex meta tag (or X-Robots-Tag HTTP response header), as stated in the other answers, but you also need to block access to your pages, returning a 404 or 410 status code.

You can read more about this on the Google Webmaster Tools help page:
Requirements for removing content

Once you have all these in place then you can use the Remove URLs tool in Google Webmaster Tools. However, this only applies to individual URLs, not an entire site, but it's a start. (Google states that using robots.txt, noindex and serving a 404 are the requirements in order to use the GWT removal tool.)

However, if you still want regular users to be able to access the site by typing the URL, then it's a problem - as your content is no longer available. You could password it, as Zistoloen suggests, however this will return a 403 (Forbidden) by default which you would need to override to return a 404/410. You could cloak your content, returning a 410 to the Googlebot and allowing everyone else to enter - but what about people clicking on results in the SERPs?

But if you want Google to remove your content in the fastest possible time then you need to remove it from the "Google" internet.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.