New answers tagged


When you have to execute your file, you allow it and you do not have to. User-agent: * Disallow: Disallow: /wp-admin/


It has very simple solution. You need to put the Robots.txt file in the root folder of your sudomain (i.e After that you can use Robots meta tag to disallow SE from crawling your site (i.e,


The correct answer to your question is to put a robots.txt file at the root folder of your subdomain or put robots meta tags on your subdomain's index page to block them from crawling and indexing.


This is definitely the case when you should use meta tag "robots" and "noindex" the subdomain or what ever pages you want to hide from serps. After that, you could consider using url removal tool from google and getting rid of pages you don't want to see in serps, which will speed up the process a bit. Do not go with robots.txt and blocking content of this ...

Top 50 recent answers are included