My site is set up so the main site is http://example.com and report pages are on Amazon S3 at http://reports.example.com.

There are obviously links between the two sites, I assumed that adding http://example.com to Webmaster Tools would also include http://reports.example.com but it doesn't. I have added that separately.

But is there way to tell Webmaster Tools they are part of the same website. And if I want to index all the pages on http://reports.example.com do I need an index.html file (or similar) which has hyperlinks to the pages as http://reports.example.com/index.html or could the files still be indexed if the listing file resided on http://example.com.

1 Answer 1


Every domain (in this case a subdomain) must has its own sitemap and its own robots.txt so you can teach to the crawlers (not only google even Yandex, Yahoo, Bing... ). The thing is that You can't mix urls from one sitemap inside the other .... so sitemap from subdomain only must contain urls from this subdomain not the main domain and same applies for domain sitemap as can't contains in its sitemap urls from the subdomain.

To summarize please create robots.txt and sitemap.xml.gz for every domain and subdomain as it's not enough to have a index.html

You can find very good resources searching for robots.txt and sitemap in Google


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.