Just checking sitemap.xml
is not enough. There are several reasons that that check might fail even though the site has a sitemap.
- Did you check
sitemap.xml.gz
? Google supports gzip of sitemaps. Large sites with large sitemaps are likely to take advantage of this feature.
- You can specify the name of the sitemap file in
robots.txt
. It doesn't have to be sitemap.xml
. A robots.txt
can have a line like this that says where to get the sitemap. Sitemap: http://example.com/mysitemapfile.xml
- It is also possible to create a sitemap file at a "secret" URL. You can tell Google about this URL through webmaster tools. That way Google will get your sitemap, but it won't be easily accessible to the public.
Looking in robots.txt, many of the sites you list do have sitemaps.
Sitemap: http://www.amazon.com/sitemap-manual-index.xml
Sitemap: http://www.amazon.com/sitemap_vendor_videos_us.xml
Sitemap: http://www.amazon.com/sitemap_vod_index.xml
Sitemap: http://www.amazon.com/sitemaps.f3053414d236e84.SitemapIndex_0.xml.gz
Sitemap: http://www.amazon.com/sitemaps.1946f6b8171de60.SitemapIndex_0.xml.gz
Sitemap: http://www.amazon.com/sitemaps.bbb7d657c7e29fa.SitemapIndex_0.xml.gz
Sitemap: http://www.amazon.com/sitemaps.11aafed315ee654.SitemapIndex_0.xml.gz
Sitemap: http://www.amazon.com/sitemaps.c21f969b5f03d33.SitemapIndex_0.xml.gz
Sitemap: http://www.ebay.com/lst/PDP_US_main_index.xml
Sitemap: http://www.ebay.com/lst/PDP_US_incremental_index.xml
Sitemap: http://www.ebay.com/lst/VIP_US_index.xml
Sitemap: http://www.ebay.com/lst/SRP_US_index.xml
...
Sitemap: http://www.microsoft.com/en-us/dynamics/sitemap.xml
Sitemap: http://www.microsoft.com/en-us/kinectforwindows/sitemap.xml
Sitemap: http://www.microsoft.com/en-us/sqlazure/sitemap.xml
Sitemap: http://www.microsoft.com/en-us/cloud/sitemap.xml
Sitemap: http://www.microsoft.com/en-us/server-cloud/sitemap.xml
...