Sitemap contains a lot of products and web index contains a few according to Google webmaster tools

Question

Our sitemap has something like 10 million pages:

This is the data in the webmaster tools:

Submitted URLs 10,873,781 2 URLs in web index.

This is a big difference and an incorrect statement, since millions of our pages are in Google's index.

The sitemap files are valid and the google status flag shows that the sitemaps are fine.

When we asked Google about it they said that all our URLs ending in index.html caused the sitemap configuration to report wrongly.

Is the statement about the urls ending in index.html true? Does anybody have the same experience that ending in index.html causes the webmaster tools to report incorrectly?

Answer 1

It's hard to say without knowing the URL... However, assuming that all of your URLs end with "/index.html" it is possible that you are seeing a quirk based on the way Sitemaps processes URLs. In general, Google removes the "index.html" from the URL, if it appears that it is redundant, before checking the indexed URL count. For most websites, that's the desired way: while they generally allow both "/" and "/index.html", for users it generally makes sense to just keep the shorter version indexed. So if your website requires "/index.html" at the end of the URLs, and is indexed like that appropriately, then it is likely that the indexed URL count based on Sitemaps will not be correct.

In general, I'd recommend using URLs without redundant information (such as "/index.html" where "/" would be sufficient). However, I know that this is not always easily possible for all CMS that are in use... If that applies to your site, then I would recommend reviewing that decision the next time you work on a redesign, but I don't think you'd have to change anything urgently just for a better indexed URL count.