Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

Google Webmaster Tools says ~14000 URLs in the index but as you can see in the following picture there are only ~6500 from the sitemap. How comes?

I mean: my website is not properly "crawl-able". So google can only use the sitemap (and that's where I'm lost actually)

For example: if you google (in France) this: "tondeur de moutons lot et gronne" my website is second. But you can't access directly to this page from my website (= you have to go through the "search page" and type by hand "tondeur".

So what I mean is that there are very few pages (~ 100) that a bot can "guess" without the whole sitemap.

  • google may not have crawled the full website but it says "14000 URLs are in the index"
  • if so, why google can display some results that are only in the sitemap?
  • if google has indexed ~6500 URLs from the sitemap, why does it say "14635"?

So... well err I'm lost... any idea?

Google Webmaster Tools URLs in the index

share|improve this question
add comment

2 Answers 2

up vote 1 down vote accepted

Just because a page isn't listed in the sitemap doesn't mean Google won't index it. Google will follow links on pages listed in a sitemap even if that page isn't itself listed on the sitemap. So if you have pages that are not listed on any of your sitemaps that are linked to by a pages that is listed on a sitemap then Google will index it and that probably is the case here. It doesn't have to be deep links either. Just one link here and there adds up quick especially if the links use query variables for any kind of search, etc. Somewhere you have links to pages not listed in your sitemap that you (apparently) do not know about. Maybe you can use Google to help identify what those pages are and figure out why you don't know why you don't have them in your sitemap.

share|improve this answer
    
So that would mean that google actually uses sitemap, even though the pages have are ranked "0", and maybe from there, it can find pages that get a higher pagerank? This page made me think sitemap were useless: webmasters.stackexchange.com/questions/4803/the-sitemap-paradox –  Olivier Pons Feb 6 '12 at 8:11
add comment

Sitemap has nothing to do with PageRank. If you find the contrary somewhere, please cite ;-)

Sitemap may help bots to index your site easier. That's all! (Of course, it also help you to know how many pages you have etc. but that's another story.)

You says "my website is not properly "crawl-able". So google can only use the sitemap". Do you mean that there are no links at all in any page in your website, therefore bots can not follow links? So, it's very important to fix it first.

Ok, let come back to your answer. Even if your site is not crawlable, even there is no link at all, bots can still guess the URLs. E.g. if you have 3 url /page/1, /page/2, /page/4, googlebot could guess and try to index /page/3 even if /page/3 is not in sitemap and no page has a link to it. Of if you have /page/1?p=1, /page/1?p=2, /page/2?p=1, googlebot could guess you'll have /page/2?p=2 and try to crawl that page. The easiest way is to turn on web server access log then look at urls crawled by googlebot.

share|improve this answer
    
That's exactly what I did and google is using my sitemap, but I guess you're right: Sitemap has nothing to do with PageRank. –  Olivier Pons Feb 6 '12 at 15:07
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.