Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

alt text

The crawl report shows this: "HTTP ‎(1)‎ In Sitemaps ‎(5)‎ Not found ‎(993)‎ Restricted by robots.txt ‎(487,544)‎"

So the discrepancy between 50,000 and 44,592 isn't coming from that. Are there any obvious reasons?

share|improve this question
1  
Restricted by robots.txt ‎(487,544) : 5,408 missing urls can't be in there ? –  Chouchenos Oct 21 '10 at 7:52
    
@Chouchenos no, because those would appear in the part where it says (x) in sitemaps. –  Michael Pryor Oct 21 '10 at 13:40
add comment

2 Answers

The difference lies in Google's priorities: you may have pages in your sitemap, however, for whatever reason (be it lack of incoming links to deep content, lack of in-site links, delays in posting back index data, et cetera) Google either doesn't have the content indexed or isn't showing you what's actually indexed.

We're aware that we sometimes return erroneous estimates for the number of results that return for a query, and we're working to improve these estimates.

Features: Incorrect search results count

Given that Google doesn't make it a priority to show you exactly how many pages are in its index for a given search, I'd say it's just a matter of Google failing to show you an accurate picture of its index.

share|improve this answer
    
Google's estimates might be inaccurate, but I would have said it was quite probable that not all submitted pages will be indexed. –  w3d Oct 21 '10 at 11:07
1  
@w3d - I'd say it's more likely that Google has more pages indexed than it ostensibly reports (pages disallowed by robots.txt are still indexed in some form, if only to track the number of links directed at the off-limits URI, et cetera) - it's just not in Google's interest to reveal everything it knows :) –  danlefree Oct 21 '10 at 12:43
add comment

I think it is simply that Google does not necessarily index all the URLs submitted. The submitted sitemap is advisory, it is then up to the Search Engines to determine whether those URLs should be (or are worthy to be) indexed.

Or, may be Google just hasn't gotten round to checking/indexing those other pages yet? By look of your screenshot you only submitted the Sitemap today?!

share|improve this answer
2  
No, the sitemap is downloaded every day by google. That is just showing the last time it was downloaded. –  Michael Pryor Oct 20 '10 at 22:37
    
@Michael So those 5,000+ pages aren't a recent addition then? –  w3d Oct 20 '10 at 22:45
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.