1

Possible Duplicate:
Why the # of indexed pages got dicreased?

My site's indexed pages drop every now and then. It suddenly drops from ~800.000 to ~20.000 pages, along with a huge drop in daily crawled pages (according to Google's webmaster tools) Then it raises again during the next 1-2 months, just to drop again, and the cycle repeats.

I happens in both site:www.example.com and example.com. I'm very worried and I can't figure out why is this happening without making any changes in the site's link structure.

This is affecting our rankings too. I suspect there's something annoying Goggle in our site, but it's driving me nuts to find the clue.

Improve this question
1
  • Does Google Webmaster Tools report any crawl errors? Have you considered telling Google to ignore URL parameters instead of using wildcards in your robots.txt file?
    – danlefree
    Commented Dec 26, 2011 at 23:51

2 Answers 2

Reset to default
2

Using the site: operator is almost useless for gauging how much Google is indexing your site; that's not what it's meant for.

To find out how many pages Google actually has indexed you'll need to register for Google Webmaster Tools and view data there. If you submit a sitemap it tells you how many of the URLs in the sitemap it has indexed.

You should also check your robots.txt file. You need a gap between separate rule groups, ie before the Google-Mediapartners rule and before the * rule. Otherwise all use agents will think every rule in the file only applies to Twiceler.

Furthermore, your last list of disallows looks like a misguided attempt to avoid duplicate content woes. Those should be instead fixed using canonical URLs.

Improve this answer
2

You said:

I happens in both site:www.example.com and example.com. I'm very worried and I can't figure out why is this happening without making any changes in the site's link structure.

you have to site: example.com and www.example.com
I think the problem is that google has recognized two different sites with the same content.

You should redirect example.com to www.example.com
In search results all the example.com pages should be gone.

Improve this answer
1
  • +1 This isn't the core problem, but the two versions of the domain do tend to produce wildly different results(google.com, www.google.com). So for whatever limited use this is worth, you should be consistent about which you try.
    – Su'
    Commented Dec 27, 2011 at 10:05

Not the answer you're looking for? Browse other questions tagged or ask your own question.