1

The website I am working on has a lot of news and much more content. We had the site since around 2010 and index was gradually growing up-to 350,000 pages. After 2016-08-15 we are losing indexed pages rapidly and now it is around 150,000 pages. (On 2016-10-25)

Note : We did not change Sitemap, Robot.txt in any way and we did not encountered any index drops before. The dropping is stabilizing now.

We identified following issues in our website.

  1. We have meta keywords tags which can be marked as keyword stuffed tags.
  2. Most of our pages are not responsive and not mobile friendly.
  3. Google index status graph shows one removed page after the above date.

We did not receive any manual action notifications or warning notifications from google.

Could someone please clarify what could have caused this issue? Is it possible google decided to de-index pages based on above listed issues or on one of their algorithm update or a penalty?

Update : I am not a SEO expert. I am a front end developer who just noticed this.

1

Google generally removes pages from the index for one of three reasons:

  1. It can't crawl them properly
  2. It found that they duplicate something else
  3. Nobody searched for them and they don't have many external links

For crawling, check Google Search Console and make sure that there are not unexpected crawl errors. Check your server logs and see that Googlebot is fetching your pages with a 200 status.

For duplicate content, search for a sentence from each page in quotes on Google. You may find that there is a scraper site out there that has stolen your content. Google occasionally indexes a scraper site instead of the original source of content.

For lack of interest, there isn't much you can do. I've observed that Google will often crawl and index a page for the first couple weeks before deciding that it isn't high enough quality to continue including in the search index.

  • yes it seems so. I checked which pages I lost and found most of such pages were static and lacks originality. They do not have any external links at all. I now understand it is not a surprise at all to get de-indexed. Your answer was much helpful. This was a good eye-opening :) – Prabhath Nov 21 '16 at 6:24
  • your no 1 "It can't crawl them properly" was spot on. In that period, our application server (TomCat) was upgraded and we lately found out the new version is buggy. Sometimes the HTTP response gives zero length for content because of the bug. The bug has been resolved in a later version. Ever since we upgraded again to a bug free version, Google index started increasing rapidly. It took some time for us to track the issue. Thanks for your value addition. It was really helpful. – Prabhath Jan 25 '17 at 12:03
1

You gain and lose indexing based on quality, importance and freshness of the content. Why does your site exist at all? If it exists just to get money from clicks, then you should lose out rapidly. That you are are keyword stuffing should help nudge that along.

So this shouldn't be surprising at all.

  • Hi Rob. the site does not earn money for clicks. Will improving quality of pages help regaining index and caching given that the crawler still crawls my site without any problem? – Prabhath Nov 7 '16 at 6:37
-1

My opinion is that your pages are losing indexing due to lack of promotion on the web. Just re-submit all the links on Google+ and fetch those links again using Google Webmaster Tools. You will find your answer then.

  • G+ doesn't have functionality for submitting links as far as I know. I also don't see how fetching links using the "fetch as Google" feature in GWT would help. – Stephen Ostermiller Nov 17 '16 at 11:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.