1

Our website has seen a pretty big drop in indexed pages over the last few weeks and we are trying to figure out what has happened. We have seen a decrease from 68.000 indexed pages to a low of 37.000 pages.

It seems to, potentially, be a result of adding two boilerplate texts on these pages: an introduction of a few lines which is, except for the parameter the website is hitting, nearly identical, and a faq section which is exactly identical. The assumption was that our core content for each site was differentiating enough to allow for such boilerplate content, which we think adds to the user experience on the website. We cannot, however, exclude that it is just the algorithm making a change based on something else (we are only 6 months old).

We have already tried to limit this duplicate content but so far it does not seem to have helped to return to the number of indexed pages we previously grew to.

Now we were wondering whether it could be helpful to make sure the meta descriptions are unique for each of these pages, to help google identify the pages as indeed different. Currently, our meta descriptions are exactly the same except for the parameter that determines what is seen on the page.

Given that I have heard some conflicting views (meta descriptions have to be unique vs it does not really matter), I wanted to ask the community what they think.

1) Do you think the drop in indexed pages can be caused by adding the boilerplate text at scale (all these pages), and; 2) Do you think differentiating the meta descriptions to a larger degree can aid in resolving the drop in indexed pages?

I hope I have been able to somewhat convey the problem and the solution we are considering trying out. Thank you very much in advance.

EDIT:

We have submitted (and google has found/crawled) over 300k pages of which a lot will have a noindex tag. This is expected. Robots.txt is not blocking any relevant pages and nothing has changed in the file that could cause the drop. I guess what I am really asking is what kind of things can cause such a sudden drop in indexed pages, and could the things that I mention above play a role (e.g. the boilerplate text and highly duplicate meta descriptions). I cannot specifically find a page that has been dropped. Also, I tried adding a suggestion page but it does not allow link shorteners and I want to prevent posting actual links of the website. Hopefully, you can still provide some insight.

Improve this question
3
  • You may be able to lower those variables by including how many variations you have added to Google Search Tool? the contents of your robots.txt and an example a page that has been dropped including the header response using CURL.
    – Simon Hayter
    Commented Apr 12, 2018 at 23:17
  • @ValentijnvandenHout Are the 300K pages you mention in your edit also part of the 31K that were dropped? Obviously having a noindex on those pages would result in that. Are you seeing any manual actions in Google Search Console, or can you correlate the time period with these dates for Google algorithm changes? Thin or autogenerated content is a pretty common reason for de-indexing that we see here.
    – dan ♦
    Commented Apr 13, 2018 at 0:41
  • Hi @dan, the 31k pages were indeed in that set of 300k. After having submitted these pages, the index pages grew over time to 68k and stayed there. Though, since a week or two, it all of a sudden dropped to 37k. It seemed to be after adding the boilerplate texts, but it might be coincidence and just Google deciding based on another factor.
    – Valentijn van den Hout
    Commented Apr 13, 2018 at 9:29

1 Answer 1

Reset to default
2

Deindexing of pages on a site can happen for many reasons, so it's hard to specify the exact cause sometimes.

Some of my sites had many pages deindexed, while others continue to grow in indexing and traffic. And it can be difficult for me to figure out the cause for this as well.

It's possible that your duplicate boilerplate content did indeed cause this, especially if it's high up on the page. But it isn't necessarily the case that this was the reason.

Your duplicate meta tags definitely are an issue though, and you should immediately try to create unique meta tags for your site if possible. If you cannot, I would remove the meta description altogether and allow Google to choose which text to display. Having duplicate meta descriptions and/or title tags can be a massive problem.

Sometimes Google crawls and indexes an entire site and then just decides that the site isn't worth sending traffic to. It can be tough to determine why it has decided that.

Overall, if you want this site to succeed on Google you can continue to implement good practices and try to improve the core of all of your SEO.

This includes:

  1. Great on page SEO (Title, meta tags, keyword rich).

  2. Amazing, unique content

  3. Extraordinary backlink profile

  4. Great user interface and design

If your website is fantastic and you build up strong links, it will be tough for Google to ignore your site. It's also important to target longer tail searches. If you're competing for keywords that are already highly competitive with many good results, it may be difficult to ever rank highly on Google. And this competitiveness can cause Google to deindex a site as there are already so many pages on the web offering good results for those keywords.

Improve this answer
1
  • Hi Michael, thanks for your response. We are hitting most of the points you are mentioning and overall we are seeing great results. However, the de-indexing seems to have significantly slowed the growth we expected the last few weeks. I will make sure to address the duplicate description issue as soon as I can and will also try to get rid of the boilerplate content on the pages where the core content is relatively thin. Hopefully, that will do the trick. Fingers crossed. Thanks again for sharing your thoughts! Take care
    – Valentijn van den Hout
    Commented Apr 12, 2018 at 22:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.