1

My website has 100,000 URLs with auto-generated content and I'm suffering from thin content (drop in ranking and alerts of duplicate content in Google Search Console).

I want now to exclude those URLs with a very-low number of visits, compared with the rest of the website. My strategy would be:

  • I'll keep the contents, just because some people access those URLs from other webpages, and it may be useful for them
  • I'll insert 'noindex' in the headers of those URLs to be excluded
  • I will not insert nofollow links to the excluded URLs (I'll mantain the dofollow links), since I want Googlebot to crawl them to discover that they are noindex
  • I will not include the excluded URLs into the robots.txt, since I want Googlebot to crawl them to discover that they are noindex
  • I'll remove the excluded URLs from the sitemaps

I would like to know your opinion.

Improve this question

2 Answers 2

Reset to default
2

You'd be correct to use dofollow on inlinks pointing to the pages you want to exclude, but you won't be able to use dofollow on the noindexed pages themselves, if that's what you were asking. Noindex on a page equates to noindex,nofollow long-term.

Noindexing isn't the perfect solution to fixing thin/duplicate content though. For one thing, the criteria for whether content has value is not set by number of visits in a date range. For another thing, you also need to take into account backlinks to these pages, as they pass link equity to the site as a whole.

I would suggest first consolidating pages using canonicals or redirects, adding hreflang if it's generated based on location, and updating pages to make them more unique/robust, before you turn to noindexing pages. I'm not sure what type of content is on your site but setting up a good foundation so thin/duplicate pages are not auto-generated will be beneficial for the future as well.

If you go the noindexing route you have the right idea of what to do based on the steps you listed.

Improve this answer
-1

You never know, how certain page will get visitors after creation. Thats why your strategy should be like

  1. Set all new urls on index, follow,
  2. After three months after publishing set all URLs with less then 10 organic visits à day to noindex, follow,
  3. After they are away from index, set them with meta robots to noindex, nofollow and links to them to rel="nofollow".

On this way you get only good urls for indexing and crawling.

Improve this answer
6
  • That's not a good strategy - a page might get less than 10 visits a day and still be of high quality and relevant, it all depends the intent it is targeting.
    – Andre Guelmann
    Mar 6, 2020 at 16:30
  • less then 10 visits a day and still be of high quality and relevant? Sure, but still not worth to be indexed and maintained. We don't speak about pages, where one conversion gives you a million bucks;)
    – Evgeniy
    Mar 7, 2020 at 1:57
  • How do you know? He didn't mentioned anything about the type of website or industry he's in... I manage a website where some pages get less than 30 visits a month and yet they generate around 4-6 leads a month, so your strategy is too generic and might hurt his website without more information.
    – Andre Guelmann
    Mar 7, 2020 at 3:10
  • Thank you @Evgeniy for your answer. I do not share the threshold of 10 visit/day (in my case, I've got a long tail of URLs and most of them count), but I appreciate your strategy of "index" > Analyze > "noindex" > Wait to be excluded > "nofollow".
    – Hookstark
    Mar 7, 2020 at 9:59
  • @AndreGuelmann for sure he mentioned this: 100,000 URLs with auto-generated content. Do you really think there are pages comparable with those you manage? If you have only a hammer, everything seems to be a nail;)
    – Evgeniy
    Mar 8, 2020 at 13:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.