1

I am using a CMS which offers a wide range of options which lead to many additional special pages mainly in a way duplicating the same content as the original article. These generated pages are accessible via URL parameters and are automatically linked to from other pages by the CMS.

My problem is that I want to get rid of these special pages because...

  1. in my eyes they contain hardly any worth for my users
  2. they generate duplicated content
  3. search engine waste their time crawling all these generated pages

Unfortunately I am not able to deny the CMS generating those special pages and even if I could many of this unwanted URL parameters are already known to the search engines.

So I plan to use a .htaccess 301 redirect to remove each URL parameter leading to such special pages. So for example:

article.php?userid=1&highlight=red&background=blue

would be 301'ed to

article.php?userid=1&highlight=red

and this in the same call to the final call of

article.php?userid=1

So from an SEO perspective, I will have dozens of different links with parameters set in my very own website which actually via 301 all lead to the same webpage. My hope is that search engines would not consider them as duplicates and after a while would stop crawling them (as I also but with no effect told them in search console and the like).

For the UX I only see the drawback that those redirected links will not have the promised effect, but I think I could neglect that as those links are very rarely used.

My question is what harm this may do concerning UX and SEO when I (multiple) 301 redirect pages my website is actually linking to?

Maybe someone even has a better way to handle this situation?

Improve this question
10
  • Any type of PARAM for both SEO & UX is not freindly. Better to use /article/username/red/
    – Simon Hayter
    Feb 15, 2016 at 12:35
  • @SimonHayter: yes, I prefer "pathified" URLS instead of URL parameters as well, but I have no influence to change that in the given CMS. So I try to find a way to heal that as good as I could via .htaccess. Especially in my case the problem are unwanted parameters leading to dublicates I want to get rid of so it would be done to simply "pathify" the URL.
    – Wooz
    Feb 15, 2016 at 12:51
  • It would help to know what these parameters do. If I can guess that highlight is for something like highlighting search terms, and background is a user preferences setting that changes the appearance of the page, I'd be inclined to use canonicals for these instead of 301s. You can also tell Googlebot to ignore certain URL params via. webmaster tools.
    – Tim Fountain
    Feb 15, 2016 at 12:59
  • @TimFountain: Actually there are various parameters which I can't name all but mainly they simly change the layout or aggregate existant content. After all they lead to many pages of mainly useless dublicate extra content. - Unfortunally I don't know how to set canonicals neither as it is hard to get into the CMS source. In addition I wonder if canonicals would stop the search engine from crawling the amount of generated pages and so wasting the "crawl budget" of the site. - I added "ignore" in webmaster tools to several params in Oct'15 but those params are still crawled by Googlebot.
    – Wooz
    Feb 15, 2016 at 13:35
  • I would not 301 them, but use noindex and/or a canonical link back to the original. I am not sure that is an option for your CMS, however, this would normally be the preferred method to remove the links from the SERPs and avoid a duplicate content issues. You only need one or the other, however, if both can be done, that would be best.
    – closetnoc
    Feb 15, 2016 at 16:41

1 Answer 1

Reset to default
1

There is a better option, at least as far as Google is concerned. In Search Console there are settings for URL parameters. You can tell Google that certain parameters don't change the content on the page. Then Googlebot will stop crawling them and instead crawl "one representative URL."

Here is Google's help page where they explain how to use this feature: https://support.google.com/webmasters/answer/6080550?hl=en

Your 301 redirects are not ideal, especially if you can't stop your site from linking to the redirecting URLs. Googlebot will continue to crawl the URLs even after you 301 redirect them. The redirects would prevent Googlebot from finding duplicate content, but Googlebot is pretty good about detecting and handling duplicate content anyway. It usually just picks one of the pages to index and ignores the others. Some duplicated pages like this on your site won't harm your rankings. See What is duplicate content and how can I avoid being penalized for it on my site?

Improve this answer
2
  • Thanks for that answer. Search Console does not seem to work too well. At least I set some params to "effect=other, crawling=No URLs" (this is a translation, maybe the wording does not exactly match with the english version) in Oct'15 which are nevertheless still crawled by Googlebot. You and the Google-article you linked also tell "you can choose a representative URL", but unfortunally I did not find/understand how to set this. - Your hint with Google continue crawling the 301s is good. So it seems that this way would not help my "crawl budget". As a result of the discussion at the OP I ...
    – Wooz
    Feb 15, 2016 at 21:44
  • therefore decided to use a combination of robots.txt exclusion and 301 redirect, even though it does not seem to be a really satisfying solution. - After reading the second link you added about duplicate content I am still getting more and more impressed about the difficulty of the matter. On the one hand it names "Printer-only versions of web pages" as candidates for duplicate content on the other hand "Store items shown or linked via multiple distinct URLs" is said to be bad as well - unfortunally that's maybe exactly what my 301 redirect will look like.
    – Wooz
    Feb 15, 2016 at 21:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.