2

In our site, we have links as such:

/{{category-name}}.html - will display all products

/category-name/filters/{{some-filter}}.html - will display filtered products

The problem we are facing is that google is crawling all pages including the filter pages. As you might imagine, we would like to prevent google from listing the filter pages in the search results.

robots.txt was my gut instinct but, I came across some information that suggests that having canonical links on the filter pages which points to the category page may be a proper solution and will lead to only a single search result rather than a myriad.

Is that correct? Is the canonical link approach the proper way of solving this problem?

Improve this question
1
  • Are the category pages paginated, or are all products on the same page?
    – unor
    Commented Nov 20, 2015 at 16:13

1 Answer 1

Reset to default
1

Canonical links does not prevent Google crawling or indexing them, it tells Google the preferred page. It is useful for when pages are duplicate, for example:

  • https://example.com/test.html
  • http://example.com/test.html

Canonical can help prevent duplicate pages entering the index, but unique pages will be indexed regardless, tag and category pages are not duplicate, they vary from one another so therefore are not duplicate, and obviously will be indexed.

If you want to stop Google then your need to use noindex, nofollow and robots.txt methods. Using either methods inform Google not to index those pages.

As you might imagine, we would like to prevent google from listing the filter pages in the search results.

Might imagine? since when? why wouldn't want Google to crawl those pages that are obviously helpful to your users. Pages such as tags, categories and so forth should be indexed period. It'll be helpful to know what type of filtering you are referring too, i.e date, time etc.

Correct me if I'm wrong... but I fear that you have read some outdated SEO guide that has informed you that you should only index pages you want to rank for, in the attempt to squeeze every little bit of juice to the pages that you want ranked.

You should always index helpful pages, if they are not helpful then don't have them period, never mind not indexing them. If they are helpful then you can still get SEO benefit from them, as people often link to TAG pages etc, if they are noindex... you won't be rewarded, Google does not punish for tag related pages, nor does it reward less for having them...

Should be noted that Stack Exchange has 'MILLIONS' of tag/category pages with no problems, they are helpful, index em.

Improve this answer
2
  • 1
    "It is recommended to use both methods." - who recommends this? And why? Google states: "Important! For the noindex meta tag to be effective, the page must not be blocked by a robots.txt file." (emphasis Googles)
    – MrWhite
    Commented Nov 17, 2015 at 1:11
  • Hey @w3d, I remember reading both, as the noindex is a fall-back if the robots.txt is deleted. But looking on their site they no longer mention this at all. Answer edited, thanks.
    – Simon Hayter
    Commented Nov 17, 2015 at 9:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.