1

For a long time I've been getting Googlebot found an extremely high number of URLs warnings, which I know don't necessarily indicate there is a problem, but I am trying to reduce the amount of links that are crawlable by Google to help him focus on what's more important.

I'm thinking of changing a URL Parameter settings with a canonical link. I have filter parameters which I've set as Narrows and Specifies in WMT URL Parameters: enter image description here
These internal filter pages have value, but bringing the user to the main page without any filters is almost good enough, and I think this is a trade off I should make considering I have too many pages compared to my reputation, thus getting those warning messages.

Two options I'm thinking of:

  1. Adding a meta canonical tag for urls with a filter parameters selected, to the page without the narrowing / specification of these filters.
  2. Adding robots.txt to avoid crawling of those pages. (this will not waste the crawler's time trying to find the canonical meta tag, but will throw away the current link power these pages have to the trash, instead of passing it on the the page without any filter selected).

Would you recommend any of these?

Improve this question

1 Answer 1

Reset to default
1

Specifying a canonical on the filter pages to the parent (non-filter) page is perfectly fine but you can also serve a meta noindex, nofollow on these filter pages as well and instruct Google what to do with these filter URL's (if they contain parameters) from the section of Google WMT you've already mentioned. Disallowing them in robots.txt is also another option but won't help remove them from Google's index if they are already indexed.

There's no harm in implementing all of these options to ensure that Google only crawls, indexes and places weight on the primary pages of your website which you'd want performing and ranking well.

Improve this answer
2
  • Isn't Google supposed to do a pretty good job on his own on placing weight on the primary pages?
    – Noam
    Apr 7, 2014 at 13:58
  • Yes but they'll still index every possible URL they can crawl unless instructed otherwise and when it comes to filters (URL's with parameters in) such as on ecommerce stores this can essentially result in hundreds of pages with essentially the same content on making it increasingly difficult for the likes of Google to determine which page is the most important for appropriate terms.
    – zigojacko
    Apr 7, 2014 at 14:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.