For a long time I've been getting Googlebot found an extremely high number of URLs
warnings, which I know don't necessarily indicate there is a problem, but I am trying to reduce the amount of links that are crawlable by Google to help him focus on what's more important.
I'm thinking of changing a URL Parameter settings with a canonical link. I have filter parameters which I've set as Narrows
and Specifies
in WMT URL Parameters:
These internal filter pages have value, but bringing the user to the main page without any filters is almost good enough, and I think this is a trade off I should make considering I have too many pages compared to my reputation, thus getting those warning messages.
Two options I'm thinking of:
- Adding a meta canonical tag for urls with a filter parameters selected, to the page without the narrowing / specification of these filters.
- Adding robots.txt to avoid crawling of those pages. (this will not waste the crawler's time trying to find the canonical meta tag, but will throw away the current link power these pages have to the trash, instead of passing it on the the page without any filter selected).
Would you recommend any of these?