I have a website for managing trading card game (TCG) collections which is built around a central search engine/filter system. This filter system is located on the
cards page which lists all cards found matching the selected filters in a paginated way. If no filters are selected, all results are listed. Each result links to a detail page of the card.
Each filter is added to the URL as a query string parameter, so there might be millions URLs giving different results. Naturally, I don't want all these querystring based URLs to get indexed. In fact, I don't really care much about the content of the
cards page at all. What I care about are the card detail pages to get indexed. Essentially, I want Google (or any other crawler) to index the
cards page without querystring and to crawl through the pagination to index the card detail pages.
This would seem possible by excluding the querystring from the
cards page in robots.txt, however, almost all backlinks (and internal links) will probably link to a specific card search result (with filters). I guess this will affect my ranking in a negatively?
My best guess to get the card detail pages crawled, would be to include
<link rel="prev|next" href="/cards?page=number"> (and add an exception for the "page" querystring parameter). This would work because all cards are listed if no filters are specified.
These are just my thoughts; what would be the most appropriate way to handle this?
If I decide to crawl the pagination list, what happens if a user has a cookie set (for example 48 results per page instead of the default 24)? I know crawlers will always see the same values because they do not set cookies, but does it matter if the user can visit the same URL with different content? Can Google detect this and/or penalize me for this?