How to best create URLs for google and other search engines, using query parameters, or as hyphenated/nested URL paths?

Question

I am torn on what is best here, I have read tons of blogs on the topic but haven't found a really clear solution/answer.

Basically I am wondering one thing which has two implications:

1. Does Google (or other search engines) follow URLs with query parameters? It sounds like it does, but not 100% sure. It must sort the query params so ?x=1&y=2 is considered the same as ?y=2&x=1. I don't see how to can get out of the potentially infinite set of combos if there are say 5 query params each with 1000 values, that is 1000^5 or bazillions of combinations, so there's that uncertainty too.

• If it does follow query params, does it treat them equally in value with hyphenated and/or nested URLs?

For example, say you have a food recipe site. You can have URLs like this, allowing people to customize combinations, and each page should return a title which is unique to that combination, as well as the subset of content unique to that query.

• /recipes?spicy=true&contains=milk, title = Spicy Recipes with Milk, and it shows only (paginated) spicy recipes with milk.
• /recipes?contains=milk, title = Recipes with Milk, and it shows only (paginated) recipes with milk.
• /recipes?spicy=true, title = Spicy Recipes, and it shows only (paginated) spicy recipes.
• /recipes?spicy=true&sweet=true&contains=tomato,cilantro, title = Spicy and Sweet Recipes with Tomato and Cilantro, etc..

Say you had 5 or 10 query parameters like that, each which returns unique content (sometimes duplicated between pages, like "spicy + milk" vs. just "spicy" will return some of the same content), each page which has a unique title.

First, will Google (and others) follow these URLs and index them? Or will they not because "for some reason perhaps they might not follow or like query parameters" is my question.

Second, would nested/hyphenated URLs be better?

• /recipes/is/spicy/contains/milk
• /recipes/contains/milk
• /recipes/is/spicy
• /recipes/is/spicy/is/sweet/contains/tomato:cilantro

TBH, I like the look of the nested/hyphenated version better (it is easier to read), but it is not as simplified/generic as the query parameter approach. And I am not sure if Google (and others) like one approach better than the other (params vs. hyphen/nesting).

For the hyphenated/nested version, you would simply have to make sure the params were ordered first.

I guest a 3rd option would be flat URLs, like:

• /recipes/spicy-and-sweet-with-cilantro-and-tomato

So question is, does Google (and others) follow query parameters like this (and index the results), and if so, do they prefer query parameters over hyphenation/nesting, or vice versa, or it doesn't matter?

Answer 1

Googlebot will crawl URLs with parameters and index the resulting pages. It doesn't matter much to Google which format of URL you choose. Google will be able to index any of them.

URLs with query parameters are longer and uglier than your other variants. Users tend to like URLs that:

• Are easy to read
• Adequately describe what the page is about
• Are as short as possible

Given those criteria, I would recommend using /recipes/spicy-and-sweet-with-cilantro-and-tomato because it reads in natural English, only uses two symbols (/ and -), and is shorter than your other variants. User-friendly URLs have a small SEO advantage because usability matters to SEO and because users link to friendly URLs more often. For extended advice on how to choose great URLs, see Are keywords in URLs good SEO or needlessly redundant?

However, you have SEO problems you need to solve regardless of your URLs. Your site is going to suffer from internal duplication, thin content, and too many pages to crawl.

Parameter order matters to search engines

Search engine bots will see these two URLs as different pages, even if they have the same content:

• /recipes/spicy-and-sweet-with-cilantro-and-tomato
• /recipes/tomato-and-cilantro-with-sweet-and-spicy

You will need to create rules to tell search engines which of those you prefer. Rules like:

• Tastes come before ingredients
• Tastes are listed in alphabetical order
• Ingredients are listed in alphabetical order

URLs that are not in your preferred format should redirect the preferred URL, or use a canonical tag to point to the preferred URL.This is something that you should do regardless of what URL format you choose. See Canonical URLs for interchangeable parameters

Search engines don't index pages with duplicated content

Since recipes can be listed with different filters, search engines will probably decide not to index many of your pages. For example

• /recipes/spicy-and-sweet-with-cilantro-and-tomato
• /recipes/spicy-and-sweet-with-cilantro

may have 80% of the recipes on them in common. When that happens, search engines will typically just choose to index one of the two pages. You should strive to make sure that search engines find few results that are substantially the same as others. See What is duplicate content and how can I avoid being penalized for it on my site?

Some pages are going to have too few recipes

• /recipes/spicy-and-sweet-with-cilantro-milk-tomato-and-turnip

Is so specific that it isn't likely to have a long list of recipes. Search engines are going to view such pages as "thin content" that isn't worth indexing. Having too many thin pages on your site can your overall SEO. To combat this you should:

• Try not to link to thin pages
• Use rel=noindex for recipe lists with zero, one, or two recipes.

For further reading see Why is preventing Googlebot from crawling some pages on my site good for SEO?

Nearly infinite URL space

It sounds like you are creating a "faceted navigation" system to allow users to explore your recipe collection. This type of system will have too many URLs for search engines to crawl and index all of them. If there are 5 tastes and 100 ingredients, there are factorial(5) * factorial(100) or 10^161 (1 with 161 zeros after it) possible URLs. There is no way for a crawler to go through all of that, you will have to find ways to limit your URL space.

Even if you restrict your URLs to only two tastes and two ingredients there are 5*4*100*99 or 200,000 possible URLs. It might be possible to get that many URLs crawled and indexed if your site gets enough reputation, but a new site will take years to achieve that.

Ideally you want to limit search engine crawlers to only your most unique and well populated recipe lists while still allowing users to filter recipes any way they want. One way to approach that is to use parameter URLs except for your best recipe lists. That way you can disallow crawlers from using parameters in robots.txt with Disallow: /recipes?. Then you can have a variety of URLs and search engines will only crawl the best ones:

• Crawlable:
  • /recipes/spicy
  • /recipes/spicy-with-cilantro
  • /recipes/spicy-and-sweet-with-cilantro-and-tomato
• Not crawlable:
  • /recipes?spicy=true&sweet=true&contains=tomato,cilantro,milk,turnip

You would need to figure out how to list all of your best URLs in your sitemap and find some way to link to them on your site.

For further reading about a similar situation see SEO-friendly way to pass multiple values in a URL parameter