3

I'm building a single page application and I'm using a client-side routing library to handle URL/state changes.

Let's say Googlebot visits example.com/my-shop-name/menu, the menu-items for my-shop-name are loaded dynamically and a URL parameter is immediately appended to the URL like so: example.com/my-shop-name/menu?top_item=some_item_id. top_item represents the item in a list of menu-items on the page that is currently at the top of the viewport. As the user scrolls, this parameter will naturally update as the top-most item changes. The point of this parameter is to remember the scroll position. It should also be pointed out that whenever the parameter is updated in the URL, the page never reloads.

My first question is: what URL will be indexed?

  • example.com/my-shop-name/menu

or

  • example.com/my-shop-name/menu?top_item=some_item_id

Secondly, if the latter is indexed, will instructing the web crawlers to ignore top_item as a URL parameter in robots.txt make it so the first URL is indexed and the latter is not?

1

what URL will be indexed?

Since Google is able to interpret JavaScript reasonably well these days then there is a good chance that both URLs could get indexed. If the second URL (with the URL parameter) is linked to then there is a very good chance that search engines will index both.

Since the content is "loaded dynamically", the URL with the parameter doesn't just represent a different "scroll position", but it potentially contains different content as well.

will instructing the web crawlers to ignore top_item as a URL parameter in robots.txt make it so the first URL is indexed and the latter is not?

I wouldn't block the URL in robots.txt. This will prevent the URL being crawled.

However, you could inform Google to "ignore top_item as a URL parameter" in Google Search Console.

Or simply include a <link rel="canonical" element in the head section pointing to the parameterless URL.

  • The url is only parameterless until the JS determines the top_item parameter. Since ~99% of the time a shop will have at least one menu item, the url will only ultimately be parameterless ~1% of the time. Basically, i want the parameterless url to be the only url indexed as the top_item parameter that is appended is constantly changing as the user scrolls. W/ that said, will the canonical link you suggest satisfy my requirement? – Vincent Aug 30 '17 at 11:28
  • google.com/maps is a good reference to what i'm trying to achieve. You access maps via that link but ultimately as soon as it determines your location, it appends your coordinates to the url. google.com/maps never ultimately exists on its own. It ultimately becomes google.com/maps@some_lat,some_lng , however google.com/maps should still be the sole representative url for all possible lat,lng values appended – Vincent Aug 30 '17 at 11:32
  • The canonical link is only advisory. Google might ignore it if it determines that the URLs reference different content (and are not in fact canonical). With regards to Google maps, there are many many (100K+) URLs of the form google.com/maps@some_lat,some_lng indexed. google.com/maps may only be the top result because of its authority (it's linked to the most). With Google you can specifically tell it how to handle certain URL params within GSC. Is there already a unique URL that identifies each product? – DocRoot Aug 30 '17 at 17:19
  • No there isn't a unique URL for each product. – Vincent Aug 31 '17 at 0:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.