3

I'm building a single page application and I'm using a client-side routing library to handle URL/state changes.

Let's say Googlebot visits example.com/my-shop-name/menu, the menu-items for my-shop-name are loaded dynamically and a URL parameter is immediately appended to the URL like so: example.com/my-shop-name/menu?top_item=some_item_id. top_item represents the item in a list of menu-items on the page that is currently at the top of the viewport. As the user scrolls, this parameter will naturally update as the top-most item changes. The point of this parameter is to remember the scroll position. It should also be pointed out that whenever the parameter is updated in the URL, the page never reloads.

My first question is: what URL will be indexed?

  • example.com/my-shop-name/menu

or

  • example.com/my-shop-name/menu?top_item=some_item_id

Secondly, if the latter is indexed, will instructing the web crawlers to ignore top_item as a URL parameter in robots.txt make it so the first URL is indexed and the latter is not?

Improve this question

1 Answer 1

Reset to default
1

what URL will be indexed?

Since Google is able to interpret JavaScript reasonably well these days then there is a good chance that both URLs could get indexed. If the second URL (with the URL parameter) is linked to then there is a very good chance that search engines will index both.

Since the content is "loaded dynamically", the URL with the parameter doesn't just represent a different "scroll position", but it potentially contains different content as well.

will instructing the web crawlers to ignore top_item as a URL parameter in robots.txt make it so the first URL is indexed and the latter is not?

I wouldn't block the URL in robots.txt. This will prevent the URL being crawled.

However, you could inform Google to "ignore top_item as a URL parameter" in Google Search Console.

Or simply include a <link rel="canonical" element in the head section pointing to the parameterless URL.

Improve this answer
4
  • The url is only parameterless until the JS determines the top_item parameter. Since ~99% of the time a shop will have at least one menu item, the url will only ultimately be parameterless ~1% of the time. Basically, i want the parameterless url to be the only url indexed as the top_item parameter that is appended is constantly changing as the user scrolls. W/ that said, will the canonical link you suggest satisfy my requirement?
    – Vincent
    Commented Aug 30, 2017 at 11:28
  • google.com/maps is a good reference to what i'm trying to achieve. You access maps via that link but ultimately as soon as it determines your location, it appends your coordinates to the url. google.com/maps never ultimately exists on its own. It ultimately becomes google.com/maps@some_lat,some_lng , however google.com/maps should still be the sole representative url for all possible lat,lng values appended
    – Vincent
    Commented Aug 30, 2017 at 11:32
  • The canonical link is only advisory. Google might ignore it if it determines that the URLs reference different content (and are not in fact canonical). With regards to Google maps, there are many many (100K+) URLs of the form google.com/maps@some_lat,some_lng indexed. google.com/maps may only be the top result because of its authority (it's linked to the most). With Google you can specifically tell it how to handle certain URL params within GSC. Is there already a unique URL that identifies each product?
    – DocRoot
    Commented Aug 30, 2017 at 17:19
  • No there isn't a unique URL for each product.
    – Vincent
    Commented Aug 31, 2017 at 0:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.