3

We have a product page in which 4 item carousels are present, showing mostly related products. All these 4 carousels are client side rendered.

1. Scenario1: Google rich results

Now, we have observed that none of these 4 carousels are rendered when tested in Google "Rich Results Test" tool in desktop mode.

2.Secnario2: User-Agent="Googlebot Dekstop"

Also, on another note, I tested the same URL in browser by using "User-Agent" as "Googlebot Desktop". In this scenario, 2 of the carousels(sponsored) are rendered but not the remaining two(non-sponsored). They are handled differently in code.

The sponsored carousels data is already available in SSR HTML(NEXT_DATA) are rendered based on intersection observer API on client.

Where as the non-sponsored carousels data is fetched on client after hydration and then rendered.

3. Scenario3: User-Agent="Googlebot Dekstop" and JavaScript disabled on browser

In this scenario, the HTML rendered contains none of the carousels and matching the Scenario1, that is same as HTML rendered in Google rich results test.

  1. What is the actual HTML rendered to bots? is it the HTML rendered in "Google rich results Test"(Scenario1) tool or the HTML rendered in browser when "user-agent" is set as "Googlebot Desktop"(Scenario2?
  2. if it's the Scenario2,
    1. The bot is able client side render the carousels that are based on intersection observer API as data readily available, but it didn't render other two carousels for which additional network call is needed. Don't bots wait for network calls after hydration and render client side components?do they have time window within which HTML need to be rendered to be consider for the crawling?
  3. what is the ideal way to serve bots here? should all the carousels be present in SSR document and avoid CSR?
2
  • 1
    Your third question I believe answers itself. The inconsistency you've noticed in rendering is likely due to the inconsistency of Googlebot's JS rendering. Having all of the carousels ready to go beforehand should solve your problem. Commented Dec 27, 2022 at 14:26
  • okay, thanks for the response. So maybe instead of complaining why scenario1 and scenario2 are different i should just render all on SSR ..Got it. Commented Dec 28, 2022 at 5:47

1 Answer 1

2

I'll try to answer all three by explaining how the crawler works:

According to Google's documentation, there is a hand off between the crawler and a renderer when Google bot crawls your site. Javascript takes a lot of resources to process and there is a cap on the amount of effort Google is willing to put in. While a small amount of Javascript being used to render your page isn't a problem - the more you add the more likely it is that some will be dropped.

How much Javascript the renderer will process depends on a few factors only known to Google. We can fairly assume they relate to the purpose of the bot how much Google wants to render your page. When you use the rich results bot - it's looking for structured data. It isn't likely to render much because it expects that data to be embedded in the HTML. When you do a Desktop Google Bot crawl, then it is more likely to assign resources to help your page reach roughly what it thinks a user would see.

I can't provide a precise answer for your case but my general advice is to keep the client side Javascript load to a minimum where possible. This is also the advice given by Google. I've seen more success with popular libraries and frameworks - but even in the case of heavy SPAs I strongly suggest limiting or pre-rendering critical HTML/CSS. And always embed your JSON+ld data when it's served to ensure it's picked up!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.