What is the actual HTML rendered to bots?

Question

We have a product page in which 4 item carousels are present, showing mostly related products. All these 4 carousels are client side rendered.

1. Scenario1: Google rich results

Now, we have observed that none of these 4 carousels are rendered when tested in Google "Rich Results Test" tool in desktop mode.

2.Secnario2: User-Agent="Googlebot Dekstop"

Also, on another note, I tested the same URL in browser by using "User-Agent" as "Googlebot Desktop". In this scenario, 2 of the carousels(sponsored) are rendered but not the remaining two(non-sponsored). They are handled differently in code.

The sponsored carousels data is already available in SSR HTML(NEXT_DATA) are rendered based on intersection observer API on client.

Where as the non-sponsored carousels data is fetched on client after hydration and then rendered.

3. Scenario3: User-Agent="Googlebot Dekstop" and JavaScript disabled on browser

In this scenario, the HTML rendered contains none of the carousels and matching the Scenario1, that is same as HTML rendered in Google rich results test.

1. What is the actual HTML rendered to bots? is it the HTML rendered in "Google rich results Test"(Scenario1) tool or the HTML rendered in browser when "user-agent" is set as "Googlebot Desktop"(Scenario2?
2. if it's the Scenario2,
   1. The bot is able client side render the carousels that are based on intersection observer API as data readily available, but it didn't render other two carousels for which additional network call is needed. Don't bots wait for network calls after hydration and render client side components?do they have time window within which HTML need to be rendered to be consider for the crawling?
3. what is the ideal way to serve bots here? should all the carousels be present in SSR document and avoid CSR?

Answer 1

I'll try to answer all three by explaining how the crawler works:

According to Google's documentation, there is a hand off between the crawler and a renderer when Google bot crawls your site. Javascript takes a lot of resources to process and there is a cap on the amount of effort Google is willing to put in. While a small amount of Javascript being used to render your page isn't a problem - the more you add the more likely it is that some will be dropped.

How much Javascript the renderer will process depends on a few factors only known to Google. We can fairly assume they relate to the purpose of the bot how much Google wants to render your page. When you use the rich results bot - it's looking for structured data. It isn't likely to render much because it expects that data to be embedded in the HTML. When you do a Desktop Google Bot crawl, then it is more likely to assign resources to help your page reach roughly what it thinks a user would see.

I can't provide a precise answer for your case but my general advice is to keep the client side Javascript load to a minimum where possible. This is also the advice given by Google. I've seen more success with popular libraries and frameworks - but even in the case of heavy SPAs I strongly suggest limiting or pre-rendering critical HTML/CSS. And always embed your JSON+ld data when it's served to ensure it's picked up!