1

We are using static html, combined with partial page hydration via javascript. Visitors download a full html page when they first arrive at the website, from there javascript intercepts any internal link clicks, fetching a json file with the content that changes and then hydrates the existing DOM with the new content.

However our schema data is included in the page as ld+json, so not contained within the content that gets hydrated.

I am wondering, when Googlebot and Bingbot visit a webpage such as ours, do they look at the html and simply fetch the url listed in links getting the full static version? Or do they simulate a link click in which case the js would preform the page hydration which they would sub-sequentially crawl?

I ask because we have not yet figured out how to extract the content within <script type="application/ld+json">.....</script> with PHP and then later reinsert it with javascript. If the bots simply go to the next url, then they will always get the correct schema data, but with page hydration we would need to update the schema data.

1 Answer 1

1

Search engine bots that execute JavaScript don't click, scroll the page, move the mouse, or simulate any any type of user interaction with the page. They only execute JavaScript that is runs on page load. If the JavaScript doesn't run within the first few seconds of the page loading or if it requires user interaction, search engine bots will never execute it.

Search engine bots:

  1. Download the initial page.
  2. Download page resources such as images, CSS, and JavaScript.
  3. Spend several seconds letting the initial on-load JavaScript execute.
  4. Scan the document object model (DOM) of the page for text and links.
  5. Index the words from the page.
  6. Add the links found in the DOM to the crawl queue.

Your approach of serving full static pages, and intercepting user clicks is a great approach for SEO with JavaScript powered sites. Google and Bing will see the static meta data for each page and users will get the experience benefits of not having the entire page have to re-load when they click on links.

On the flip side, if you omit <a href= links in your document and have users click on <div>s or other elements to do navigation, search engines won't be able to crawl your full site.

3
  • Thanks. All of our internal links have <a href= links as we want the website to work even for folks who have disabled JS.
    – MrCycling
    Commented May 6, 2021 at 11:32
  • If you have a href links then the search engines will add them to the crawl queue. Later they will also be crawled via the same process outlined here. So each url needs to include its own structured data is it is loaded. Commented May 6, 2021 at 23:14
  • @TonyMcCreath, yes each webpage already has a complete static html version, with schema data, plus a matching json file with only content, no schema. Initial requests to the website always get the complete version with schema.
    – MrCycling
    Commented May 7, 2021 at 2:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.