4

Lets say I have a site with a page of 20 products which are fed through an API. The products are loaded using infinity scroll so the initial page render has 5 products tops. Upon scrolling it adds 5 then 5 and so on.

JSON-LD is used to pass information to the search engines/crawlers. So instead of creating a JSON-LD of 5 products that the search engine will find its HTML, I create a JSON-LD of 20 products...with links to the products.

code snippet

<script type="application/ld+json">
 {
  "@context": "https://schema.org",
  "@type": "Product",
  "description": "Blah blah blah....",
  "name": "A product",
  "image": "/images/product.jpg",
  "offers": {
   "@type": "Offer",
   "availability": "https://schema.org/InStock",
   "price": "20",
   "priceCurrency": "USD"
  },
 "url": "/products/product-name"
}
</script>
.....

Will google follow the links of the products and index? Is it allowed to have a JSON-LD of products for which the crawler cannot find their HTML?

How google says they use JSON-LD from the google developers documentation:

Google uses structured data that it finds on the web to understand the content of the page, as well as to gather information about the web and the world in general. see here

1 Answer 1

4

Google will almost certainly find links in JSON/LD and will crawl those pages. Regardless of how Google uses JSON/LD, Googlebot has long had heuristics to find and crawl URLs embedded in JavaScript.

I wouldn't count on Google indexing a page whose only reference is a URL embedded in JSON/LD. It isn't clear if Google passes PageRank in URLs that it finds outside of normal HTML links. In any case, you should create alternate link paths to get to each and every one of your products. I usually suggest linking every product to several other products. That is usually done with a "Related Products" type of section in the product page. Then Googlebot can easily find multiple links into each and every product and will index the most product pages.

5
  • My case, they are so many products sometimes they reach 200+ having related links just hogs the DOM, web performance drastically reduces....tried whitelisting only google bot crawler user agents to load all products while users were served five five products. When you run page speed insights it just crashes...you get oops sorry...blah blah blah.
    – xaander1
    Sep 21, 2021 at 11:25
  • You wouldn't put all 200 into each related list. Each product would link to at most 5 or 10 others. Sep 21, 2021 at 11:35
  • I somewhat wanted a way for all products to be discovered and indexed automatically instead of doing a manual API crawling and providing a sitemap. 5 to 10 means i all products wont be discovered.
    – xaander1
    Sep 21, 2021 at 20:12
  • Each product page would link to 5 to 10 other product pages but each list of related products would be different. Every product page would link out to 10 other pages and get 10 links from some other pages. Sep 21, 2021 at 20:56
  • Another way of getting everything indexed it works decently well is creating categories and tags with pages for each of those so that there are more than like 20 products in any category. Sep 21, 2021 at 20:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.