2

I have the following UI:

enter image description here

There are tabs which are <button>'s and when you select one there will be <a>-Links to different pages. It seems like Google will only index the Links on the first page (which is open by default) and not click the other buttons to the following links. The UI is built with react so only the links which are currently visible are in the HTML.

How can I achieve that Googlebot will also index the links of the other pages? Is it possible to tell the Googlebot to click a button? Any experiences?

Improve this question

2 Answers 2

Reset to default
3

Googlebot does not simulate any user interaction with the page. It doesn't click, scroll, or type. There is no way to make it do so.

What Googlebot does instead is to scan the document object model (DOM) and look for links to follow. That leads to two potential solutions to your problem:

Put all the tabs in the DOM

If all the tabs' content are in the document, Googlebot will follow all the links, even if some of the tabs are currently hidden. Rather than implement the tabs such that the content gets loaded via AJAX when the user clicks, you can implement them so that all the content for all the tabs is loaded at page load. Then when the user clicks, the current tab can be hidden and the other tab shown.

Convert the buttons to links

Alternately, you can assign a different URL to each tab and convert each tab to link rather than using a button. Then Googlebot will discover each tab as its own page to crawl. When it crawls each tab, it will see the content and be able to crawl all the links.

Improve this answer
3
  • Thank you for your ideas! I now have all my information I wanted to know :) Another solution I came up today is to create another "view" when the "Googlebot" visits the page.
    – Lars Flieger
    Aug 2, 2022 at 11:42
  • 1
    Creating a separate experience for Googlebot is known as cloaking and it is against Google's webmaster guidelines. Doing so could even get your entire site penalized such that it doesn't show up in Google at all.
    – Stephen Ostermiller ♦
    Aug 2, 2022 at 12:18
  • Thank you! Good to know :D
    – Lars Flieger
    Aug 2, 2022 at 12:22
1

There are many ways to do this, you should do as many as you can. For example -

  1. Put the url of the pages that the buttons generate, them in your sitemap.xml
  2. Link One page to another at the bottom with something like See Also
  3. Have a sitemap page and put all the links there.
Improve this answer
2
  • 1
    Sitemaps are not great at getting Googlebot to index pages that don't have crawlable links. Then even if those pages are indexed, they don't get ranked well. See The Sitemap Paradox. It is almost always better to try to get Googlebot to see all the links you have rather than trying to rely on sitemaps to replace the links.
    – Stephen Ostermiller ♦
    Aug 1, 2022 at 15:22
  • @StephenOstermiller - True, that's why I am suggesting as many ways as possible to present the links. And the more internal links there are, the better.
    – Rohit Gupta
    Aug 1, 2022 at 22:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.