Does Googlebot call every link separately (with a new request)?

Question

Does Googlebot call every link separately (with a new request) or does it follow all links by JavaScript (for example by click event)?

I have a vue-js website which contains all the links and of course all the navigation is handled by JavaScript.

But the meta tags and open graph meta tags are only generated and output directly from the server via PHP when a URL is requested. For example, when I go to my main page example.com/index, my PHP code outputs the meta tags specifically to that URL. If a visitor of the website now click on a link "contact" (example.com/contact), the meta tags are not adjusted and are still those for example.com/index.

But if I now directly open the URL example.com/contact in my browser, the correct meta tags and open meta tags are displayed by the PHP script.

Does Googlebot crawl each link individually as a new request/URL call or does the bot go to example.com/index and then execute the link calls via Javascript on the website (in this case the meta tags would be wrong and not updated).

Answer 1

Googlebot will look for any real links on the rendered page. i.e. an tag, and store the links URL for later crawling. So it crawls every link separately. When crawling it does not click on things.

Answer 2

Googlebot first looks at your robots.txt before doing any type of crawling activity. This file gives it instructions on what to visit and not visit.

Googlebot then crawls the page and adding all URLs found to the crawling queue. When resources are available, Google will then render and process those URLs.

Every request is a new request.

Answer 3

Googlebot views each page with a fresh request. It does not use onclick events to load new content into the same page.

In fact, even though Googlebot executes JavaScript, it does not simulate any user interaction with the page. The only JavaScript that gets run, are the scripts that run when the page loads. Googlebot does not click, scroll, type, or move the mouse. Googlebot will miss any content that doesn't show up when the page is loaded.

After the page loads and all the onload JavaScript runs, Googlebot scans the document object model (DOM) for <a href=...> links. It records those URLs and will crawl them later. Googlebot might crawl them within a few minutes, but it will always use a new page request for each URL.

Googlebot may not find URLs that are not specified through <a href=...> links in the DOM. Even if you intercept clicks on those links and prevent whole new pages for loading for users, Google needs to see the links in the document. Using click events on <div>s or other HTML elements for navigation makes your site uncrawlable so that it won't get indexed in Google.

Google allows JavaScript to change meta tags. So if you were to implement the meta tags with JavaScript during the page load, Google would see them. Setting them server side allows other search engines and social media bots to see them as well.