How to find out the referrer of Googlebot's crawling URL?

Question

Googlebot crawls 100s of 404 URLs from my website. I want to know from where it gets those links?

Is there anything like HTTP Referrer?

In detail: Googlebot crawls

• example.com/no_such_files.php
• example.com/not-there/no_such_files.php

I want to know from where Googlebot got these links? I recently see lots of URLs like the above.

Answer 1

Googlebot doesn't send an HTTP Referer request header when it crawls your site.

However, the 404 report within Google Search Console should tell you from where these URLs are linked from:

1. Crawl > Crawl Errors > Desktop | Smartphone
2. Select the "Not found" sub tab.
3. The URLs that generate 404s should be listed at the bottom of the page.
4. Click on any one of these URLs to get a popup of more details of the error.
5. The "Linked from" tab should contain all the URLs that link to the 404 URL.

(Don't "Mark as fixed", since these are genuine 404s that can't be "fixed".)