1

I have a verified web property on search console. It's been collecting traffic data for months now. I am using the crawl errors report to look at 404s on my site. I'm curious if this data collects traffic from bots.

I am looking at the logs on my server, and I see that a request from the user-agent 'bingbot' has hit several pages that don't exist on my site. For example, traffic from bingbot has come to www.example.com/this/is/a/404.html however, I don't see this URL come up in Search Console when I filter for it. Does Search Console not log 404s from bots? Or is this indicative of an issue I'm not aware of?

  • The only 404's that google webmaster tools sees is ones which are encountered by Googlebot. Other bots won't be detected by webmaster tools. – Chris Rutherfurd Jun 20 '16 at 2:16
0

The search console logs 404 pages as encountered by the bot

The logs that you see on the server and on the console are different. A 404 on the server log won't necessarily reflect under Crawl Errors in the console.

  • The question asks about a hit from Bingbot. How is Google going to track that? – Stephen Ostermiller Jun 15 '16 at 19:14
  • To rephrase my question: does Google Search Console only track 404s from its own crawler, or does it work similarly to Google Analytics where any traffic is examined? (my Search Console and Analytics profiles are linked) – smilebomb Jun 15 '16 at 19:50
  • 1
    @smilebomb Yes, Google Search Console only tracks 404s from its own crawler, ie. Googlebot. – MrWhite Jun 15 '16 at 22:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.