2

I have been reviewing 404 error logs on a WooCommerce site I have been working on (logs held within the website server) and there are a lot of requests to product pages which have /false on the end of the url. For example,

  • Requested https://www.example.com/product/stretchy-bracelet-crackled-quartz/false
  • Valid URL https://www.example.com/product/stretchy-bracelet-crackled-quartz/.

There are no links within the site which link to these .../false addresses, yet some of the requests seem to be from GoogleBot according to whois searches on the requester's ip addresses such as https://www.google.com/search?q=whois+66.249.75.45.

There is no referer information in the logs and the website has just been put on Google Search Console. I am now waiting for the console to finish processing its data which may take a while as there are a lot of products pages it will be looking at whilst crawling.

Can anyone shed any light on a possible cause for this phenomenon?

Improve this question
4
  • Can you find the URLs in the Google Search Console 404 error report? If so, Google often says where it found the URL.
    – Stephen Ostermiller ♦
    Commented Oct 3, 2019 at 14:37
  • Do any of the requests send a referer header? Googlebot won't but maybe some other user agent is giving you a clue.
    – Stephen Ostermiller ♦
    Commented Oct 3, 2019 at 14:38
  • @StephenOstermiller - There is no referer information and the website is not on Google Search Console as of yet. Going to put the site on there to see what it comes up with
    – Chris Rogers
    Commented Oct 3, 2019 at 14:50
  • That could take a few weeks until Google starts giving you useful information.
    – Stephen Ostermiller ♦
    Commented Oct 3, 2019 at 15:07

1 Answer 1

Reset to default
0

My guess is that Googlebot isn't following links to these URLs. Rather Googlebot is evaluating JavaScript and using heuristics to determine possible URLs. I've seen Googlebot do this on my own sites and it causes annoying 404 errors. Things like this in your JavaScript look enough like URLs that Googlebot will try them:

  • s = 'with/slash'
  • s = 'ends-in.html'
  • s = './false'

Google may actually be getting the false from the result of evaluating an expression or calling a function.

In any case, it isn't anything to worry too much about. If you can find and change the JavaScript, it would stop Googlebot from crawling the weird URLs. However, even if Googlebot continues to crawl them, it won't hurt your site or your SEO. Google's John Mueller says that 404 errors don't hurt your site in any way.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.