1

Google Crawler generates 520 errors daily. I will get about 50-100 of these errors in a 60.000 internal links website.

The thing is though, that my links don't have any issues that I can dig into resolving. Therefore when I click on an URL error and use the Fetch as Google functionality the URL is crawled and indexed normally without any issues.

Any clues where to start the troubleshooting of this one?

Thanks

1

I found this article on cloudflare.com that may help:

https://support.cloudflare.com/hc/en-us/articles/200171936-Error-520-Web-server-is-returning-an-unknown-error

Also, what I'd suggest is to temporarily take your website away from a 3rd party caching service (such as cloudflare) and try running it for a week like that and observe your server logs to see if anything unusual is going on. Also, test it a few times in webpagetest.org to see if the loading times are acceptable.

Also, get a tool that scans every link on your site to make sure they are all valid. I use Xenu Link sleuth (found at http://home.snafu.de/tilman/xenulink.html#Download).

Once things are acceptable, then try cloudflare again and if you still get HTTP error 520, then there is something wrong with either their service and/or with your account on their service in which case you should contact them directly about the issue.

  • Thanks for the tips Mike. I have a look into the article and for now I opened a ticket to cloudflare. I'll see how it goes there. – pierostz Oct 28 '17 at 11:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.