Webmasters Stack Exchange is a question and answer site for webmasters. It only takes a minute to sign up.
Sign up to join this community
Google Crawler generates 520 errors daily. I will get about 50-100 of these errors in a 60.000 internal links website.
The thing is though, that my links don't have any issues that I can dig into resolving. Therefore when I click on an URL error and use the Fetch as Google functionality the URL is crawled and indexed normally without any issues.
Any clues where to start the troubleshooting of this one?
Thanks
I found this article on cloudflare.com that may help:
Also, what I'd suggest is to temporarily take your website away from a 3rd party caching service (such as cloudflare) and try running it for a week like that and observe your server logs to see if anything unusual is going on. Also, test it a few times in webpagetest.org to see if the loading times are acceptable.
Also, get a tool that scans every link on your site to make sure they are all valid. I use Xenu Link sleuth (found at http://home.snafu.de/tilman/xenulink.html#Download).
Once things are acceptable, then try cloudflare again and if you still get HTTP error 520, then there is something wrong with either their service and/or with your account on their service in which case you should contact them directly about the issue.
