Google Webmaster Tools - Crawl errors with 500 status code

Question

I will try to explain the situation:

In December 2012 I deployed a website with a sitemap.xml, etc... there were circa 2000 URLs.

After a while I changed something on my website, upgrading my pages to circa 30,000 and I deployed again updating the sitemap.xml.

I also updated the routing of my urls: ie: from mysite/id=myName to mysite/id/my-name.

Now for some reason I'm receiving from Google Webmaster Tools 8 crawl errors with a 500 status code on some pages with an old URL... how can it be possible? And how can I fix it?

Answer 1

The problem with 500 (Internal Server) Errors are that they are generic errors when the server is unable to return a valid response and no other error is appropriate. It does not actually tell us anything about the error, other than there was an error! It could be anything.

In order to find the cause you need to examine your server error logs. And find out what was happening on your server at the time. Have these errors continued to occur?

However, your old URLs should be 301 redirecting to the new URLs, so there should no longer be any processing going on at the old URL to cause an error to begin with.

Answer 2

Google webmasters will tell you the source of the error.... if this is simply pointing to a file in a directory, you may consider preventing anyone, including Google from browsing that folder and find such files, this can be done by creating a default php file in that folder:

Step 1- create a file called: index.php
step 2- add the following code to it:

<? php

// Silence is golden.

?>

(remove spaces) If you then browse this folder again you no longer be able to see the contents of the folder, including the file which Google found.

Answer 3

In Google webmaster tool when it shows 500 status code error, it indicates that web pages have internal server error while trying to process the request. Let’s glance as to when the error appears:

1. If there is a DNS issue.

2. Once you have completed the process of 301 redirect and blocked the old web pages in robots.txt file.

3. If you have excessive page load time due to the site's hosting server getting overloaded or misconfigured.

Find out the possible ways which are responsible for showing the 500 error and solve the issue.

You can also find a complete list of various status code definition and why they showing erros from http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html