8

There are (at least) two common reasons why strange and mangled URLs may show up as crawl errors in Webmaster Tools. The first possibility is that someone has copied your pages (or some other pages that link to yours) and mangled the links in the process. This happens more often than you might think; see e.g. the sixth question in this Google Webmaster ...


5

Why don't you try "Alternative methods" to verify your website, like upload the generated HTML file ?


4

Yes, send an email for every error. It's also a good idea to send as much info as you can get as part of the email. Think about grabbing: the contents of the form collection, all session variables, the browser user-agent string, cookie values, the page URL, including any querystring parameters. This helps greatly for debugging purposes. It's trivial to get ...


4

Finally realised what the problem was when, after deleting the tag in both files, the error "Invalid command '\xef\xbb\xbf EF BB BF = BOM Moral of the story: Always check your encoding* and hex dump of your files when you get weird issues. *On Notepad++ you can use the Encoding -> Encode in UTF-8 without BOM option to remove it


4

It is my experience that Google generally does nothing for a previously indexed page when it sees a 5xx error. It may drop it from the SERPs temporarily if the 5xx error exists for too long. The page is not de-indexed. The reason is simple. 5xx errors are seen as temporary and likely to be fixed once discovered. I would focus on why your site is going ...


3

I've been able to reset a lot of my site's values using appcmd, but I had to dig into specific configurations I wanted to change, which I'll describe a bit below. I know for sure that this works with IIS 7, but I'm not 100% certain whether it passed on to 7.5 so YMMV. appcmd isn't in my cmd path on any of my installations (though these machines are 2k8sp2) -...


3

Just because your site appears to display fine, doesn't mean it is returning a 200 status code. You can technically show any page with any 4xx or 5xx status code. However, I checked your site with web-sniffer.net and it returns a 200 status code. You are supposedly verifying your site using Google Analytics, but the error message suggests that you are ...


3

You can use Fetch as Googlebot to see how the Googlebot sees your site to try and track down the problem.


3

If your initial support is expired and you want to try debugging the issue yourself, I would first check the web server's error logs. This is the first place you should look whenever you encounter a 500 - Internal server error. If you're using Apache, this is typically located in /var/log/httpd/error_log (though for vhosts a separate location is likely to be ...


3

Have you had them do an IISReset yet? Don't ask questions. If you don't have a reliable history with this host, then you really can't trust anything they say. If you cannot even do echo(); or phpinfo();, then PHP is not running on the server. phpinfo(); has been around for ages. Also, make sure that your file extension is '.php'... I've made that mistake ...


3

So I finally reached out and contacted GoDaddy support. After an hour, we determined there might be an error in my code. And after a little bit of digging and testing with some basic files, he was right. I added the following to my index.php file... error_reporting(E_ALL); ini_set('display_errors', 1); AND LOW AND BEHOLD: Fatal error: Uncaught Error: ...


3

The screenshots indicate you are looking at the http and non www version of your site. The home page inspection report indicates that the home page for that site (http without www) is redirecting to the https with www version . https://www. and that it is not indexed because of that. That's from the "page with redirect" statement and what Google says it has ...


3

For SEO, the title and the meta tags of a 5xx page don't matter. Search engines bots see the HTTP status code and base their decisions only on that. They will ignore titles, meta descriptions, and noindex. For the most part, search engines do what you want. They never index content from the error page itself. For most error statuses, they retry the ...


2

I use this: Options +FollowSymLinks RewriteEngine On #RewriteBase / # check if the file don't exists then redirect RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)\.html$ $1.php [NC,L] and it works fine, the $a should be $1. RewriteEngine On Enables the rewrite engine IF mod_rewrite is enabled. RewriteBase / Executes all the rules from the root (/) ...


2

If you can create a .htaccess file (or already have one), put this code at the top: # enable PHP error logging php_flag log_errors on php_value error_log /path/to/htdocs/PHP_errors.log Create a blank PHP_errors.log file in the directory specified in your .htaccess file and set its permissions to 777. Now refresh the page that's giving the 500 error and ...


2

Make sure your Config.php file iS WRITEABLE! CHMOD to 777 Joomla is fairly easy to install and should be supported on 99.999% of all shared hosts.


2

Also ask and check for directory/file permissions in file manager.


2

Have you tried adding the paths to your Robots.txt so that GoogleBot and others don't attempt to crawl them? I've noticed that GoogleBot in particular is very "good" at finding what it thinks are paths and links in script string literals and attempting to follow them - which can lead to exactly this sort of behaviour. If there's nothing legitimate about ...


2

Google likes to know if your pages are having trouble. In doing so it can keep up-to-date information of your pages. If you send back 500 error, Google + your users will know you are having problems and come back later. I suggest keeping the 500 response fix up the issue.


2

This is my python script, it will scrape though google cache and download the content of your webiste, and it can run without trouble with 503 504 404 error (Google blocks IP that send many request): https://gist.github.com/3787790


2

Not guaranteed this is it, but try removing a line at a time until the error stops. That'll help you see which particular bit is causing the problem. Getting a 500 from htaccess is sometimes a sign you're trying to do something you're not allowed to. Is mod_rewrite actually enabled, for example? The same for the Options line; If the server isn't configured ...


2

You shouldn't have a problem using your working directory as the web root folder since apache2 and subversion are two separate services running each can use the same folder. I normally use rsync to copy from the trunk to web root. Check your permissions and if there's an htaccess file in the WD


2

Server errors are just that - errors that occurred with your web hosting server. You can Google error codes for an idea of what's going on, for example: Error 500 Since you're able to reach the domain, it's not an issue for the domain registrar - contact your web hosting company's support team for help. If you have access to your web hosting server logs (...


2

Definitely leave the URL as it as it is typed into the browser. That way search engines, bots, users, etc. will know that that particular URL is not a valid URL. If you were to redirect to another URL, it's possible that bots, etc. could still index the bad URL, which is not what you want.


2

Sounds like you have not taken into consideration that Google crawls sites all across the globe but primary these crawls do occur more often in .US data centers. You could override this by using a user agent detect script that allows the Googlebot regardless of its GEO location, however you should also take into note that Google doesn't view this solution as ...


2

I would try two things - Check permissions of 500.html (try setting it to 777 to be very sure - modify it later). Try 500.htm (or 500.txt) instead of 500.html (just to be sure that your other rules in htaccess are not messing up with the ErrorDocument 500.html page). Also, remember to change the htaccess ErrorDocument rule to 500.htm (or 500.txt). Also, ...


2

but when I encounter a 500 Internal Server Error, it gives the default Internal Server Error The problem is that custom 500 error documents simply don't get triggered for errors in .htaccess - which is what's happening here. As Aakash has already quoted, this may come under the realm of a "malformed request". If you check your error log it should state: "...


2

There is no SEO Risk from temporary errors on your site. Google is very forgiving of small amounts of down time. Down time lasting less than a day will generally not hurt your site from an SEO perspective. When Googlebot encounters a 404 or 500 error on a page that it has found in the past, it tries again for 24 hours before removing the page from the ...


2

I'm guessing that your URL change is unrelated to the error. A 500x error usually means a server side processing error, like unable to render the page or there's bug in the server side code on the page (PHP, ASPX, etc). 400x errors typically mean the page doesn't exist. Since Google is showing you a 500 error, are you sure it's because the file doesn't ...


2

It is always best to show errors at the original URL rather than redirecting. For SEO, Google knows how to deal with 500 errors. It will give the page some grace time (probably 24 hours) and if the error doesn't go away it will stop indexing it. Google will probably treat a redirect to a 500 error the same way, but there is no guarantee. In general, ...


Only top voted, non community-wiki answers of a minimum length are eligible