Google can no longer fetch pages after successful URL rewrites using an .htaccess file

Question

I tried to rewrite my URLs so they no longer have the .html extension. But when I did this (successfully) Google can no longer crawl my pages when I use "Fetch as Google". Instead, it returns 404 errors and says they're unreachable.

This is what I have in my .htacess file:

Options +Includes
AddHandler server-parsed .html
RewriteEngine on
RewriteBase /

RewriteCond %{REQUEST_fileNAME} !-d
RewriteCond %{REQUEST_fileNAME} !-f
RewriteRule (.*) /$1.html [L]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /([^.]+)\.html\ HTTP
RewriteRule ^([^.]+)\.html$ http://www.example.com/$1 [R=301,L]

This works great, and if I type the URL in a browser's address bar it takes me there. However, Google is still returning 404 errors.

I think it has something to do with the above code rewriting it to www.example.com while Google is requesting http://www.example.com. I'm not sure how to fix this though.

Answer 1

I suspect you are over thinking this. I am really confused over all your code.

One of the things I am finding these days is that people are using example code that is already unnecessarily complicated. As well, people seem to select {???} that offers too much instead of the narrowest selection. Often these things only require 2 lines or 3 at the most and only one RewriteRule. Always seek the most simplest options when doing any regular expression. Otherwise you run a risk of unintended consequences.

Now I may be assuming too much. You may have been trying to do two things and my tired brain could not figure this out. Please let me know.

I tested this here and it did what you described:

RewriteCond %{REQUEST_URI} (.*)\.html [NC]
RewriteRule .* http://www.example.com/%1 [R,L]

Remove the RewriteBase, and the RewriteCond(s) and RewriteRule(s) that you are using. RewriteBase is not necessary.

If you need more, please give a comment and I can/will be happy to update the answer. If I missed what you are trying to do, please let me know.