Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

I tried to rewrite my URLs so they no longer have the .html extension. But when I did this (successfully) Google can no longer crawl my pages when I use "Fetch as Google". Instead, it returns 404 errors and says they're unreachable.

This is what I have in my .htacess file:

Options +Includes
AddHandler server-parsed .html
RewriteEngine on
RewriteBase /

RewriteCond %{REQUEST_fileNAME} !-d
RewriteCond %{REQUEST_fileNAME} !-f
RewriteRule (.*) /$1.html [L]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /([^.]+)\.html\ HTTP
RewriteRule ^([^.]+)\.html$ http://www.example.com/$1 [R=301,L]

This works great, and if I type the URL in a browser's address bar it takes me there. However, Google is still returning 404 errors.

I think it has something to do with the above code rewriting it to www.example.com while Google is requesting http://www.example.com. I'm not sure how to fix this though.

share|improve this question
    
I am confused by the last sentence. Can you clarify it for me?? –  closetnoc Jul 20 at 2:00
    
Yah sorry, in google webmaster tools the url that it fetches is "http:// www.example.com/whatever" but my redirect rules seem to redirect everything to "www.example.com/whatever" ignoring the "http://". Im not sure but I think that is proving problematic for the fetch as google. –  user2070057 Jul 20 at 2:28

1 Answer 1

up vote 2 down vote accepted

I suspect you are over thinking this. I am really confused over all your code.

One of the things I am finding these days is that people are using example code that is already unnecessarily complicated. As well, people seem to select {???} that offers too much instead of the narrowest selection. Often these things only require 2 lines or 3 at the most and only one RewriteRule. Always seek the most simplest options when doing any regular expression. Otherwise you run a risk of unintended consequences.

Now I may be assuming too much. You may have been trying to do two things and my tired brain could not figure this out. Please let me know.

I tested this here and it did what you described:

RewriteCond %{REQUEST_URI} (.*)\.html [NC]
RewriteRule .* http://www.example.com/%1 [R,L]

Remove the RewriteBase, and the RewriteCond(s) and RewriteRule(s) that you are using. RewriteBase is not necessary.

If you need more, please give a comment and I can/will be happy to update the answer. If I missed what you are trying to do, please let me know.

share|improve this answer
    
Forgot to say this worked perfectly, as you suspected I was using example code that was intended to accomplish the same thing and as far as I could tell was, except when googlebot got to it. –  user2070057 Jul 22 at 19:09
    
@user2070057 Thanks for the feed back! There seems to be a new wave/trend in the past year or so where a lot of example code is posted on-line that is overly complicated. It is not only confused, but confusing. When it comes to regular expressions, and I code using regex nearly everyday, simplicity is king. The simpler it can be made the more likely it will be reliable and not have exceptions that escape the expression. –  closetnoc Jul 22 at 21:35

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.