5

I've implemented a simple mod-rewrite rule that internally rewrites urls without a file extension (that are not directories) to .php files and removed the .php extension from all our internal links. I've also updated the sitemap with the extensionless urls. However, the urls in Google's index obviously still have the extension.

  1. Will Google crawl the site or download the sitemap and update its index automatically with the extensionless urls?
  2. Should I be implementing a 301 redirect from .php to extensionless urls? At the moment, both url's will resolve to the same page which could be seen as duplicate content.
  3. Should I get a list of all the urls in Google's index which have extensions and submit a request to have them deleted via Webmaster Tools?
| improve this question | |
12
  1. Yes, if you have your URLs redirected properly.
  2. Yes. You should be using a 301 redirect instead of internally rewriting the URLs.
  3. No, this is completely unnecessary. If your old URLs are 301 redirected to the new URLs, then Google will know that those resources have been permanently renamed/moved. That's the whole point of having a 301 redirect code (as opposed to using the other 3xx response codes).
| improve this answer | |
  • Agreed on all points. +1 – Oldskool Jan 21 '12 at 21:40
2

I recommend using 301 Moved Permanently redirect. Until then, if you're concerned that Google may categorize the pages as duplicate content then providing a sitemap or adding the rel=canonical link element into the head of each document should resolve the issue. If it is not possible to add the rel=canonical markup to the document, e.g. PDF, the recommendation is to append a Link: in the HTTP headers.

LINK Syntax Example:

<link rel="canonical" href="http://www.example.com/product"/>

HTTP Header Syntax Example:

Link: http://www.example.com/product; rel="canonical"
| improve this answer | |
  • 1
    I'd never heard of the link header before. That's good to know. However, rel=canonical was created to canonicalize duplicate content. Google prefers that webmasters eliminate duplicate content whenever possible by using a redirect (or just not creating the duplicate URL in the first place), but rel=canonical can be used as a fallback if a 301 redirect isn't possible (e.g. the webmaster doesn't have access to .htaccess or when presenting alternate views). You wouldn't use it if 301 is already in use. – Lèse majesté Jan 23 '12 at 9:32
  • I agree. After posting this answer, I read the question and your answer again and realized that my answer was out of scope for this question. I'm happy to delete my answer in order to prevent confusion. – Melioratus Jan 24 '12 at 6:18
  • It might be a corner case, but it is possible for a webmaster to be able to rewrite a URL but be unable to redirect in a scenario similar to this question. Regardless, at least one other person found your answer useful. Perhaps you could simply reword it so that it doesn't sound like Google is suggesting webmasters use rel=canonical on top of a 301 redirect. – Lèse majesté Jan 24 '12 at 10:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.