2

I prefer clean URLs... so my internal links lack the .html or .php extension, and I'm omitting the www. subdomain. The links are resolved by .htaccess rules, and the browser shows example.com/my-page. This makes me smile.

For consistency, I entered the clean URLs as canonical links and in the sitemap (as https://example.com/my-page) that I submitted to Google. My hope was to see these clean URLs used everywhere consistently.

Unfortunately, I now see a mess in Search Console. Some indexed pages still have .html. Some still have www. Some pages were not indexed.

I'm attempting to clean up the mess by resubmitting individual URLs, but I have dozens to fix. It seems each link has to be submitted separately. It takes about 30 seconds to submit each one, and I hit my submission cap each day. This will take forever.

There must be a better way to clean up the mess. Any suggestions to make it easier?

Also, did I make the wrong choice with the canonicals and sitemap? If so, what should I have done?

Here's what I have in .htaccess:

Options +FollowSymLinks
RewriteEngine on

RewriteCond %{HTTPS} off [OR]
RewriteCond %{HTTP_HOST} ^www\. [NC]
RewriteCond %{HTTP_HOST} ^(www\.)?(.+)$
RewriteRule (.*) https://%2%{REQUEST_URI} [NE,L,R=301]

# if x.php is a file, add .php to x
RewriteCond %{REQUEST_FILENAME}\.php -f
RewriteRule !.*\.php$ %{REQUEST_URI}.php [NC,QSA,L]

# if x.html is a file, add .html to x
RewriteCond %{REQUEST_FILENAME}\.html -f
RewriteRule !.*\\.html$ %{REQUEST_URI}.html [NC,QSA,L]

# if xindex.html is a file, add index.html to x
RewriteCond %{REQUEST_FILENAME}\index.html -f
RewriteRule !.*index\.html$ %{REQUEST_URI}index.html [NC,QSA,L]

3
  • What does "links are resolved" mean? Do you mean that you are using redirects for non-canonical URLs in .htaccess? Are they "301 Permanent" redirects? Have you updated your HTML and PHP so that you only link to your canonical URLs? Jun 29, 2022 at 8:32
  • How long ago did you make these changes? It usually takes Google a couple weeks to react to changes to your URLs. Jun 29, 2022 at 8:33
  • @StephenOstermiller - Thanks for responding. By "links are resolved," I meant that rules will match my-page with my-page.html or my-page.php if they exist. So the user sees the clean URL but the page is served. I have shared the redirect rules above to help clarify. The pages only link to the canonical URLs. Some of the sites have submissions that are weeks old. Most are more recent. Reviewing Search Console, Google doesn't seem to be changing any of their indexing preferences over time. Jul 1, 2022 at 17:39

2 Answers 2

2

Here is the solution I came up with after researching for many hours...

Options +FollowSymLinks -MultiViews
RewriteEngine On
RewriteBase /

RewriteCond %{HTTPS} off [OR]
RewriteCond %{HTTP_HOST} ^www\. [NC]
RewriteCond %{HTTP_HOST} ^(www\.)?(.+)$
RewriteRule (.*) https://%2%{REQUEST_URI} [R=301,NE,QSA]

# 301 external redirect all .php extensioned requests to extensionless
RewriteCond %{THE_REQUEST} ^[A-Z]{3,}\s([^.]+)\.php [NC]
RewriteRule ^ %1 [R=301,NC,NE,QSA,L]

# 301 external redirect all .html extensioned requests to extensionless
RewriteCond %{THE_REQUEST} ^[A-Z]{3,}\s([^.]+)\.html [NC]
RewriteRule ^ %1 [R=301,NC,NE,QSA,L]

# if x.php is a file, internally add .php to x
RewriteCond %{REQUEST_FILENAME}\.php -f
RewriteRule !.*\.php$ %{REQUEST_URI}.php [NC,QSA,L]

# if x.html is a file, internally add .html to x
RewriteCond %{REQUEST_FILENAME}\.html -f
RewriteRule !.*\.html$ %{REQUEST_URI}.html [NC,QSA,L]

# if xindex.html is a file, internally add index.html to x
RewriteCond %{REQUEST_FILENAME}\index.html -f
RewriteRule !.*index.html$ %{REQUEST_URI}index.html [NC,QSA,L]
1

Best solution: 301 redirect

Whenever a user or bot opens https://www.example.com/my-page or https://example.com/my-page.html or https://example.com/my-page.php or https://www.example.com/my-page.html or https://www. example.com/my-page.php they should be redirected to https://example.com/my-page

OKish solution: canonical tags Make sure that whatever version of https://example.com/my-page is opened (with/without www, with/without .html/.php) there is always a self-referencing canonical tag, pointing to https://example.com/my-page

Hygiene

  • Make sure all internal links are pointing to the version you want.
  • Make sure the sitemaps only contains links to the version you want.
  • As far as possible, make sure backlinks point to the version you want.
1
  • @MalteLandwehf - Thanks for your reply. I updated my question by adding the .htaccess rules I'm using. I'm redirecting in the other direction. This is what I meant by "the links are resolved." Are you suggesting that I redirect in the opposite direction? My understanding is that the files themselves must have file extensions. I want to to see the clean URLs in the browser, but shouldn't the htaccess file be pointing the clean URLs to the files with extensions so they can be found? Jul 1, 2022 at 17:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.