2

I have a substantial number of rewrite rules in place for a variety of reasons such as:

disallow indexes, set custom error pages, force non-www, https, strip trailing slash, redirect index.html or.php to root, pretty URLs (hides file extensions & queries), 404 any URL with Additional Path Info

I was having trouble with nonsense URLs (URLs with Additional Path Info) invoking broken pages & 500 errors that were being indexed as duplicates by search engines. Sample: example.com/index.php/somefolder/another/file/query...

I set up rules to redirect to a default page rather than rendering broken pages or 500 errors. I then modified that same block to send such nonsense URLs to 404 instead so they would not be indexed.

MrWhite educated me on the proper use of AcceptPathInfo Off but unfortunately this had no effect.

The current solution (full array shown below) requires additional .htaccess files inside any sub-directories containing accessible pages but it worked like a charm.

This is the full array of rules currently in place. Everything is now working perfectly. I'm currently working to eliminate the need for the additional htaccess files in the subdirectories.

AcceptPathInfo Off    
Options -Indexes

Options +FollowSymLinks
RewriteEngine On
RewriteBase /

## 404 ANY URL WITH ADDITIONAL PATH INFO ##
RewriteCond %{THE_REQUEST} /([^.]+)\.php/? [NC] 
RewriteRule ^ /%1 [NC,R=404,L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^([^/]+)/?$ /$1.php [L,NC]

## SET CUSTOM ERROR PAGES ##
ErrorDocument 400 /error/error_400.php
ErrorDocument 401 /error/error_401.php
ErrorDocument 403 /error/error_403.php
ErrorDocument 404 /error/error_404.php
ErrorDocument 500 /error/error_500.php

## FORCE HTTPS & NON-WWW ##
## RewriteEngine On 
RewriteCond %{SERVER_PORT} 80 
RewriteRule ^(.*)$ https://dev.example.com/$1 [R=301,L,NE]

## STRIP TRAILING SLASH ##
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)/$ /$1 [R=301,L]

## REDIRECT INDEX TO ROOT ##
RewriteRule ^index\.php$ / [R=301,L]
RewriteRule ^index\.htm$ / [R=301,L]

## PRETTY URLS FOR DYNAMIC FILES ##
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^items/([a-zA-Z0-9_-]+)$ item.php?item=$1 [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^items/([a-zA-Z0-9_-]+)/$ item.php?item=$1 [L]

## PRETTY URL FOR ANY STATIC FILE ##
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^([a-zA-Z0-9_-]+)$ $1.php [L,QSA]

The block labeled "404 ANY URL WITH ADDITIONAL PATH INFO" would be unnecessary if the "AdditionalPathInfo off" directive would work. This setting can also be controlled in the httpd-conf file on the server. Editing httpd-conf requires command-line, ssh access and can be dangerous. Consult your hosting company or server admin if that sounds scary.

Currently using feedback I've received here to clean this up. Updates to come.

1 Answer 1

3
example.com/index.php/somefolder/anotherfolder/file/query...

Everything after the file index.php in the URL is called "additional pathname information" (or path-info for short). Whether this is valid on the URL is, by default, dependent on the file handler - in this case the PHP handler. And the PHP handler allows path-info. This information is available in your script if you wanted to use it, ie. $_SERVER['PATH_INFO']. (By contrast, the text/html handler does not allow path-info, so a request like /index.html/anything will - by default - result in a 404.)

If you wish all these "nonsense" URLs to return a 404 then you can simply disable path-info on all requests. For example, at the top of your .htaccess file:

AcceptPathInfo Off

Aside:

## STRIP ANYTHING AFTER .php/ ##
## CREATES CHAIN OF 3 REDIRECTS 302-301-301 NOT GREAT ##
RewriteCond %{THE_REQUEST} /([^.]+)\.php/? [NC] 
RewriteRule ^ /%1/ [NC,R,L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^([^/]+)/?$ /$1.php [L,NC]

It wasn't clear from your earlier question, but you'd put this code in the wrong place - it should go nearer the start of your .htaccess, not at the end. It is also conflicting with your other rules, eg. you are appending a slash, but you have an earlier rule that STRIP TRAILING SLASH - so this will naturally result in a redirect chain.

You also have code that appends the .php extension (via an internal rewrite) immediately above this code block - so this would seem to be unnecessary duplication.

You are also missing some L flags on your rules, so processing is going to continue unnecessarily through your file. Whilst it may not strictly be necessary in every case, you can include an L flag on each of your RewriteRule directives.

However, if you just want to return a 404, then use AcceptPathInfo Off instead, as mentioned above.


UPDATE: AcceptPathInfo off had no effect. Hosting company can't answer if it's set at the httpd-conf level, can't even provide the correct location for the httpd-conf file. I was able to make it work with rewrite rules.

If you have other mod_rewrite directives (rewrites/redirects) then these can override AcceptPathInfo. However, I see nothing in your .htaccess file that would do this (except for your more recent "workaround")*1. And the fact that it seemingly "does nothing" is a bit strange. Maybe it has something to do with how PHP is handled on your server?

What happens if you request something like:

/foo.html/bar/baz

...where /foo.html is a valid file?

Yes, AcceptPathInfo can be set in the server config, however, you should still be able to override this in .htaccess. (It would also be a bit silly for a shared host to set AcceptPathInfo On in the server config - I assume you're on a shared host?) It is possible that the host has blocked the override of AcceptPathInfo in .htaccess, however, they should know if they have and this would again be silly if they have. However, on Apache this would result in a 500 Internal Server Error, not a silent fail (unless perhaps you are on a LiteSpeed server?!).

*1 EDIT: Although you seem to be using multiple .htaccess files in subdirectories - any of these could potentially be conflicting. Depending on circumstances, multiple .htaccess files is best avoided as it can become a maintenance problem (and confusing to debug).

However, your .htaccess file can be simplified....

The final solution required an additional .htaccess file inside the directory containing my custom error pages but it worked like a charm.

You shouldn't need to do that? What did you add?

The last block (PRETTY URLS FOR ANY STATIC FILE) is also added to a secondary htaccess which is placed in any subdirectories.

Again, you shouldn't need to do that? By default, the mod_rewrite directives in the subdirectory .htaccess file will completely override the mod_rewrite directives in the parent. (So no HTTP to HTTPS redirect, no strip trailing slash, etc.)

## 404 ANY URL WITH ADDITIONAL PATH INFO ##
RewriteCond %{THE_REQUEST} /([^.]+)\.php/? [NC] 
RewriteRule ^ /%1 [NC,R=404,L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^([^/]+)/?$ /$1.php [L,NC]

This would seem to "block" (404) any URL that simply includes a .php extension, rather than URLs that contain path-info. (Your earlier version of this rule stripped the .php extension.)

When you specify a R code outside of the 3xx range then the substitution string (ie. /%1) is ignored. (The L is alos not required in this case - it is implied.)

As mentioned above, the second rule is superfluous. This has nothing to do with "blocking" URLs that contain path-info. The later rule that states "PRETTY URL FOR ANY STATIC FILE" does the same thing.

To "block" (404) URLs that contain path-info (not just on .php files) then you can do the following instead:

RewriteCond %{PATH_INFO} .
RewriteRule ^ - [R=404]

OR, to block only .php URLs that contain what "looks-like" trailing path-name information (whether it is really path-info or not, ie. whether the .php file exists or not) then you can do this in a single directive:

RewriteRule \.php/ - [R=404]
## REDIRECT INDEX TO ROOT ##
RewriteRule ^index\.php$ / [R=301,L]
RewriteRule ^index\.htm$ / [R=301,L]

(What about requests for .html?) This can be done in a single directive:

RewriteRule ^index\.(php|html?)$ / [R=301,L]
## PRETTY URLS FOR DYNAMIC FILES ##
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^fonts/([a-zA-Z0-9_-]+)$ item.php?item=$1 [L]
RewriteRule ^fonts/([a-zA-Z0-9_-]+)/$ item.php?item=$1 [L]

RewriteCond directives only apply to the first RewriteRule directive that follows. So, the 2nd rule above is processed unconditionally. Although this is really just one rule - you can make the trailing slash optional. However, the preceding rule already "STRIP TRAILING SLASH", so the trailing slash is never present anyway? (Having the trailing slash optional for internal rewrites potentially creates duplicate content.)

However, the first condition would seem to be redundant anyway, since the RewriteRule pattern ^fonts/([a-zA-Z0-9_-]+)$ is unlikely to match a real file (unless you have physical files that don't have file extensions?!). It could potentially match a directory - but that's your call (do you have subdirectories of the /fonts directory? (Is /fonts even a real directory?)

So, this could certainly be simplified to (and possibly further):

RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^fonts/([\w-]+)$ item.php?item=$1 [L]

The \w shorthand character class is the same as [a-zA-Z0-9_].

## PRETTY URL FOR ANY STATIC FILE ##
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^([a-zA-Z0-9_-]+)$ $1.php [L,QSA]

Again, the preceding "file" condition would seem to be unnecessary here (for the same reason as mentioned above). The QSA flag is not required, since you've not included a query string in the substitution string, so the query string from the request is appended by default.

You mention that you are copying this block to manage extensionless URLs in those subdirectories. That would seem to be unnecessary repetition. You should look into modifying this one rule to cover all necessary subdirectories. Although exactly how you do this could be dependent on your system.

Summary

Bringing the above points together we get something like the following:

AcceptPathInfo Off    
Options -Indexes +FollowSymLinks

## SET CUSTOM ERROR PAGES ##
ErrorDocument 400 /error/error_400.php
ErrorDocument 401 /error/error_401.php
ErrorDocument 403 /error/error_403.php
ErrorDocument 404 /error/error_404.php
ErrorDocument 500 /error/error_500.php

RewriteEngine On
RewriteBase /

## 404 ANY URL WITH ADDITIONAL PATH INFO ##
RewriteCond %{PATH_INFO} .
RewriteRule ^ - [R=404]

## FORCE HTTPS & NON-WWW ##
## RewriteEngine On 
RewriteCond %{SERVER_PORT} 80 
RewriteRule (.*) https://dev.example.com/$1 [R=301,L,NE]

## STRIP TRAILING SLASH ##
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule (.*)/$ /$1 [R=301,L]

## REDIRECT INDEX TO ROOT ##
RewriteRule ^index\.(php|html?)$ / [R=301,L]

## PRETTY URLS FOR DYNAMIC FILES ##
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^fonts/([\w-]+)$ item.php?item=$1 [L]

## PRETTY URL FOR ANY STATIC FILE ##
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^([\w-]+)$ $1.php [L]
15
  • Thanks MrWhite. I accidentally posted the first question while not logged in so I figured I'd start over. Please see response below.
    – Raseone
    Commented Mar 1, 2020 at 4:53
  • Understood on moving it near the top. Definitely conflicting with striping trailing slash. am I appending the slash in both rules or only the 2nd from this block? I see what you mean with appending the .php in the previous block I may have to live with that but I'll try to remove. Thanks a million for the AcceptPathInfo tip. That should simplify matters. Trying now.
    – Raseone
    Commented Mar 1, 2020 at 4:59
  • Could I trouble you to point out where you see missing [L] flags? Now I'm thinking every rule without one needs one. Sounds like this would help isolate the effect of each block.
    – Raseone
    Commented Mar 1, 2020 at 5:06
  • "am I appending the slash in both rules or only the 2nd from this block?" - From "this [last] block", just the first rule appends the trailing slash. The 2nd rule is an internal rewrite that appends the .php extension (to make it work, because it is stripped by the previous rule). "I may have to live with that" - you should just be able to remove it. I doubt that that last rule is even being processed. "missing [L] flags? ... every rule ... help isolate the effect of each block." - Yes, every rule (whilst it's not strictly necessary in all cases - it is a good idea to "isolate" each block).
    – MrWhite
    Commented Mar 1, 2020 at 10:15
  • Thanks MrWhite. Please see my edits to the original post. I've updated to the current status. I killed the last block entirely, added the [L] flags, added the AcceptPathInfo off directive and read up on the Apache Docs.
    – Raseone
    Commented Mar 1, 2020 at 17:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.