9

I have a website that uses a set of quite simple RewriteRules of the format:

RewriteRule ^p/(.*)$ /product.php?$1 [L]

All was working well and still is working perfectly fine on localhost. For some reason, without me making any changes, those pages have started returning a 403 Forbidden error.

The problem seems to be spaces (encoded as %20) in the url query string. Looking in the logs shows this error

[rewrite:error] [pid xxxx:tid xxxxxxxxxxxxxx] [client xxx.xxx.xxx.xx:xxxxx] AH10411: Rewritten query string contains control characters or spaces

Sure enough, example.com/p/parameter%20name=value gives a 403 whereas example.com/p/parameterName=value doesn't.

Just adding spaces to a query string on a page that has a rewrite rule that doesn't use the query string doesn't cause an error. e.g. the rule RewriteRule ^contact-us/$ /contact.php [L] doesn't have a problem with the url example.com/contact-us/?query%20string=value

I'm aware that using + instead of %20 is a possibility or not using spaces at all. Both are not very practical options at the moment and using %20 was perfectly fine until about a day ago.

I can't see any permissions issues anywhere either.

Has anyone else come across this issue?

UPDATE:

As Mr White points out in his answer, the trick here is to make sure the B flag is set on the Rewrite rules in htaccess.

Because I am accessing the full query string using $_SERVER['QUERY_STRING'] I then needed to use urldecode as well.

This then returns the same output that I used to get before the Apache update.

1
  • 1
    What version of Apache are you running?
    – MrWhite
    Mar 10, 2023 at 22:52

1 Answer 1

9

For some reason, without me making any changes, those pages have started returning a 403 Forbidden error.

It's likely the version of Apache has been updated on your server.

I answered a similar question on StackOverflow just yesterday. This would seem to be a bug (IMO) in a recent version of Apache (2.4.56-1 / 2.4.52-1 ?), although this has also been cited as a "security fix" in the other answer on that question.

The "solution" is to use the B (escape backreferences) flag on your rule, to URL encode the (unencoded) backreference before it is used in the substitution string. For example:

RewriteRule ^p/(.*)$ /product.php?$1 [B,L]

(Presumably this URL-encodes the space as +, as opposed to %20 - which is what "normally" happens.)

Arguably, you should perhaps be using the B flag anyway here, although this depends on what other "special" characters you are expecting. eg. Without the B flag, a request of the form /p/param=v&lue would not be rewritten correctly since the & would not be URL encoded and the parameter value in the query string would effectively be broken.

However, you shouldn't need to use the B flag if all you are expecting is spaces. mod_rewrite (should) automatically URL encode an "invalid" substitution string. In this case the space should be URL encoded as %20 - which is perfectly valid, but now breaks with an error. In fact, it has been reported that even manually rewriting to %20 in the query string also breaks with this error. For example:

# Now breaks with error "AH10411: Rewritten query string contains control characters or spaces"
RewriteRule ^foo$ bar?q=one%20two [L]

The above is not invalid, but is now deemed to be.

(DISCLAIMER: I've not been able to test this myself yet.)

I'm not sure exactly what this "security fix" is meant to resolve (if indeed it is intentional), but it is certainly a breaking change that is affecting many sites. It also now invalidates several examples in the official Apache docs, which makes me think this is more of a "bug" than a "fix".

Reference:

5
  • This is a great answer and very informative. I will test this out asap. For the record, I tried using the B flag but that broke my local version so didn't try it on the live server but will do that now. Mar 11, 2023 at 16:42
  • @JimboVader How did the B flag break your local version? What URL were you requesting? What Apache version are you running locally and on the live server?
    – MrWhite
    Mar 12, 2023 at 0:50
  • I've just upgraded apache on my local server and that now has the same 403 issue so it looks like you are correct on the cause of the issue. The problem now is that when using the B flag, getting the query string using $_SERVER['QUERY_STRING'] returns something different from before. By that, I mean my functions for processing it don't work. However, currently investigating exactly what is different and will report back... Mar 12, 2023 at 10:39
  • 1
    @JimboVader The only thing that I would expect to be different is that spaces in the query string are now encoded as + and not %20 (which is a perfectly valid encoding in the query string) and this will be reflected in the $_SERVER['QUERY_STRING'] superglobal. However, if you are accessing the URL params via the $_GET superglobal then you should not notice any difference since this is already URL-decoded (whereas the $_SERVER['QUERY_STRING'] superglobal is not).
    – MrWhite
    Mar 12, 2023 at 11:42
  • 1
    Correct again, after adding the B flag I then need to urldecode the query string and all seems to work again. Mar 13, 2023 at 12:48

Not the answer you're looking for? Browse other questions tagged or ask your own question.