Optimal way to implement very big redirect list in Apache2

Question

I have about 6000 URLs that need to be redirected after changing the CMS. All those URLs are stored in a DB table together with an ID that allows me to tell what is being referenced. Using this table I could generate one huge .htaccess with a lot of Rewrite rules but I suspect that is very inefficient. What's the best way to go? As far as I can tell there are two options:

• Using a RewriteMap where the key is the old URL (it contains no ID, just text, so it has to be the whole thing). I wonder if having 100-150 char long keys is gonna work well.
• Generating a DB table that exactly matches every old/new URL pair and calling some small script to make the redirect.

(Could be summed up as: Is a hash map stored in the filesystem more efficient than a indexed DB table?)

Second part of the question. The new URLs contain the ID of the page being invoked. Something like

www.example.com/abc/21-defghi.html

If instead I call

www.example.com/21-xxx.html

I get a 301 Redirect to the correct URL. Should I bother to generate the exact new URL or is it ok to concatenate two redirects? This means either storing just an ID or the whole new URL.

Answer 1

Creating a huge .htaccess can have a serious performance impact on your system as it is read linearly for all requests, at least until a rule with the L attribute (Last Rule) is matched.

The way I did something similar (about 3000 redirects) is to put a custom error page in the .htaccess. From memory the syntax is something like this:

ErrorDocument 404 404.php

Then in 404.php, implement the logic using a MySQL database, searching for the new URL using the requested URL. If it is found, redirect the request using the PHP location header, if not, then display a custom error page.

Unlike simply issuing the 404 redirect in the .htaccess, the 404.php file can keep track of requests and keep stats on the redirects being returned.