I have a website with about 1.5M pages. I am condensing pages that do not have large amounts of content to improve the quality of the site. As a result I have about 70K pages that I have added a < canonical link> to, pointing towards the new master page for the content. Now I want to completely work the existing pages out of the system. I want to 301 redirect the existing pages to the new master pages that the content now resides upon. I am running IIS and normally would use < rewriteMaps>. I am concerned that adding 70k lines to my web.config, to be parsed for every URL, is going to affect performance. Any suggestions on a known/tested way to accomplish this? TY for your time.
There are two solutions I can think of:
- Find patterns in the redirects and see if you can cut the number down. For example, if you had a large chunk that redirected from one folder to another, you could match something like
/folder/([A-Za-z0-9-]+)and redirect to
/newfolder/$1. This might still leave a large number of redirects though.
- Redirect all unfound pages to one script, where you look up a list of redirects in a database or other file. If you find a match then redirect, otherwise serve a 404. Using only one redirect at the server level means serving all your regular pages lose no speed. Your redirects may technically be slower but it should likely be negligible, and they will not be the most popular pages.