Redirect to homepage based on server error code with htaccess

Question

I just took over management of a small HTML website. I discovered the security on the website was almost non-existent because full directory browsing was possible.

I have since disabled directory browsing through .htaccess and now if someone tries to access a folder directly it gives an unauthorized access. The error it reports is:

Not Authorized to View This Page [CFN #0004]

This is great but it doesn't help people who might not know what to do next. I would like this error to redirect to the homepage. What is the best way to do this through .htaccess? I'm not sure if it should be done based on the error or whether it should be done with something like an if/then as in "if no index.html then redirect to /"

I also want to be sure this only happens at the browser level and doesn't affect the crawlers. For example I have a downloads directory /downloads so if someone goes to http://example.com/downloads/ (and I guess http://example.com/downloads) then the browser redirects to the homepage but I don't want the search crawlers to be denied access to the downloads directory so they can index the pdfs. So the search crawlers should be able to access http://example.com/downloads/file.pdf. There are about five directories I want to redirect to the homepage. They're the standard /scripts, /css, /images, etc. Besides the root that contains the HTML files for the site the only other directory that has actual content is the downloads directory. For simplicity sake I guess it would be good to have something that does this for all directories instead of specifying each directory.

I also have three more questions beyond how to achieve this redirection:

1. What type of error is "Not Authorized to View This Page [CFN #0004]"? (Meaning what type of webserver gives this... Apache?... IIS?... I haven't seen it before and is it a 401 error?) I tried googling this but couldn't find any info on it other than people posting that they have the error and what to do about it.

2. Does disabling directory browsing affect the search crawlers? I haven't seen any errors on Google Search Console but want to be sure. I'm assuming this just stops directory browsing through the web browser?

3. I haven't done a lot of stuff with .htaccess and I know it's quite powerful. Is there more I can do with .htaccess to secure the directories/files from being hacked besides disabling directory browsing?

I tried googling for a solution but couldn't find anything. I found a lot of directory redirection questions but they seemed more for 301s. A similar solution might work for me but the problem is I don't know what kind of error is being produced by the server. If there is a solution already posted please direct me to that page.

If you need more info don't hesitate to ask.

Answer 1

If you are using .htaccess, then you are using Apache. .htaccess files are Apache's directory level configuration files. Other web servers don't use them.

The specific error code should be 403. That is a "Forbidden" error which should be returned when the user is not authorized to view the directory content. A 401 Unauthorized error should only be returned when a user has attempted to log in but the user name or password were not valid.

If you want other behavior instead of that message, you could implement an ErrorDocument for that error. You can accomplish this with a line in .htaccess:

ErrorDocument 403 /403.html

In that HTML document you could put a custom error message, or use JavaScript to change the location.href to the home page.

Alternately you could write code in PHP or Perl to issue an actual redirect instead of showing a message. For example if you used

ErrorDocument 403 /403.pl

in your .htaccess, then 403.pl could have the contents:

use strict;
use warnings;
print "Location: http://example.com"\n\n";

Another way of handling the situation would be to redirect each index.html for each directory individually:

Redirect /downloads/index.html http://example.com/
Redirect /js/index.html http://example.com/
Redirect /css/index.html http://example.com/

When you allow your webserver to generate a directory index for the directory, it has links to all the files in it. Search engines would use that that get a list of all the documents contained in the directory and to crawl all the items in it. If you don't allow the index functionality, you will need to link to the files you want search engines to crawl from elsewhere.

There are lots of things you can do in .htaccess for security or otherwise. Here is a pretty good list of some common use cases: https://github.com/cavo789/htaccess