What is blocking some visitors in htaccess?

Question

How do I figure-out what is blocking some visitors in htaccess when it's NOT the IP range, User-Agent, Referer nor anti-hotlinking?

Forbid empty Referer & all other domains from hotlinking to images, Except for your domain, google, bing or other good domains

<IfModule mod_rewrite.c>
RewriteCond %{HTTP_REFERER} !^($|(http|https):\/\/.*(mySite1\.com|mySite2\.com|ask\.com|bing\.com|duckduckgo\.com|google\.com|ixquick\.com|msn\.com|startpage\.com|yahoo\.com)) [NC]
RewriteCond %{REQUEST_URI} ^.*\.(bmp|gif|ico|jpg|jpeg|pdf|png|svg|svgz|swf)$ [NC]
RewriteRule ^(.*)$ http://i.imgur.com/qX4w7.gif [R,NC,L]
# RewriteRule ^(.*)$ - [F]
</IfModule>

Answer 1

First I would adjust the code slightly so that its more like this:

RewriteCond %{HTTP_REFERER} !^(|(http(|s):\/\/))(mySite1|mySite2|ask|bing|duckduckgo|google|ixquick|msn|startpage|yahoo)\.com$ [NC,OR]
RewriteCond %{HTTP_REFERER} ^$
RewriteCond %{REQUEST_URI} ^.*\.(bmp|gif|ico|jpg|jpeg|pdf|png|svg|svgz|swf)$ [NC]
RewriteRule ^(.*)$ http://i.imgur.com/qX4w7.gif [R=301,NC,L]

How visitors are being blocked

The first line in the code means search for a pattern that may start with http://, https:// or no prefix, then add to the search query any of: mySite1, mySite2, ask, bing, duckduckgo, google, ixquick, msn, startpage, yahoo, then add .com. If a match is found, then go to the image extension check. Otherwise check for an empty referrer string and if it matches, then continue to the image extension check.

If the URL (minus domain) contains any of: .bmp, .gif, .ico, .jpg, .jpeg, .pdf, .png, .svg, .svgz or swf, and the referrer check is true then all requests that match are redirected to http://i.imgur.com/qX4w7.gif.

Why redirecting to image on another server may be bad

Also, with your method, when users come from an unauthorized URL such as facebook, they will see the image, but the image is on another server unless you're running imgur.com. The downfall to this is if that server goes down and someone without authorization wants an image, they will either end up with a 404 page from imgur.com or a 500 internal server error page, or the browser may hang (depending on the bad health of imgur.com).

Another reason its bad to link to a remote image to handle error is because the image on a remote server could change at any time, especially if you have no authorization to manage the image file. You wouldn't want the image to show the hotlink message one day as intended and the next day, it suddenly switches over to a picture of a country flag, would you?

What you should do instead is make a much simpler image that takes up almost no space (like less than 1 KB). Try a static GIF that's no more than 200px wide by 200px high and that contains no more than a few colors. Then upload that image to your server and use it instead of one from imgur.com.

That way, if everything else crashes around you then people without authorized access to the images will see the simpler image (hot-linking message) when they want the protected images.

Answer 2

Here's other code that is similar to yours that makes it easy to understand what is blocking:

RewriteCond %{HTTP_REFERER} !^http:\/\/mySite1.com$ [NC]
RewriteCond %{HTTP_REFERER} !^http:\/\/mySite2.com$ [NC]
RewriteCond %{HTTP_REFERER} !^http:\/\/ask.com$ [NC]
RewriteCond %{HTTP_REFERER} !^http:\/\/bing.com$ [NC]
RewriteCond %{HTTP_REFERER} !^http:\/\/duckduckgo.com$ [NC]
RewriteCond %{HTTP_REFERER} !^http:\/\/google.com$ [NC]
RewriteCond %{HTTP_REFERER} !^http:\/\/ixquick.com$ [NC]
RewriteCond %{HTTP_REFERER} !^http:\/\/msn.com$ [NC]
RewriteCond %{HTTP_REFERER} !^http:\/\/startpage.com$ [NC]
RewriteCond %{HTTP_REFERER} !^http:\/\/yahoo.com$ [NC]
RewriteCond %{HTTP_REFERER} !^https:\/\/mySite1.com$ [NC]
RewriteCond %{HTTP_REFERER} !^https:\/\/mySite2.com$ [NC]
RewriteCond %{HTTP_REFERER} !^https:\/\/ask.com$ [NC]
RewriteCond %{HTTP_REFERER} !^https:\/\/bing.com$ [NC]
RewriteCond %{HTTP_REFERER} !^https:\/\/duckduckgo.com$ [NC]
RewriteCond %{HTTP_REFERER} !^https:\/\/google.com$ [NC]
RewriteCond %{HTTP_REFERER} !^https:\/\/ixquick.com$ [NC]
RewriteCond %{HTTP_REFERER} !^https:\/\/msn.com$ [NC]
RewriteCond %{HTTP_REFERER} !^https:\/\/startpage.com$ [NC]
RewriteCond %{HTTP_REFERER} !^https:\/\/yahoo.com$ [NC]
RewriteCond %{REQUEST_URI} ^(.*)\.(bmp|gif|ico|jpg|jpeg|pdf|png|svg|svgz|swf)$ [NC]
RewriteRule ^(.*)$ http://i.imgur.com/qX4w7.gif [R=301,NC,L]

RewriteCond %{HTTP_REFERER} ^$
RewriteCond %{REQUEST_URI} ^(.*)\.(bmp|gif|ico|jpg|jpeg|pdf|png|svg|svgz|swf)$ [NC]
RewriteRule ^(.*)$ http://i.imgur.com/qX4w7.gif [R=301,NC,L]

I know its long, but the first several lines are simple. Each line works with one URL and if the referrer isn't the url listed, then the list continues on down to try to match the image extension and if it matches then redirect to the no-hotlink image from imgur.com

The second set of lines are easier to understand. It basically looks for an empty referral string and does the same image extension verification as usual and redirects if theres a match.

You can add and remove sites as you please.