I have been looking at our Apache log file to find the sources of referrals to our website from online publications but there are a lot of hits that I can't explain. For instance, how did this happen:
18.104.22.168 - - [22/Feb/2017:02:52:37 -0500] "GET /drg/ahrq/drg2mdcxw1989.sas7bdat HTTP/1.1" 404 26074 "http://washpost.bloomberg.com/market-news/" "Mozilla/5.0 (Windows NT 10.0; WO W64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36" "-"
According to that log entry, there was a referral from a popular column on Bloomberg to a very obscure binary file that no longer exists on our site. There is no possibility Bloomberg ever referred to it. There are so many thousands of these dubious references (many to real files) that I wonder if the referrer field is even worth looking at. These are all from prestige publications that are unlikely to be referral spammers. Is referrer spoofing that widespread? Is there another explanation? Any suggestions on dealing with it?