3

I have been looking at our Apache log file to find the sources of referrals to our website from online publications but there are a lot of hits that I can't explain. For instance, how did this happen:

   130.185.153.42 - - [22/Feb/2017:02:52:37 -0500]
   "GET /drg/ahrq/drg2mdcxw1989.sas7bdat
   HTTP/1.1" 404 26074
   "http://washpost.bloomberg.com/market-news/"
   "Mozilla/5.0 (Windows NT 10.0; WO W64)
   AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102
   Safari/537.36" "-"

According to that log entry, there was a referral from a popular column on Bloomberg to a very obscure binary file that no longer exists on our site. There is no possibility Bloomberg ever referred to it. There are so many thousands of these dubious references (many to real files) that I wonder if the referrer field is even worth looking at. These are all from prestige publications that are unlikely to be referral spammers. Is referrer spoofing that widespread? Is there another explanation? Any suggestions on dealing with it?

Improve this question
1
  • 1
    Referrers can easily be fraudulent. It seems fairly likely in this case.
    – closetnoc
    Commented Mar 4, 2017 at 15:51

1 Answer 1

Reset to default
1

Referrers are only as trustworthy as the browser sending them. In other words they are wrong when:

  • Spammers use them
  • Crawlers get them wrong
  • Users add extensions to their browsers to change them (like this)
  • Browsers have bugs

The browser bug seems particularly likely to me. I think browsers often get their tabs crossed. I see referrers sent to my site that seem likely to be something that somebody has open in another tab.

Improve this answer
2
  • The browser bug seems very unlikely to me!? This would be such a fundamental/critical bug I struggle to imagine it could be a thing. Unless perhaps... a rogue browser extension is responsible?
    – MrWhite
    Commented Jul 14, 2017 at 20:22
  • @user82217 I run a local web server that I use for development. I sometimes see referrals in my log files with pages that I know I've visited. I've never tried to replicate the behavior, but I've seen it happen.
    – Stephen Ostermiller ♦
    Commented Jul 15, 2017 at 1:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.