Incorrect Referring URLs - What can be the cause?

Question

I am running a website with affiliate links .

When the visitors of mydomain.com/page.php click on such an affiliate link, they are being sent to a link on a domain owned by the affilate network (network.com/link), and then redirected through the affiliate network, to the relevant page in the store (store.com/page.asp).

Over the last two months, the reports of the affiliate network indicate that about 13,000 clicks that I sent to such links, carried mydomain.com/page.php as the referring URL, as I would expect.

However, about 20 other clicks carried abnormal referring URLs, such as:

http://app.mam.vaccint.com/getapp/CT3297962/mam.html

http://www.store.com/page.asp

http://www.network.com/link

http://apnwidgets.ask.com/widget/everest/radio/4/radio-button.html

http://search.yahoo.com/search

http://www.google.com/webhp

http://www.bing.com/

http://192.168.1.1/spyware/blockpage

Unfortunately, This has led the compliance team of my affiliate network to believe that I have a hidden traffic source apart from my website, they claim that it appears to be as if I am using some kind of a third party software to send traffic to store.com, which is not true of course.

They are holding me accountable for this situation and I am required to provide explanations to this situation.

What could have caused my website visitors to arrive at network.com / store.com while carrying the above referring URLs?

Answer 1

One can spoof a referrer relatively easily in order to hide scans, harvesting, crawls, etc. Makes sense that a spoof would appear as an innocent search -- ie google, yahoo, bing, etc. Generally they hit the link once, so it may be 20 crawls. Also, they can sometimes spoof for advertising reasons. Think if you clicked a couple of those weird referrers, thats a hint more traffic. Or, think if you were one of those sites that still shares public logs...the referrer would be in SERPs. Moral of the story: just like many headers, the referrer constant is highly unreliable. Wikipedia

You should try to find the origin IP's of these 20 visits, surely it would tell you more than just a referrer would. What if your affiliate system doesn't track IP's or proxy sniff? Well thats their fault and they should not come down on you for their inferior platform. They should have all that information available and be looking to their own logs rather than expecting you to somehow understand out/in traffic that you don't even track on your own server.

Ask the "compliance team" for the full constant + header + timeframe stack for each of those visits :) Make sure they provide an implicant "no proxy" if one is not detected. If they can't comply then you should consider moving to a better affiliate network.

Answer 2

You might point out that if you were actually using some illicit method of traffic generation it would be an extremely inefficient means of doing so with little benefit to you. Twenty clicks is only 0.15%, i.e., a tiny fraction of a percent, of 13,000, so there would be little incentive for you to do that.

For the ones that point to specific webpages rather than a search engine page for Bing, Google, or Yahoo, have you examined the source code for those pages, other than the store.com and network.com ones, to ensure that none of them have your affiliate ID somewhere in the code on the page? Sometimes people will copy a section of code verbatim from another site and put it on their own web page. Or someone else who is part of the same affiliate network could have made a typo in the code for one of the ads they placed on their page, so your affiliate id is used instead of their own id. But, if someone made such a mistake a month ago and then later fixed it, you wouldn't know from looking at the current version of the web page.

For the ones for search engines, I see entries in my Apache access log with referrer URLs that include bing.com, yahoo.com, and google.com when someone searches on keywords in those search engines and then clicks on a link to my website that is returned in the search engine results page. I don't know how doing something like that would lead directly to a click that takes someone to the affiliate network, other than, perhaps, a click on a cached copy of one of your web pages stored by the search engine, but I also don't see how just a few clicks with a search engine appearing as the referrer warrants a conclusion that you are using nefarious means of generating affiliate traffic.

For the last URL you listed, 192.168.1.1/spyware/blockpage", the IP address is part of RFC 1918 private IP address space, which isn't even routed over the Internet, so there's no way for you to determine the origination point to determine the cause and prove your innocence.

Unfortunately, since those 20 clicks weren't generated from any site you control, it will likely be difficult for you to prove your innocence by accounting for what some people unknown to you may have done, but I think 20 odd clicks versus 13,000 legitimate clicks should indicate to whomever is reviewing the data for the affiliate network that, though some of them may seem odd, they are probably not due to any attempt by you to generate illegitimate traffic.

I understand that a referring URL of network.com or store.com might seem unusual, but I don't know what they might think would be some illicit means of generating traffic with those referrer values that would be of value to you.