Drop in referral data due to HTTP, changed to HTTPS no recovery referral data since (in the last year)

Question

We changed our site from HTTP to HTTPS about a year ago.

We have a static website, with no functionality for user input, so did not see it as an imperative to change to HTTPS anytime sooner, the reason we changed, is that we used to gets lots of Google Analytics refer data a few years ago in Google Analytics, but noticed more and more of the traffic being listed as "direct" we presumed that this was because the sites that where referring traffic to us had also made the switch to HTTPS.

Since the change we have NOT seen a recovery of Google Analytics referral data going forward.

I wonder if this is due to any older backlinks to our website being to the HTTP version rather than than HTTPS version, even though we 301 between them.

Would that be the case, and if so is there anyway we could fix it to get more referral data ?

Answer 1

I just tested whether or not implementing HSTS and getting your site on the preload list would allow browsers to pass a referrer through a http link to your site. Sadly it doesn't work.

Here is how I performed the test:

1. I found a .dev site with a PHP script that echos back the referrer. I did this by searching google for site:.dev HTTP_REFERER. Since the entire .dev top level domain is on the HSTS preload list, any .dev site with a script that shows the HTTP variables will work for testing.

2. I created a document on my https site with two links pointing to the .dev site with a dummy parameter to ensure no caching between clicks:

   <a href="https://example.dev/test/php/test.php?HTTPS">HTTPS link</a><br>
   <a href="http://example.dev/test/php/test.php?HTTP">HTTP link</a>
   

3. I opened the page in my browser and clicked on the two links.

Both Firefox and Chrome behaved the same way. They passed the referrer on the HTTPS link but not on the HTTP link. I was hoping that because HSTS automatically upgrades HTTP to HTTPS without over hitting your server, that the referrer policy would also automatically be updated. It wasn't.

Because HSTS doesn't help, the only method I know for getting your referrer data back would be to contact all the sites and ask them to change their sites. They could either change the links to your site to https or change their Referrer-Policy header to unsafe-url.

Answer 2

The external HTTPS site would need to also update the backlink to link to your HTTPS site, otherwise if the link itself is still HTTP then the Referer header is not sent. Old backlinks probably have not been updated.

This has nothing to do with your HTTP to HTTPS redirect - which would otherwise pass the Referer (still user-agent dependent). The problem is that the Referer is probably not being sent in the initial request.

HSTS + preload list (as mentioned in comments) may help with this as this should instruct the browser to upgrade the initial request (as if the backlink was already HTTPS) and thus permit the Referer to be sent. HSTS alone may not be sufficient, unless you have many repeat users that follow backlinks. Since HSTS (without being on the preload list) is only "remembered" by the browser after the user visits your HTTPS site the first time. EDIT: However, it seems from @Stephen's testing that this doesn't help after all!

Also note that websites today have a lot more control over the Referer that is sent if they implement the Referrer-Policy. This allows sites to simply block the Referer on all outgoing requests if they so wish.

Answer 3

Now that you have your site updated with HTTPS I would circle back around to the referring sites and ask to have the link updated to https (not a full fix but setting the right foundation).

The second factor to consider is if your site and referring sites may have been punished for not having HTTPS. My understanding (reading between the lines) is that Google will not reward unsecured sites regardless if users pass confidential information or not. So have the sites that were sending you traffic seen a decline in traffic over that same period? i.e., are there just less users to send you YoY?

Remember that users are going to hit a 'SITE UNSECURE -- TURN AWAY' page before being able to access an unsecured site.

I ask because if your referral traffic is down it may have more to do with the site referring you the traffic than your own per se, particularly if these sites didn't secure themselves.

My biggest advice though is that Google in my experience rarely backtracks, I wouldn't focus too much on regaining old referral traffic. Rather making the fixes that may have caused it, improving your security and user signals to benefit on the next uptick.