4

We're implementing a feature where users can generate unique referral links that they can share and receive rewards when people sign up. The pages themselves have identical content.

There are two approaches:

  1. https://example.com/referral?code=abc with canonical to https://example.com/referral
  2. https://example.com/referral#abc

I know that everything after the hash # symbol is not being sent to servers altogether, so Google will not know that there's anything going on at all.

But when using query parameters, Google does acknowledge that those pages are duplicates of the /referral one. Therefore, a canonical meta tag is needed to avoid duplicate content penalties.

So it all boils down to this - do canonical relations somehow change Google's crawling behavior? For example, could it treat pages as "nofollow" because they're essentially duplicates?

Would one approach be better than the other in any way?

3 Answers 3

3

I don't think either approach is ideal.

The parameter approach is likely the better of the 2 as it does not try and jump to a non-existent anchor point on the page.

A more appropriate solution might be to use the form

  https://example.com/referral/abc

And combine them with a robots.txt file which says not to index /referral and maybe produce a 301 redirect to the correct page after setting the appropriate tracking cookie or similar.

You can combine the above with a .htaccess file or server level config to rewrite it to a request with the appropriate query code to include the appropriate query code as well.

Update (Untested)

To do something like the above in an Apache .htaccess file I expect you would need to add something along the following lines -

  RewriteEngine On
  RewriteRule /referral/(.*) index.php?referrer=$1

Which would have the affect of rewriting https://example.com/referral/abc to https://example.com/index.php?referrer=abc

To implement a block in robots.txt create a robots.txt in the root directory with the following, or if it exists add the following:

  User-agent: *
  Disallow: /referrer/
4
  • I would love to see an example of how you'd handle this backend config (pseudo code works) that gets the appropriate query code. Nice answer! Aug 30, 2023 at 14:58
  • 1
    @MikeCiffone See updated answer with example of how to implement.
    – davidgo
    Aug 30, 2023 at 20:36
  • Got it...then I assume you're just going to do something like if (isset($_GET['referrer'])) { $referral_code = $_GET['referrer']; // etc } Aug 31, 2023 at 16:23
  • 1
    @MikeCiffone Broadly yes - If Im writing PHP (and I dont hold myself out as a coder) I'd grab $_REQUEST['referrer'] making sure to sanitize the input. when assigning it to a variable.
    – davidgo
    Aug 31, 2023 at 18:37
1

What value does the /referral page have to your organic search presence? Should you care if Google perceives your referral pages as duplicate?

Google, amongst other search engines, use the canonical URLs to understand content and page quality. Google will crawl a canonical URL more frequently than non-canonical ones. Non-canonicals will not be indexed. If a non-canonical version of a page has some unique content, it will still not be indexed.

For these referral links the most simple and straightforward solution is the query string. It's usage is common practice for this sort of thing, and makes tracking via Google Analytics convenient. If you want the /referral page indexed, and want to ensure that the query string versions don't get indexed, canonicalize it.

davidgo provides a crafty solution which could definitely be appropriate. My concern is one of effort, regardless of the approach.

If you don't care about the referral page being indexed, simply setting no-index would be easier to manage.

If your site is large, and you have a lot of users that will be generating these code URLs, you don't want search engines wasting your budget crawling them. A sweeping robots.txt disallow is probably responsible.

Note that you will not see organic search traffic in Google Analytics for anything blocked by robots.txt, but you will see direct and referral traffic.

0

So it all boils down to this - do canonical relations somehow change Google's crawling behavior? For example, could it treat pages as "nofollow" because they're essentially duplicates?

I don't think that declaring canonical changes the crawling behaviour. It changes the indexing behaviour. If you don't declare a canonical, it will decide. Even when you do declare one, it is still a suggestion to Google.

You should just take the easy route of declaring teh canonical. Setting nofollow is not necessary.

A canonical tag (rel=“canonical”) is a snippet of HTML code that defines themain version for duplicate, near-duplicate and similar pages. In other words, if you have the same or similar content available under different URLs, you can use canonical tags to specify which version is the main one and thus, should be indexed.
...

Google states that they usually respect the canonical URL you set, but not always. That’s because canonicals tags are hints not directives. As long as they are respected then any signals such as links should consolidate to the canonical URL.

Using canonical tag best practices also helps mitigate the risk of Google seeing an undesirable version of the page as canonical.

From Canonical Tags: A Simple Guide for Beginners

P.S. I Cant link Google references as they appear to be down at the moment


Suggestion, assuming that the page content does not change because of these referrals.

  • Use query parameters
  • In the back end, grab the query string
  • Then do a redirect to the main page without the query string.

No canonicals or 301s required.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.