1

The use of rel="canonical" is often required, in order to deal with the issue of duplicate content. Simply put, it 'tells' search engines which is the preferred URL version.

To achieve this, the following example code needs to be entered within the <head> tags:

<link rel="canonical" href="https://www.example.com/product/product-name/" />

Simple enough.

My question is really orientated on how this code is treated by search engines. Which scenario best describes said treatment:

Scenario A:

Search engines will read the duplicate page, in its entirety, and then look at the rel="canonical" link. Upon reading the Canonical link, it will then decide on whether the pages are duplicates or not and index accordingly.

In other words, the use of rel="canonical" is merely a suggestion to search engines, for their own discretion.

Scenario B:

When a search engine Bot arrives onto the page, it will start to read the source code, starting at the top of the page, with <!DOCTYPE html>. It will then work its way down the source code. In the event it comes across the rel="canonical" entry, it will stop reading the source code and crawl through to the Canonical link, picking up where it left off, as to read the rest of the page, to be indexed.

This scenario, indicating that the rel="canonical" entry acts as a redirect for Bots.

Improve this question

2 Answers 2

Reset to default
3

It is closer to your Scenario A - the canonical tag is a "suggestion".

However, duplicate content (ie. two URLs that return similar/same content) are still both indexed. Except that Google will try to return just one or other of the URLs in the SERPS (as otherwise it's generally considered a bad user experience). Without the rel="canonical" tag Google decides which URL to return. This is the duplicate content "problem", as it might not be the page that you want to be returned in the SERPS.

The rel="canonical" tag resolves this "problem" by putting the developer back in control. You can now decide which URL (of the duplicates) you would prefer to be returned in the SERPs. ie. the canonical URL.

However, if the pages are not deemed to be duplicate/similar in content then the rel="canonical" tag will likely be ignored. Pages can only be deemed "duplicate" if they are already indexed.

As noted in the Google Webmaster Central Blog post from Feb 2009:

Is rel="canonical" a hint or a directive?

It's a hint that we honor strongly. We'll take your preference into account, in conjunction with other signals, when calculating the most relevant page to display in search results.

Improve this answer
5
  • Thanks for your answer. I wasn't aware that search engines would still index the 'duplicate' content; just not return it in the SERPs. I guess it makes sense. My initial thought was that a search engine would drop the duplicate content from their indexes. Close call between yours and Closetnoc's. In end, chose the first answer posted.
    – Craig
    Commented Jun 1, 2018 at 18:39
  • @Craig Both are right. For a search engine to know anything about a page, it must be indexed. Google indexes everything before making decisions. And it can take it's time making decisions. Some pages are dropped from the index if a severe penalty is issued. This is rather rare. Otherwise, almost all conditions are handled using SERP filters.
    – closetnoc
    Commented Jun 1, 2018 at 22:21
  • @closetnoc ... I guess that makes logical sense. I guess the process is: We send search engines our pages. They then archive everything we send it, within their indexes. When a search query is then performed, the search query gets put through the relevant 'algorithmic filter(s)', which then puts the user through to the relevant SERP. That's my logical take on it; though I am sure it is likely to be a little bit more complex than that. :-)
    – Craig
    Commented Jun 1, 2018 at 22:52
  • @Craig Bingo! You said it perfectly. Cheers!!
    – closetnoc
    Commented Jun 2, 2018 at 0:17
  • "search engines would still index the 'duplicate' content; just not return it in the SERPs." - you can often see this by doing a site: search and comparing it with the normal search results. site: searches should return all indexed pages, including duplicates, redirections and non-canonical pages, etc. (Note that site: searches are limited in the number of results returned, but you can narrow this search to just a part of your site.)
    – DocRoot
    Commented Jun 2, 2018 at 11:13
2

Make no mistake. Search engines do not process parts of a page then suddenly do something else.

When a page is discovered, it is fetched, stored, rendered, analyzed, and indexed. If a link is found in a page, the SE searches the link index to see if it exists. If so, then nothing. If not, then the link is added to the fetch queue where it is fetched, stored, rendered, analyzed, and indexed.

Canonical links are treated the same way. If the target URL does not exist, it is added to the fetch queue where it is fetched, stored, rendered, analyzed, and indexed.

Get where I am going?

There are quite a few criteria to determine the original content. This is not easy to do and sometimes fails. This is why canonical links are important. However, from a programming point of view, canonical links may not exist or even be correct. Think about it this way, I am a spammer. I steal content, copy it to 100 sites, create 99 canonical links to one spam page, who is to say that 99 canonical links are of value when the original content is not one of the 100 pages? As a programmer, how do you one, determine original content, and two, determine if a canonical link is valid? Google has not abandoned its original algorithms since they are critical to determining if a canonical link is valid. However, even the original algorithms can be wrong. On the web, there is a giant hole in the construct where a pages creation date cannot be queried from the web server, only the modified date. The inception date is as close as it comes. This is the date any page was first fetched. This can be a problem where an unauthorized copy is found before the original. Fortunately, there are some tricks that help determine the original content. For example, the modified date can be used. This is because a pages modification date will match the creation date if unmodified.

Canonical links are signals. Where they can be determined to be correct, they hold value and can promote the original content. Where they can be suspect, the may be ignored completely.

Your use of the term suggestion is perfect. This is the term Google uses for such scenarios.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.