5

I have a URL on website A that contains content from website B. On-site A at the moment we include canonical that shows site B and at the moment we do not include this URL into site A sitemaps.

Should the URL on-site A have content=" index, follow" though we exclude it from the sitemaps because of canonical?

3

What you describe is what we call a "cross-domain canonical". Google introduced it around ~2009-2010.

Typically, we use cross-domain canonicals to share content from another website that we really like and want to re-post it for our users. If it's another one of your sites, the way we deal with it doesn't change much.

Cross-Domain Rel=Canonical

rel=canonical indicates the authoritative or "original" page. Think of it like "re-tweeting". Google will always rank the canonical URL in organic results.

When used cross-domain, it carries a lot of benefit for the site being on the receiving end of the canonical. In your example, the authority of any/all links that the page on site A might get will get passed to site B.

Within the SEO community, we think it's similar to a 301 redirect - something greater or equal to 90% of the link authority and ranking signals will transfer from site A to site B.

The following would be best practice for your situation:

Revised instruction

  • On pages where you list a cross-domain canonical don't list any meta robots tag. Leaving it blank will default to the directive all which says There are no restrictions for indexing or serving.
  • Include the URL in site A's sitemap. Treat the URL on site A exactly the same as you would with any other page. Site A will not get any rankings benefit from hosting this content. By the very nature of the rel=canonical tag, Site B get's the credit.
  • On site A make sure your page has the same content (including images and media) and the same headline.
  • Include the same links within the content on site A as are in the content on site B - This will ensure that Google counts your rel=canonical correctly.
  • Link to the page on site B from the page site A. This is optional, but a cool thing to do.

Moreover, when you re-post content from someone else's website on yours, if you declare their page as the canonical you will not be punished for duplicate content.

Additional Reasoning

  • noindex follow is fine as well if you do not want page indexed, but follow is necessary for search engines to crawl/verify the canonical. This is how they will corroborate that the content is the same.
  • noindex nofollow doesn't help anyone. Here's why: under some circumstances maybe Google will want to show the result from site A. If both sites are in english, what if site A offers translation for non-english speaking users and Site B does not? What if site B's SSL cert expires? What if site B is super slow on mobile and site A is optimized?

So I digress if we are saying that site B is the canonical, site A will not get punished for duplicate content. Most likely Google will not index the page on site A because it is not the canonical noindex nofollow is redundant. The best route is to let Google decide.

2
  • "Site B will not get any rankings benefit from hosting this content. By the very nature of the rel=canonical tag, Site A get's the credit." - Haven't you got A and B the wrong way round here?
    – MrWhite
    Aug 14 at 0:51
  • @MrWhite oof...yep...answer is updated Aug 14 at 3:48
1

No, you should not use content="index, follow" on website A if you want it to not indexed by search engine. Because Google will also index it and show it in search result pages. But if you want the content of site A should be indexed by the search engine then you should use the index along with canonical. Here is the screenshot from Google Search Central enter image description here Here is article from Google Search Central Blog https://developers.google.com/search/docs/advanced/guidelines/duplicate-content

9
  • 1
    Google does use the word "also" there, but I take it to mean that you can "instead" ask other sites to use noindex. I don't think that it actually means that index with a canonical is problematic. Your interpretation could be correct, but I'm not sure. You get an upvote for finding this relevant documentation. Aug 14 at 0:04
  • <meta name="robots" content="noindex, nofollow" /> - The nofollow would not seem necessary here, or even desirable in most cases. Like @Stephen, I would also question whether noindex is really necessary when you already have a cross-domain rel=canonical tag as stated in the question - although I can see it as a belt & braces approach, particularly if there could be some doubt as to whether B is really canonical.
    – MrWhite
    Aug 14 at 1:08
  • Transparency is your best friend...this answer is wrong. Aug 14 at 4:21
  • I think it's your choice to get indexed the content by search engine on site A.
    – Khalil
    Aug 14 at 16:09
  • @Khalil Sure that could be fine, but no-follow defeats the purpose of the canonical. Aug 14 at 18:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.