1

My site has multiple language-country combos for which I have implemented HREFLANG. We have a few language-region combos that show up as errors in Search Console. What's the best thing to do here?

For example, mysite.com/us/en/page1 is easy, as is mysite.com/es/es/page1. These are correctly HREFLANG tagged.

However, we have some catch-all codes like mysite.com/xf/en/page1 where "xf" equals "The Rest of Africa". Obviously no country codes for this.

So should I...

  1. Robots.txt - Disallow url's that contain /xf/ since we don't want Google to index them?
  2. Add some other kind of HREFLANG tag for them?
  3. Leave them alone and just ignore the HREFLANG errors in Search Console?
  • You can't just make up regions and expect targeting to work. – Stephen Ostermiller Feb 19 '18 at 19:38
  • Right, I didn't make them up myself, just trying to figure out the best way to handle these url's from an SEO perspective. – badghost Feb 19 '18 at 20:54
1

If you cannot determine what language /xf/ should be because of multiple languages then you should remove the hreflang tag since you cannot define the language. This should remove the hreflang errors that you're seeing in Webmaster Console.

If you cannot remove the hreflang tags from the /xf/ page then you can disallow access to that directory in robots.txt as you suggest. Bots won't be able to crawl those pages and as a result Google will not index them.

You can also add

<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">

to your head tag. This will prevent Google from indexing the pages in the /xf/ directory as well as prevent Googlebot from crawling further links to /xf/ within that page which will help maximize the use you get out of Google crawling your site.

Using the NOINDEX NOFOLLOW tag will allow Google to crawl those pages when it discovers them from links. The potential benefit of adding this meta tag is that permitting Google access to the page helps Google know what your site is about and indicates to Google that you're not hiding any malicious content on your site that is blocked by bots. I can't guarantee that this will help your rankings, but it couldn't hurt.

The benefit of blocking bots in your robots.txt file is that since Google will only crawl your site a limited number of times a day, you know that some of those crawls won't be wasted on your /xf/ pages that you have notified to Google not to index.

Overall, I recommend that you consider trying to remove the hreflang tag from your /xf/ directory while allowing Google to index those pages. Unless of course you don't want them indexed altogether.

  • Thanks, this was very helpful. I decided to remove the HREFLANG tags from these pages and disallow them in Robots.txt. They aren't high volume pages, and we have plenty of other pages in the same languages on the site that Google will index, so it is probably best just not to index them since they serve an ambiguous purpose. – badghost Feb 20 '18 at 16:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.