4

So, we have an international site with multiple subdomains for each language. We have hreflang markup in place (+ sitemaps), but we also detect where users come from and redirect them to appropriate sudbomains.

What's important is that we're setting a cookie with following logic:

  • If EN user with no cookie comes to fr.example.com URL > don't redirect him
  • If FR user with no cookie comes to example.com > redirect him to fr.example.com
  • If FR user with EN cookie comes to example.com > don't redirect him
  • If FR user with or without cookie comes to de.example.com > don't redirect him

If users pick languages from a language menu on the site, language always changes to the one they select. But if a user sets a FR language (and comes to a FR subdomain), if he then pastes an EN URL in browser's address bar, he's being redirected to FR subdomain.

First of all, don't blame me for that logics and don't say that automatic redirects are bad for UX; it wasn't me who made it like that, and I'm just trying to find if it may be harmful to our SEO because it may mislead Google Bot who may come from various international IPs.

What I suspect, is that if Google Bot comes to the site, it crawls the pages from links and hreflang links it finds, and if he once visits a FR page and then he tries to visit EN page not though our language selection menu, he's getting redirected to FR page and this might probably hurt the SEO and our rankings.

Questions:

  1. Is this redirect scenario bad for SEO and our SERP rankings?
  2. If YES, how exactly it is hurting us?
3
  • Fantastic question!! Interesting new info as a result. Thanks!!
    – closetnoc
    Nov 9, 2015 at 15:52
  • @closetnoc thank you. I follow your answers on ProWebmasters, by the way, you write lots of valuable info. I'd like hearing your input on this matter too.
    – CamSpy
    Nov 9, 2015 at 20:23
  • Thanks for the complement! I do not know anything about websites with different languages. I have never had to deal with them. I also semi-retired almost 14 years ago!! I up-voted @w3d answer because it seems like a damned good start.
    – closetnoc
    Nov 10, 2015 at 0:12

2 Answers 2

3

Your redirects for users without cookies are a bad idea. Googlebot never sends cookies, but Google has data centers around the world. When Googlebot crawls from a data center in France, it won't be able to get to your English website and your English website could get dropped from the Google index.

Location based redirects are also problematic for real users. Somebody who speaks only English who is vacationing in France will be unable to use your site. IP address locations are often also wrong 10% of the time. You should not use them to make automatic decisions for the user.

Instead you should place notices on your site when you suspect the user is in the wrong place. Such messages should be in the language that you have detected for the user. (In this case French):

You appear to be located in France but this is the English website. Click here to view this page in French.

or

You have a French web browser that is sending a FR-fr Accept-Language header but this is the English website. Click here to view this page in French.

2
  • we have canonical tags and hreflang tags on all international pages, so even if Google visits FR pages, it will know about all others
    – CamSpy
    Dec 10, 2015 at 8:16
  • 1
    Knows about and "is able to crawl" are very different things. If they can't crawl it, Google won't put it in the index. Dec 10, 2015 at 10:55
0

A couple of key points about Googlebot probably answers your questions...

  1. Googlebot does not use cookies. It is completely stateless. Every visit is a "no cookie" visit.

  2. Googlebot predominantly visits from the US. Googlebot is always(?) seen as a US visitor. So, any automated GeoIP redirection has potential to cause problems for SEO.

    EDIT: (Jan 2015) It does appear that Googlebot can now crawl using non-US IP addresses: Locale-aware crawling by Googlebot and Crawling and indexing of locale-adaptive pages

If EN user with no cookie comes to fr.example.com URL > don't redirect him

If by "EN user" you also include "US user" then you should be OK.

if Google Bot ... visits a FR page and then he tries to visit EN page not though our language selection menu, he's getting redirected to FR page

Googlebot (and most search engine bots) don't work like that. They are stateless (as mentioned above). They first build a bunch of URLs, having found them by whatever means. Then they index them, individually, page by page, in no particular order. No state is passed from one visit to the next (the same as a user clicking on a search result for the very first time).

6
  • Google is now using cookies when crawling. You can check these out luckyboost.com/google-is-now-using-cookies-when-crawling and goo.gl/mukz8V Also, Google bot started crawling from non US IP addresses: support.google.com/webmasters/answer/6144055?hl=en "Geo-distributed crawling: Googlebot appears to be using IP addresses based outside the USA, in addition to the longstanding IP addresses Googlebot uses that appear to be based in the USA." goo.gl/sUZNM2 Please consider these facts, as your statements seem to be obsolete.
    – CamSpy
    Nov 9, 2015 at 12:22
  • The article you link to regarding "cookies" is not conclusive at all and could simply be put down to a "glitch" with the website. It does not make sense for Google to maintain state between visits with cookies. If Google indexed a page based on the value of a cookie and then a new user (with no cookies) follows a link in the SERPs for that page, the user is going to see a different page. This is a bad user experience and something that Google actively tries to avoid.
    – MrWhite
    Nov 9, 2015 at 13:28
  • However, there is one exception that I've seen when Googlebot does send a Cookie. If a page contains an iframe. If the parent document sets a cookie then that cookie is also sent in the request to the document contained in the iframe (if the cookie is valid - same domain, valid path, etc.). But this is correct behaviour. It is the same as what a new user would see, since it is "one page" as far as the user is concerned.
    – MrWhite
    Nov 9, 2015 at 13:32
  • Thanks for the link regarding the "Locale-aware crawling by Googlebot", using non-US IP addresses - I was not aware of that. However, how widespread are these other IP addresses - the documents don't appear to say. Is there a French IP address? Worth examining your own logs to determine where Googlebot is crawling from.
    – MrWhite
    Nov 9, 2015 at 13:36
  • Google bot should at least use cookies to test sites against cloaking, no?
    – CamSpy
    Nov 9, 2015 at 14:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.