2

I followed the recommendation of Google concerning multilingual website, I have:

  • www.example.com/fr in french
  • www.example.com/en in english
  • www.example.com automatically redirects (header) either to www.example.com/fr or www.example.com/en.

In my sitemap.xml, I have not mentionned www.example.com but instead the two subdirectories, still following Google on sitemap policy: 

<url>
  <loc>http://www.example.com/fr/</loc>
  <xhtml:link rel="alternate" hreflang="en" href="http://www.example.com/en/"/>
  <xhtml:link rel="alternate" hreflang="fr" href="http://www.example.com/fr/"/>
  <changefreq>daily</changefreq>
 <priority>1.00</priority>
</url>
<url>
  <loc>http://www.example.com/en/</loc>
  <xhtml:link rel="alternate" hreflang="fr" href="http://www.example.com/fr/"/>
  <xhtml:link rel="alternate" hreflang="en" href="http://www.example.com/en/"/>
  <changefreq>daily</changefreq>
  <priority>1.00</priority>
</url>

Problem is that in Google, the main results is www.example.com. For many websites, Google displays for the first results .com/fr or .com/en depending our location but not juste .com.

I was thinking of using Webmaster Tools and try remove www.example.com but I'm not sure since www.example.com is not really a sitelink and I'm afraid it will juste remove my site from Google. Do I miss something? Or maybe I should just wait?

| improve this question | | | | |
  • 2
    How are you determining which language to redirect towards? Is it based on geographic IP address or Accept-Language header? How is Googlebot treated? It almost always crawls from the US and it doesn't send a Accept-Language header. – Stephen Ostermiller♦ Jan 20 '14 at 11:08
  • 2
    Did you used to have on of the languages hosted in the root folder? If so, how long ago was that? – Stephen Ostermiller♦ Jan 20 '14 at 11:09
  • 1
    Very good points : If the user doesn't browse .ocm/fr or .com/en but just .com, I redirect using the ip adress and making a call to an online api. Once I get the answer, I redirect to /fr (if france) or /en else. Now, 2 months ago, I had only one url : ".com" and language of the this page was also found using ip to set a session variable. At this time, my website was indexed in english (i guess because googlebot crawl from us). But now (and for 2 months now), i have different urls for the two languages. – Mansur Khan Jan 20 '14 at 13:24
3

You need to include the "/" URL in your Sitemap file and specify it as the hreflang "x-default". That way, Google will know that this is a part of your set of pages, and will be able to handle it appropriately. Note that hreflang is a signal, and not a directive for Google, so it might be that the "/" URL still occasionally shows up.

| improve this answer | | | | |
2

Your general setup sounds fine. A few things to check:

  • When you do your geo-IP redirects, ensure that they are the "301 permanent" variety. If you are using 302 temporary redirects, Google will be hesitant to change the URLs
  • Test Googlebot specifically.

I tend to use a command line program like curl to test these items. Make sure the information you get back looks similar to the following:

curl --head --user-agent 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)' http://example.com/
HTTP/1.1 301 Moved Permanently
Location: http://example.com/en/

If you need to make any additional changes, it may take Googlebot a week to realize the home page has moved and a month or more to get to all the pages on the site.

| improve this answer | | | | |
  • Ok. For now, it's redirected using a header location request, so it's 302 by default. I can set it to be 301. What do you mean by "test googlebot specifically" ? You mean, in my php code, to detect if it's googlebot and do specific actions in that case, or is the action in robots.txt ? – Mansur Khan Jan 20 '14 at 21:34
  • Use a Googlebot user agent when contacting the server. That is why I use the --user-agent flag with curl. – Stephen Ostermiller♦ Jan 20 '14 at 21:41
  • Curl is sending me 406 answer : "LiteSpeed technology is not responsible for administration and contents of this web site!" – Mansur Khan Jan 21 '14 at 15:58
  • 406 is a very unusual response code. It would be worth getting to the bottom of what is generating it and why. stackoverflow.com/questions/14251851/… – Stephen Ostermiller♦ Jan 21 '14 at 16:18
  • Actually, "curl --head example.com" gives 406 code. your request, gives : curl: (6) Could not resolve host: (compatible; curl: (6) Could not resolve host: Googlebot curl: (1) Protocol +http not supported or disabled in libcurl HTTP/1.1 301 Moved Permanently Date: Tue, 21 Jan 2014 16:28:12 GMT Server: LiteSpeed Connection: close X-Powered-By: PHP/5.3.23 Set-Cookie: PHPSESSID=6d45be72d9847aaa8112d2d51efcec5c; path=/ Expires: Thu, 19 Nov 1981 08:52:00 GMT Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-che Pragma: no-cache Location: /fr/ Content-Type: text/html – Mansur Khan Jan 21 '14 at 16:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.