5

HTTP offers Content Negotiation (by reacting on the Accept-Language request header) to deliver translated representations of a resource under one URI. In January 2015 Google finally implemented this feature and calls it "Locale-aware crawling by Googlebot".

Could anybody get different representations under one URI successfully into the index?

I'm trying this for http://www.mixcloud-downloader.com/ which can additionally to its default English representation serve a German representation. This site is since one month in Google's index. Additionally to the Vary: accept-language response header, I also added an indication which languages are supported:

<link rel="alternate" hreflang="en" href="http://www.mixcloud-downloader.com/" />
<link rel="alternate" hreflang="de" href="http://www.mixcloud-downloader.com/" />

If I would search for something from the German representation which is unique in the internet (e.g. "Mixcloud Internetadresse" with quotes) google.de doesn't find it.

What do I have to do, to make Google successfully aware of a translated representation under one URI?

Edit: I noticed that my application was sending a bogus Content-Language header, which might have confused Google. I fixed that on 2017-01-19. Previously it did respond with e.g. a zh-CN header and English content (if the client requested zh-CN).

  • The code looks correct, how have you created the sitemap.xml? – Karan Shah Jan 12 '17 at 11:49
  • 2
    There's no sitemap.xml. Also from this document I don't see a reason how a sitemap might help. – Jessy James Jan 12 '17 at 11:50
  • The Accept-Language header is notoriously unreliable. Non-English users regularly use English browsers because they are the easiest to download. Not to mention problems using somebody's computer when traveling. The header should at best be taken as an indication that the user is in the wrong place with something on page to let them switch. It should not be used as the sole determinant of the language of the page. I recommend sticking with something in the URL to indicate language. – Stephen Ostermiller Jan 12 '17 at 22:13
  • When you searched for the German text did you have your Accept-Language header set to "de"? I wouldn't expect Google to return results to you that you couldn't actually get to without the proper settings. – Stephen Ostermiller Jan 12 '17 at 22:18
  • Yes I did set my client's header to "de". And guess what, now Google finds this question as the only result when searching for "Mixcloud Internetadresse". That was fast. – Jessy James Jan 12 '17 at 22:32
1

In the Google's documentation you rely Google mentions to recommend to use separate URLs for each language version, considering sensitivity and error vulnerability of local adaptive setup.

Further, in this documentation, Google mentions

You can help Google determine the language correctly by using a single language for content and navigation on each page ... Keep the content for each language on separate URLs

If you stay with it come hell, you should rely on both of Accept Language AND geolocation to point visitors and Googlebot to correct language version. Because your troubles begin not with Accept-Language, but with its absence.

Googlebot comes not often with defined locale parameter, and you can't constrain it. If so, visitors of your site like Googlebot, without Accept-Language, get, as now, only english content served, not dependently of their location.

You should enrich your setup with the second check routine, for the visitor's location, somehow like this:

- visitor comes
- check 1: Accept-Language yes/no? If yes - serve matching content, if no:
- check 2: geolocate visitors IP-address.
           If Germany/Austria/Swiss- serve german, if another country - server english.

But even with this setup you will be never sure and can never actively influence that Google indexes content in all languages. The most possible variant would be, that Google gets english content to 100%, and german... my bet for best case - 30%.

Your site has a handful of pages - make just unique URLs for each language and you haven't this headache.

PS: On answering your question found a cool tool for checking exactly the question's subject: https://technicalseo.com/seo-tools/locale-adaptive/.

| improve this answer | |
  • Thank you for your opinion. I'm aware on Googles recommendation. However Google states it does use the Accept-Language for crawling. My questions are "Is there any existing site which does Content negotiation and got into the index" to ultimately answer "What does my site wrong". I don't see how you answer any of this. – Jessy James Jan 22 '17 at 18:29
  • I'm currently evaluating your recommended tool. Interestingly it states "Vary:Accept-Language No", which is not the truth. Not so cool that tool. – Jessy James Jan 22 '17 at 18:35
  • your site must not mandatory do something wrong to not to be indexed. Its just like shit happens in more then 50% of cases - thats the cause why Google recommends to use unique url for unique language - doing so you actively and already guaranted avoid the shit will not happen. To answer your question in short - no, i never seen and read about sites, which serve content in different languages on the same url and is indexed enough good to drive this schema. – Evgeniy Jan 22 '17 at 19:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.