2

At this moment our website both uses the language in the url like "en" and localizes the url itself ("books" in english and "boeken" in dutch). Because of the history of making our website multilingual we have a system that takes the browser language for the localization if the url doesn't contain a language like "en".

This means:

  • www.test.com/books = browser language
  • www.test.com/en/books = english language
  • www.test.com/boeken = browser language
  • www.test.com/nl/boeken = dutch language

Now for the sitemap this makes it a little troublesome for me because which hreflang is used for which url?

1) The first thing I thought of was using x-default for all urls that get the language of the browser.

<url>
    <loc>http://www.test.com/books</loc>
    <xhtml:link rel="alternate" hreflang="x-default" href="http://www.test.com/books" />
    <xhtml:link rel="alternate" hreflang="x-default" href="http://www.test.com/boeken" />
    <xhtml:link rel="alternate" hreflang="en" href="http://www.test.com/en/books" />
    <xhtml:link rel="alternate" hreflang="nl" href="http://www.test.com/nl/boeken" />
</url>  

But as you can see we now got 2 times x-default.

2) Another solution I thought of was just use the localization of the url to determine the language like:

<url>
    <loc>http://www.test.com/books</loc>
    <xhtml:link rel="alternate" hreflang="en" href="http://www.test.com/books" />
    <xhtml:link rel="alternate" hreflang="nl" href="http://www.test.com/boeken" />
    <xhtml:link rel="alternate" hreflang="en" href="http://www.test.com/en/books" />
    <xhtml:link rel="alternate" hreflang="nl" href="http://www.test.com/nl/boeken" />
</url>  

But now we got 2 of each language for the same page.

3) The last solution I thought of was removing links without a language in the url (except for the homepage, which will still have an x-default) like:

<url>
    <loc>http://www.test.com/en/books</loc>
    <xhtml:link rel="alternate" hreflang="en" href="http://www.test.com/en/books" />
    <xhtml:link rel="alternate" hreflang="nl" href="http://www.test.com/nl/boeken" />
</url>  

But for this solution I need to put 301's at pages that are "deleted" and also need to change the system to 301 to the right page. Although the last point isn't really a problem I'm kind of worried that I will lose some of the "seo points" with a 301. (When we changed our domain in the past we had a bad experience with the 301 of our old domain)

What do you think would be the best solution for SEO? Or do you have any other suggestions or solutions I haven't thought of.

  • 2
    I'm a little confused why there are 2 URLs for each language? Why not just have www.test.com/books for English and www.test.com/nl/boeken for dutch. – Max Mar 2 '15 at 10:37
  • Can you clarify what this does: we have a system that takes the browser language for the localization if the url doesn't contain a language like "en" – dan Mar 2 '15 at 10:37
  • @dan there is a user which goes to /books enen though the url is english there is no language (like /en/books) in the url. So what our system does then is take the browser language of the visitor and use that url to localize the page. So if your browser is english the page will be english, if your browser is dutch the page will be dutch. – Julian Mar 2 '15 at 12:40
  • Relying on the language reported by the browser is not recommended. It is wrong over 10% of the time. The most common reasons are: 1) Users around the world tend to install English browsers because those are the easiest to download. 2) Users traveling may be guests on computers set to other languages. Instead offer a suggestion that the user should change languages. – Stephen Ostermiller Mar 3 '15 at 20:39
3
+100

I think that what you'll want to do is change the way your system works. If no language is specified in the URL, but you detect the language via the browser settings, then your site should redirect the user to a URL with that language in the path. This is for two reason:

1) Duplicate content. Spiders/Bots will come to test.com/books, test.com/booken, and test.com/en/books URLs, and see the same content at all 3 URLs if the Spider's Accept-Language header is English. For example, Googlebot comes and crawls test.com/books, test.com/booken, and test.com/nl/booken with Accept-Language header of Dutch, Google will see the exact same content at all 3 URLs. The hreflang x-default will not help you, because in your case the "default" is not in fact a default language, but detects which language to display.

2) It solves your problem about not knowing which URL applies to which language, and makes the language hierarchy of your site much more easily understood to bots (and humans). Thus it will make it much more likely that users land on the page that they actually want, without any redirects.

You could go about this in 2 ways. The way that I do with one of my clients is we have the test.com/books as the English variant (and default), however if the user comes to that URL with a Accept-Language that is non-English, and one that we support, we 302 redirect them to test.com/nl/booken. If they're English, we leave them at test.com/books (we don't specify English in the URL, basically. This was not my choice, but my bosses who just didn't want to do it "right" for his own reasons).

Alternatively in your case, you could also redirect English speakers to test.com/en/books which would make it even more explicit to Spiders which URL is which language, and then leave the non-language URL as the only one that would do language detection. This way Google would know that English speakers go to test.com/en/books, Dutch speakers go to test.com/nl/booken, and if the searcher is working in neither of those languages, they go to test.com/books (you really should try to have only one "default" URL) and Google lets you sort out the language on your website.

Then to clean up your old Dutch URL's and retain any link juice from those, you'd probably want to 301 redirect test.com/booken to test.com/nl/booken

  • We choose option 3 now. And like @Mike said we use 301 from the urls without a "language" to the url with a language. The only page that detects the browser langauge now is the home page. Also like you said to fix the "duplicate" content we have alternate links from each page to all the other "same" pages with different language. We don't use accept-language at this moment but I'll look into that. Thanks. – Julian Mar 4 '15 at 9:27
  • The reason I suggest a 302 instead of 301 is that you don't want Spiders to get confused thinking that test.com/ is permanently redirected to test.com/nl/, because if the Spider only crawled in that one language, it would assume that your homepage had moved to test.com/nl/ and may de-index your main homepage at test.com. The accept-language header is the same thing as "getting the browser" language as you said in your question. It's just the header that the browser sends with the request, so you're already using it :) But glad this made sense to you. – nathangiesbrecht Mar 4 '15 at 17:18
0

I want to add that the redirects should be 301's not 302's because 302 stand for "found", but 301's do a permanent redirect and display the correct URL in the address bar for people to be able to bookmark the page in the correct language.

Also, with the accept header, you need to take the language with the highest preference from the list of languages.

See this RFC 2616 - Section 14 - Header Field Definitions
http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html

It states the following:

The Accept-Language request-header field is similar to Accept, but restricts the set of natural languages that are preferred as a response to the request. Language tags are defined in section 3.10.

...

Each language-range MAY be given an associated quality value which represents an estimate of the user's preference for the languages specified by that range. The quality value defaults to "q=1". For example,

Accept-Language: da, en-gb;q=0.8, en;q=0.7

would mean: "I prefer Danish, but will accept British English and other types of English."

So your best bet is to write server-side code that takes the value of the accept-language header as input strip the preferred name from the input and then issue a 301 redirect to the proper page based on the code.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.