Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

Possible Duplicate:
How can I get search engines to crawl my site and see a localised view of my data?

Sorry, this post became a bit loquacious - but I haven't found any satisfactory answers on the web thus far.

I've been developing a website for a while now, into which I've built support for multiple languages. All the terms, words and phrases are stored in a file in JSON format which is decoded and converted into an array. With this, it's easy to create the entire site in a different language. The website consists of 100% home-cooked code... no Wordpress, Drupal or any other framework, so none of the multi-language plugins for the aforementioned will work.

To switch language, the user clicks a link at the top of the page (positioned at the top, in reality, it's at the bottom of the HTML structure). Each page of the site has this menu, whereby a query string parameter is added. /hotels/index.html becomes /hotels/index.html?lang=fr. The script sees the lang parameter, changes the user's cookie and loads the appropriate JSON file, and they are returned back to the same page they were on without the lang parameter. This has been a very convenient way to switch languages, as the user remains on the same page, and the links are exactly the same. I built it this way after another site that required multiple language support where SEO was not a priority (an internal company-only program).

If a visitor comes to the site for the first time, the script looks at the HTTP_ACCEPT_LANG constant and loads the appropriate language and sets the cookie (instead of geo-locating, which takes longer, and I personally dislike - I live in Thailand, and even Google serves the Thai version even though I have my OS and browser in English).

My question is - I'd eventually like Google and other SE's to index the site in these other languages, but without the lang attribute in the URL. If the user uses Google in French or Russian, they get served the site in French or Russian, respectively. Since the URL /hotels/index.html would have a version in English (default), French and Russian, but still having the same URL, I assume this is a problem for Google and other SE's.

Can someone explain how Google indexes multiple-language sites setup in this manner? Would using a sub-domain work best to separate them? I do not wish to alter the way URL's are handled on the site, as I like to keep as few query strings as possible from being indexed.

On another note, I accidentally opened the site for indexing with the language links, and somehow, the redirects did not work properly, and I have links such as /something/index.html?lang=ru?lang=ru indexed in Google. I've detected these in the script and placed a 301 redirect to remove these lang parameters... will these bad links be replaced with the redirected URL's in Google over time?

share|improve this question

marked as duplicate by John Conde, Mark Henderson Oct 14 '10 at 22:01

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

2 Answers 2

from your description i would say ... your site is in no way SEO optimized for multiple languages and it will be a pain to trim it in this direction.

there is now this approach: http://googlewebmastercentral.blogspot.com/2010/09/unifying-content-under-multilingual.html

basically you can use it to push language switch URLs to google. but this is more than a fix for not SEO optimized sites than a real solution.

share|improve this answer
    
That's a good article, but as the author mentions, the use of that new solution is really for pages with duplicate content, and just localized secondary text (e.g. menus, headings, etc.). If you have fully localized articles, then you would want to use rel="alternate" but not canonical. –  Lèse majesté Oct 13 '10 at 10:20
    
hi lese majeste - the googlewebmaster blogpost above only mentions rel="alternate" in combination with canonical / http 301, they also outlined in google.com/support/webmasters/bin/… that rel=alternate is only be used for templates. i asked google here google.com/support/forum/p/Webmasters/… if it's wise to use rel=alternate without the canonical stuff, but never got an answere. do you have any experience with it? –  Franz Oct 13 '10 at 11:09
    
I believe that Google support page is incorrect, as it directly contradicts one of the primary uses of the alternate rel type as specified by the W3C here: w3.org/TR/html401/struct/links.html#h-12.3.3 and here: w3.org/TR/html401/types.html#type-links . Despite what Google says, using alternate with the hreflang attribute to specify target language is the standard way to link to translated content. –  Lèse majesté Oct 13 '10 at 12:52
    
@lese: very interresting, thx for the link. yes, it seems that the google supports page contradicts the w3 spec .... i updated the discussion here google.com/support/forum/p/Webmasters/… –  Franz Oct 13 '10 at 13:51

This is why it's a good idea to take a look at popular frameworks and CMSes before designing your own. These are very common problems that have been solved countless times by many different developers. Some of them are not obvious when you first sit down to design a CMS, so it helps to take a look at what's out there and see what some common problems/solutions/conventions are when building a CMS.

First off, basic content ought to be served statelessly from unique URLs. It's fine to save user language preferences by a cookie or session variable, but what you should do is redirect the user to the right URL/subsite to view the content in their preferred language. Otherwise, you're effectively cloaking your content from search engines (not in a malicious way, but in a self-harming way).

So, whatever primary language is used on your site, that version of a page would be:

http://foo/bar

Then the localizations will be:

http://ru.foo/bar
http://fr.foo/bar
http://de.foo/bar
etc.

You don't necessarily need to use clean-URLs so long as each page can be accessed from a unique URL without needing a cookie; so you could still do something like:

http://foo/bar?lang=ru
http://foo/bar?lang=fr
http://foo/bar?lang=de

But it's generally accepted that clean URLs are more user-friendly and thus also search-engine-friendly.

Now, implementing this change should be fairly easy on a conventional CMS or CMS framework, since most have a routing engine that allows one to easily add custom routes to content. If you don't, and your URLs are not decoupled from your application architecture, then it may take more work.

share|improve this answer
    
Thanks. I've decided to go with subdomains for the different languages. What do I do regarding canonical if I want the French translated site (fr.foo.bar) to be indexed in the French Google, for people searching in French? I would not want to set the canonical link to the English version for French searches. –  Mike Oct 13 '10 at 12:15
    
@Mike: As long as there is no more than one unique URL for each version of the page, you don't need canonical--since they're technically not duplicate content. Otherwise, Google might not index your non-canonical pages. Just use rel="alternate" and hreflang="ru" in the <link> tags. –  Lèse majesté Oct 13 '10 at 12:56

Not the answer you're looking for? Browse other questions tagged or ask your own question.