Tell me more ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.
up vote 2 down vote favorite

I have a games site and I'm writing an API that will pull relevant Q&A from my copy of the SE data dump. I will abide by all licensing rules.

That said, I'm hesitant to do this because Google will penalize my site for serving duplicate content.

Is there a legal way to serve data-dump content, while abiding by licensing terms, without incurring search engine penalties? I want to serve this content, but I don't want to compromise the main function of my website.

I plan to use Google API to translate content to my native language. I will not serve the content in English.

share|improve this question

3 Answers

up vote 1 down vote accepted

From Google's article titled 'deftly dealing with duplicate content':

"...our algorithms won't view the same article written in English and Spanish as duplicate content."

If you're reproducing licensed content in a different language from the original, Google will see your content as original.

If you're reproducing content in the same language, you should link back to the source to ensure that Google ranks that higher than your version. The content reproduced on your site won't incur any 'duplicate content penalties' that effect your whole site; it will simply mean that those particular same-language pages will be seen lower in search results than the original version.

For reliable human translation, I use and recommend mygengo.

share|improve this answer
up vote 2 down vote

While I would strongly advise against using google for translations (unless you plan on editing them afterwards), you will very likely not be penalized for duplicate content since google will not check content against translations to other languages for indexing.

share|improve this answer
thank you. Is there any reason why you advise against using google for this function? – Mohamad Jul 14 '11 at 12:28
I don't advise against using google to translate, if you will edit the result. most automatic translations will significantly differ from content written in the target language, however. Google translations rarely make sense on their own, they're good enough to figure out what it all means, not good enough to have any sort of flow in the translated text (let alone jokes, much less puns, even a creative writing style). – xyious Jul 14 '11 at 13:24
up vote 1 down vote

If you're worried, just make sure you set up a robots.txt file on your server to tell Google (and other search engines) not to index the data dump content. Your users will be able to access the information, but Google will ignore it.

A search on "robots.txt" will yield all the details of how to set this up.

share|improve this answer
That's not possible, I'm afraid, since the content will be displayed along with other information. – Mohamad Jul 15 '11 at 18:45

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.