Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

This is a follow up question from here: Duplicate title tags due to translations

Since asking the question I have removed the duplicated content that was being generated by language/translations on the website in question.

In the answer I was told that I should return 410 errors for the removed duplicate pages, use robots.txt to disallow crawling of duplicate pages, and to use the Fetch as Google feature to re-trigger crawling.

My question is now that I've actually removed the pages (I've removed all of the translation options), will this self-correct when Google next crawls the site and notices that the links are gone? Do I still need to do the things recommended, given that the duplicated content is now gone?

share|improve this question

1 Answer 1

up vote 1 down vote accepted

Will this self correct when Google next crawls the site and notices that the links are gone? Do I still need to do the things recommended, given that the duplicated content is now gone?

It would certainly be a good idea to make sure that your website is returning 410 errors for URLs that were indexed but no longer available, as other error codes might result in search engine robots checking again in the future. Disallowing them in your robots.txt will also cut down on future requests by Googlebot and other robots.

Specific to Google, you can also request an entire directory to be removed, if the translations were served in a directory. See this for how: Remove your entire site or a directory

Using the Fetch as Google tool will help to trigger re-crawling, but it can still take up to a few weeks for Google to drop the removed URLs. You can check for them in Google with the following search operator: site:domain.com/url.

In summary, doing the above will help to get the duplicate content removed quicker and potentially improve your SEO.

share|improve this answer
1  
Thanks again Dan –  Doug Firr Sep 28 '13 at 20:47

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.