Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

I have a "classified advertisement" website that user can submit their advertising posts on it. Each post will be valid for 3 weeks. After then the post will become "expired". Expired posts will not be physically removed, we just remove the link between the website and the post. So users cannot find the expired posts from the website, but they can find the expired posts if they type in the URL of the post directly.

After the post become expired, users can create another new post with content same as the expired post so as to extend the advertising period of the posts. I have to allow this.

My problem is, google will index the expired posts and the new posts, and is reporting duplicated content found on the website.

So what's the best practice to remove the expired content from search engines?

share|improve this question
add comment

1 Answer

up vote 1 down vote accepted

Just send out a robots no-index HTTP header or use the equivalent meta tag. This will tell Google to remove the page from their search results:

X-Robots-Tag: noindex

or

<meta name="robots" content="noindex">

You can also let Google know when a page will expire:

X-Robots-Tag: unavailable_after: 7 Jul 2007 16:30:00 GMT

or

<META NAME="GOOGLEBOT" CONTENT="unavailable_after: 25-Aug-2007 15:00:00 EST">
share|improve this answer
    
are there any similar things (unavailable_after) for bing & yahoo? –  LazNiko Aug 30 '11 at 3:55
    
This should work for Bing which powers Yahoo, too. –  John Conde Aug 30 '11 at 3:59
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.