Possible Duplicate:
What is duplicate content and how can I avoid being penalized for it on my site?

I have a nature photography site with multiple types of photo galleries. Each photo and associated caption on my site appears in several galleries. For instance, a photo of a goldfinch that was taken on a trip to New Mexico in 2008 will appear in the "goldfinch.php" gallery, in the "finches.php" gallery, and in the "New_Mexico_2008.php" gallery. This duplication is useful for my site visitors - User A may want to see goldfinch photos, whereas User B wants to see photos from New Mexico - but I am concerned about the SEO implications.

The typical suggestions to deal with duplicate content, such as 301 redirects and canonical tags, probably won't work in this case, because the page content is substantially different (ranging from ~1% to ~90% duplication, depending on the specific example chosen).

The obvious solution to me would be to edit robots.txt to only allow search engines to crawl one type of gallery - for instance, if they crawled only the galleries organized by species(e.g. goldfinch.php), all the photos on my site would be found exactly once. However, the Google content guidelines recommend against blocking crawler access to duplicate information.

Should I go ahead and use robots.txt anyway? Or is there a better solution?


Do not block crawling of your partially duplicate pages via robots.txt; this is likely to harm your indexing situation more than any of the alternatives.

As you describe them, your pages are unique combinations of photos, so the pages are distinct. Search engines are quite used to dealing with overlapping content and generally don't penalize for it. Your pages are similar to product search results pages on a retail site.

Make sure you have appropriate ALT text in the IMG tags for each photo. I would go ahead and use the same, good, alt text for the same photo accessed via different pages. You may also want to use the title attribute in the IMG tag.

  • + 1 Totally agree – Dave Jul 18 '12 at 14:36

The pages are not duplicated in the truest sense, you have just have some duplicated content. If possible, make sure the ALT tags and title tags are different (as per mgKrebbs suggestion).

The key question is how much text is on the page and how much that differs. Make sure the titles / heading tags etc are unique for the page.

Not the answer you're looking for? Browse other questions tagged or ask your own question.