I am working on a simple Wordpress blog where when an article is published, it appears on ALL these pages:

  1. Homepage - Headline(clickable) + 3-line summary

  2. Parent category page - Headline(clickable) + 3-line summary

  3. Child category page - Headline(clickable) + 3-line summary

  4. Author page - Headline(clickable)

  5. sitemap.xml

A good example of what I mean is:


The "Worth A Thousand Words" article is available on the homepage.

But it's also available on the category page: http://wp-themes.com/?cat=1

And finally, it's also available on the author apge: http://wp-themes.com/?author=1

Does it makes sense to let Google crawl http://wp-themes.com/?author=1 knowing that all links on that page were already crawled elsewhere?

I've been in fact told that I should add all author pages to my robots.txt, under disallow, so as search engine bots do not spider /author/*(or anything with ?author for the above example) since all links on these pages are indeed available elsewhere.

Is this a good approach or maybe rel=nofollow is better, or maybe I shouldn't worry about this at all?

  • What is the issue? Aug 21, 2014 at 19:56
  • Apologies - post now edited.
    – WPRookie82
    Aug 22, 2014 at 7:18
  • 2
    Ok, if you are worried of duplicate content, then setting canonical links is the solution in your case. Don't fiddle with robots.txt or nofollow. This would not be the right to solve your issue. Aug 22, 2014 at 7:23
  • I should have in fact mentioned that canonical links was another area I checked out - will re-check again.
    – WPRookie82
    Aug 22, 2014 at 14:38

1 Answer 1


Googlebot is very well aware of WordPress sites and it's highly unlikely that it will punish a WordPress site for having content on several very common WordPress pages.

If you start playing around with robots.txt and rel=unfollow it's easy to shoot yourself in the foot.

You could install one of the SEO plugins that only places short versions of articles on author and category pages, but I don't think it's worth the effort in most cases. Moreover you're more likely to attract long tail keyword traffic with author and category pages as they are.

So in short: Don't worry about it.

  • I understand. As a matter of fact I'm already showing just a 3-liner of the article on the category and author pages so I'm glad there's no need to worry about it. Thanks!
    – WPRookie82
    Aug 22, 2014 at 14:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.