Is it possible to hide a page from users, but still have search engines crawl the content?

I have RSS feeds which are really messy and don't fit my theme, but they have content that I would benefit from having crawled.

  • Sounds like you should be using an xml sitemap instead
    – John Conde
    Commented Jul 24, 2013 at 23:57

1 Answer 1


I am not entirely sure the logic is correct here. If you have pages that you think are messy and not of use to users, generally it is spammy or poor content, and various search engines would have no need crawling those pages. When indexed, it could either start sending users there, or dock your websites overall ranking due to things like this.

In addition, showing different content to users and search engines is generally considered a form of black-hat SEO, and most search engines will treat it as such.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.