Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

Is it possible to hide a page from users, but still have search engines crawl the content?

I have RSS feeds which are really messy and don't fit my theme, but they have content that I would benefit from having crawled.

share|improve this question
    
Sounds like you should be using an xml sitemap instead –  John Conde Jul 24 '13 at 23:57

1 Answer 1

I am not entirely sure the logic is correct here. If you have pages that you think are messy and not of use to users, generally it is spammy or poor content, and various search engines would have no need crawling those pages. When indexed, it could either start sending users there, or dock your websites overall ranking due to things like this.

In addition, showing different content to users and search engines is generally considered a form of black-hat SEO, and most search engines will treat it as such.

share|improve this answer
    
good points. thanks –  Andrew Findlay Jul 24 '13 at 23:03

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.