Is it OK to have a page that will not show content to browsers unless they are logged in - but will recognise a search bot and show it the content regardless?

I.e. googolebot get the content - normal users get a login screen.

I was under the impression that this was a no-no but am increasingly seeing this (Microsoft MSDN for example) - any authorative sources for yes/no?

  • Experts Exchange does it, too. (or did it, seems they're changing this up a bit lately) – Chris Adragna Nov 22 '10 at 17:22

Yes, you can do that. It's called "First Click for Free for Web Search."

This seems like a subjective question/answer, but your intent is not to deceive here. No doubt, the fear is rooted in having something different for the spider than you normally serve to a user, but you:

a.) need to get your content spidered b.) need to have users login (based on your business rules)

What you're doing is referred to by some as "cloaking" and that's not allowed because of the usual blackhat intent of most cloaking efforts (i.e. maximize SEO juice for the spiders, but do something less for real users).

There is a post on the Google Webmaster Central Blog about this: http://googlewebmastercentral.blogspot.com/2008/10/first-click-free-for-web-search.html

So, go for it!

  • Just to be clear - this is not something I am implementing - just something I've seen done and didn't think it was allowed. You've confirmed how I've seen it done (not the 'first click for free' method) is indeed frowned upon. – Ryan Nov 22 '10 at 17:57
  • 2
    It's really only cloaking if what the user sees is different from what's displayed in the search results. Filtering by referrers isn't really cloaking since the user will see exactly what they expect to see when they click on the result link. If you use a bait-and-switch tactic as described in the question, then that does qualify as cloaking. – Lèse majesté Nov 22 '10 at 18:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.