I'm building a member based social network that requires users to be logged in to view content, in addition to that the content feeds (blogs, forum posts, comments, etc) are being stored in a database. Will google be able to spider this site for key words? If not then what do I need to do for them to be able to spider it? If someone could link me to a descriptive article that'd be awesome because I'm not getting much info from google and google searches.
migrated from stackoverflow.com Aug 29 '11 at 19:26
If you make content available to Google but not available to users who are not logged in then you are violating Google's terms of service. The result of this is your pages been removed from their index or your site being blacklisted.
The closest you can come to this is to do what is called "first page free" where you allow Google, and anyone coming from Google to view that content without being logged in. Experts-exchange.com is an example of a website that currently does this. If they want to see more content then they need to login. Of course, they can work around this by continually coming to the pages they want to view from Google's search results.