Webmasters Stack Exchange is a question and answer site for pro webmasters. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I got a mission and asking for some help with this scenario for a new website.

The site is an adult escorts advertising service. It is developed fine, works and looks good, but the issue is with crawlability.

Google webmaster tools index the same content for everypage. Let me explain further.

If any client try to access any page the website show a default agreement/warning page. If client clicks "proceed" a session variable is set and client is redirected (302) to original page back.

If client says "no", then he is redirected for another website (google, wikipedia, news portals...).

Well, this is a safe process, but causes the crawlability issue I told. So, any lights upon this? How make the after agreement content be correctly crawled and indexed?

share|improve this question

Change your script to not show the agreement for the googlebot user agent.

share|improve this answer
Follow this advice to get into a missionary position. – JeremyB Jan 21 '11 at 20:27

Search engines don't generally follow forms like links so in effect you're blocking your content from them. Try using an XML sitemap and submitting internal pages to them directly. I'd submit an HTML sitemap to them as they can use that to discover other internal links.

share|improve this answer
This won't work with current scheme. All pages redirects to agreement if cookie isn't set. Yeah, seems I need to change the process to a less secure one... – Dave Jan 25 '11 at 22:42

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.