Tell me more ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.
up vote 0 down vote favorite

Recently, a site of mine was rejected due to "Login Only":

"Login Only: During our review of your website, we found that the majority of pages on your site are behind a login, or there is restricted access. Please note that we will not approve applications for login-protected pages, as we are not able to review their content for acceptance into the program."

Although the site does require login to send content, it doesn't require any to view any page. How do I tell the Googlebot or whatever is used to crawl pages to adsense that all the content is publicly available but registration is needed to post?

Current robots.txt

User-agent: *

Disallow: /moderation/

User-agent: Googlebot-Image

Disallow: /

User-agent: Mediapartners-Google

Allow: /

share|improve this question
can you link to the website? – YardenST Nov 10 '12 at 23:26
@YardenST I'm afraid not. Can you help me with the provided information? – Renan Nov 10 '12 at 23:49
that's a situation! Can you change the way pages are accessed? Maybe you are using some cookies or something that is blocking google crawlers from seeing the pages as public? – Mohamed Anis Dahmani Nov 11 '12 at 2:33
@Renan Do you have any mechanism in your website, that prevent bots from accessing the site? – YardenST Nov 11 '12 at 7:55
@YardenST I do have some restrictions in robots.txt. I added it in the question. – Renan Nov 11 '12 at 15:19

1 Answer

up vote 1 down vote

I had the same problem and I resolved it.

Google sends crawler to visit your site. It´s automatic robot control, not human control. Your rejection is also automatic from this crawler.

If you want see what crawler really gets when he visits your site, you must using some text web browser like Lynx. After using this browser, I saw what was the problem:

  • crawler see only clean link. If you have some other element in A element, maybe like span, crawler don´t see it as a link. For crawler I have only one link and it´s a link to login page.
  • crawler doesn't get content with ajax filing

Take a look to your page with Lynx browser to find a reason why you get rejection.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.