2

Say I have a page page.example.com/page. In my (python) server, you only get served html if you are logged in, which you can do at page.example.com/login. If you aren't logged in, the server responds with a 404 error.

Does this mean that web-crawlers would be unable to index this page? Or, more accurately, they wouldn't index it, because it "doesn't exist".

  • if you don't make any link to this page they won't find it nor index it. – gael Nov 26 '19 at 8:14
  • So no links + 404 will do the noindex trick? – Legorooj Nov 26 '19 at 8:20
  • The only page that won't raise 404 will be the login page. – Legorooj Nov 26 '19 at 8:21
3

When Googlebot see 404 HTTP status code then they don't index that page at all. It will even remove the indexed pages if they continue to see 404 status for a long time. So hope you understand how Google seriously see 404 status code.

You should add functionality like quora, they allowed to view n number of pages before they force user to logged in. Google called it first click free, but now it's called flexible sampling

  • Thanks for the answer - I might do that, but not on this site - invite only. – Legorooj Nov 26 '19 at 22:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.