2

Say I have a page page.example.com/page. In my (python) server, you only get served html if you are logged in, which you can do at page.example.com/login. If you aren't logged in, the server responds with a 404 error.

Does this mean that web-crawlers would be unable to index this page? Or, more accurately, they wouldn't index it, because it "doesn't exist".

3
  • if you don't make any link to this page they won't find it nor index it.
    – gael
    Nov 26, 2019 at 8:14
  • So no links + 404 will do the noindex trick?
    – Legorooj
    Nov 26, 2019 at 8:20
  • The only page that won't raise 404 will be the login page.
    – Legorooj
    Nov 26, 2019 at 8:21

1 Answer 1

3

When Googlebot see 404 HTTP status code then they don't index that page at all. It will even remove the indexed pages if they continue to see 404 status for a long time. So hope you understand how Google seriously see 404 status code.

You should add functionality like quora, they allowed to view n number of pages before they force user to logged in. Google called it first click free, but now it's called flexible sampling

1
  • Thanks for the answer - I might do that, but not on this site - invite only.
    – Legorooj
    Nov 26, 2019 at 22:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.