Say I have a page page.example.com/page
. In my (python) server, you only get served html if you are logged in, which you can do at page.example.com/login
. If you aren't logged in, the server responds with a 404 error.
Does this mean that web-crawlers would be unable to index this page? Or, more accurately, they wouldn't index it, because it "doesn't exist".