This should be indexed :

but not this :

In other words, it is possible to automatically index the subpages, without a page to link them all ?


Simple way to prevent crawling /products in your situation would be to,

  • Make no reference (Link) to /products or /products/
  • If you fetch /products or /products/ return 404

Depending on your website build, Google may or may not attempt to crawl or find pages in between. If you will ensure above 2 points, you should be fine.

  • Yes but how would google know about (=index) the individual products pages then ? ( http://example.com/products/3984239) which are automatically generated ? – lapin May 24 '18 at 2:07
  • As well, noindex for these pages if possible would suffice. I realize not all CMS make this easy. Cheers!! – closetnoc May 24 '18 at 2:21
  • @closetnoc I'm not sure I understand you comment right. My goal is to index products/3984239 without having to link them from the top page, /products middle page wouldn't even exist in my case. How is it possible ? – lapin May 24 '18 at 2:23
  • That means if you can't ensure 404 on /products then its worth having noindex when the page /products loads. support.google.com/webmasters/answer/93710?hl=en – TopQnA May 24 '18 at 2:52
  • @TopQnA Again, I don't think this is relevant : /products page doesn't exist. I just want to index the subpages but not page actually link to them – lapin May 24 '18 at 6:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.