1

I have a webpage on which there are around 30 data points which link to 30 different pages on my own site which are non-crawlable and have been blocked via robots.txt

My question is if we use href over these blocks with the non-indexable links does this has any impact on the SEO of the page?

Should this be handled by JS rather than href to avoid above or if href's are used then nofollow should be used. But if none of above has been done and href has been used, would that be impacting the SEO in a negative sense?

Improve this question

2 Answers 2

Reset to default
-3

First of all. A robots file does not in any way block crawlers. If you want to actually block them you need to use Apache (.htaccess) or PHP. Using a robots file is like putting up a sign in your yard that says "no trespassing." Using Apache is like putting a big fence around your yard.

Non indexed pages will neither help nor hurt your SEO unless there are things on those pages that really should be indexed.

Improve this answer
6
  • 1
    "A robots file does not in any way block crawlers" What? Googlebot and other search engine crawlers certainly honor robots.txt. Disallowing something in robots.txt will prevent crawlers that matter for SEO from accessing the content on those URLs.
    – Stephen Ostermiller ♦
    Jun 22, 2017 at 9:48
  • They are not asking about "Non indexed pages", they are asking about pages with links that can't be crawled.
    – Stephen Ostermiller ♦
    Jun 22, 2017 at 9:49
  • Steve, basic English buddy. Robots do not block crawlers. Period. I don't know how to be clearer than that. Google ignores them all the time, Google it. And do you know what happens when you have pages with link that can't be crawled? You get non indexed pages.
    – I wrestled a bear once.
    Jun 22, 2017 at 10:45
  • Google sometimes indexes pages that it doesn't crawl, but Googlebot is very good at obeying robots.txt. Google indexes pages even if they contain links that can't be crawled.
    – Stephen Ostermiller ♦
    Jun 22, 2017 at 10:52
  • Why are u playing word games? Yes, Google sometimes indexed pages disallowed in robots, even though it's not supposed to. Sometimes they crawl it anyway. I would say that makes Google bad at obeying the robots directives. Besides Google is only one out of thousands that are out there. The robots file is a great way for bots to learn about directories and files that wouldn't have been discovered through the course of a crawl. If indexing is turned off in Apache and a page isn't linked to anywhere then a crawler has no way of knowing about it... unless it's disallowed in robots.
    – I wrestled a bear once.
    Jun 22, 2017 at 11:04
-3

yes it will affect your seo strategy. Put apppropriate content on page

Improve this answer
1
  • 1
    A one line answer is not high quality. Please edit this to let us know why this is the case? Do you have articles or documentation that can back this up? Do you have personal experience? I also think you may be missing the point of the question. It doesn't say this is the only content on the page. There may already be appropriate content on the page, they are asking if 30 blocked links are going to be a problem.
    – Stephen Ostermiller ♦
    Jun 22, 2017 at 9:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.