3

I know that nofollow prevents link juice from being passed across a link. But if search engine robots discover a link with a nofollow on it, will they add that link to their crawl queue?

In other words, if I create a link to a brand new page and put a rel=nofollow attribute on that link, will it prevent search engine bots (particularly Googlebot) from crawling the page. (Assuming that this link remains the only link into that page.)

I've read conflicting reports about this over the years and I'm looking for authoritative references about the current state of affairs. Official statements from Google or published results of independent testing would be ideal.

Improve this question
2
  • 1
    If authoritative references means Google, they don't specifically state either way. Tests by people indicate that nofollow links are used for discovery. Google does acknowledge that even content blocked in robots.txt may get indexed ... support.google.com/webmasters/answer/93708?hl=en
    – Wayne Smith
    Nov 5, 2013 at 17:39
  • I'm pretty sure that it is just URLs and incoming links that are indexed, not the page contents when the page is listed in robots.txt. Here is a reference: webmasterworld.com/google/4490125.htm It might be good to get some answers to that question here as well.
    – Stephen Ostermiller ♦
    Nov 5, 2013 at 17:56

2 Answers 2

Reset to default
3

This article from 2006 explains an experiment where a new page was created an linked to only with nofollow.

It finds that Google:

  • Follows the link (crawls the URL with Googlebot)
  • Does not index the content of the page

The article also publishes results for MSN and Yahoo.

In 2009 Google significantly changed the way that nofollow works. Results regarding nofollow from before before the change would be good to re-test. A 2012 experiment on the discovery of a URL from a nofollow link found that the page does not get indexed (although it does not report on whether or not it got crawled.) [Credit: moobot from the comments.]

Improve this answer
1
0

Implementation of rel=nofollow on links will instruct a search engine not to follow the link. It won't, however (necessarily) prevent the linked page from getting indexed (purely as a URL) but it will prevent the contents at that URL from being indexed providing that there are no other followed links to the same destination elsewhere on the web.

This confirms what's already stated by both comments but also personal experience also confirms this is what happens also.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.