Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

What I have is a page with almost all offsite links. Each clicked link is forwarded on to the destination. What I would like the search engines to do is to index the text between the anchor tag and not follow the link itself.

<a href="somelink">Index This Text Only</a>

I've read several articles and they all seem to contradict themselves as to when to use nofollow.

What's been happening over the past 2 months that the site has been live is that both Google and Bing are crawling the site as well as all the links on the site that it has been forwarded to. The search engines are now generating a lot of 404s for images and files that never existed on my site but rather seems to correlate to the site it was forwarded to. The search engines don't seem to honor the 302 header when forwarded.

I would like to get a definitive answer on the nofollow tag as it relates to my situation. Can I use nofollow to stop the 404s and if so, will it affect my page ranking negatively?

share|improve this question
    
I think I understand your reasoning for the 302, but I would have let the 404's continue. They would have stopped eventually. Search engines make mistakes sometimes. It is better to just let them fail from my experience. –  closetnoc Apr 26 at 0:50
    
You can't get 404s from "offsite links", like the example you have posted? You also say you are getting 404s for images? Are you hotlinking images? Are you intercepting these links and doing a 302 instead? –  w3d Apr 26 at 1:19
    
@closetnoc, OK, I have noticed that the 404's are diminishing somewhat. –  Jack Apr 26 at 4:24
    
@w3d, The 404's are being generated from my site, not the sites I'm linking to. The 404's are generated from a directory structure that's not even close to mine. I can see the path from my logs. I can only conclude that the search engine thinks that the image or file it's looking for is located on the server I'm pointing to. I am hosting all of my own images and there is nothing hotlinked. I'm not intercepting the links. What I'm doing is copying some links from other sites and posting them on my site. If there is an image I want to show, I'll download the image and upload it to the server. –  Jack Apr 26 at 4:28
    
@w3d - you are great so I thought I would help you understand what is going on. Sometimes even Google makes weird mistakes. It has happened to me several times. For a few months I got requests from Google for a extremely well known site that I linked to or they linked to me- I cannot remember. I got 100k's of requests like this before it ended. These things happen. Late last year I got Google requests for resources on my site that were malformed and never existed as requested. I know it is weird. I thought I would let you know his complaint is actually not that uncommon. –  closetnoc Apr 26 at 21:05
add comment

1 Answer 1

Nofollow isn't going to help you solve this problem. Nofollow prevents Google from assigning the authority to the links that it crawls, but it doesn't prevent Googlebot from discovering and crawling those links. If there is a bug in Googlebot as you suggest, you would need to stop it from crawling the links if you want the 404 errors to stop.

One way of doing so would be with robots.txt. It sounds like you are using a 302 redirect script on your site. The URL in your href might look something like /redirect.php?id=123456 which would then issue a 302 redirect to the final URL. In that case, adding Disallow: /redirect.php to your robots.txt file would prevent Googlebot from crawling the URLs and would effectively stop Google from finding new 404 errors.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.