Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

For two months now, googlebot is trying to get a file which does not exist anymore. This is just one example out of many: I had renamed the file to a better name and removed the old file. Now, why does Google insist on getting a file which it already has seen does not exist for months? doesn't it just give up and get on being a happy bot?

My Error log file is filled with these repeating lines all wanting to get that one file, although they know for already long time that its not there:

  1. What to do in these situations?
  2. Are there automatic redirection rules that handle via 301 to the home page?

    [Sat Mar 05 01:55:41 2011] [error] [client 66.249.66.177] File does not exist: /var/www/vhosts/website.org/httpdocs/extraNeus.php

    [Sat Mar 05 01:58:20 2011] [error] [client 66.249.66.177] File does not exist: /var/www/vhosts/website.org/httpdocs/extraNeus.php

    [Sat Mar 05 02:03:57 2011] [error] [client 66.249.66.177] File does not exist: /var/www/vhosts/website.org/httpdocs/extraNeus.php

on and on ... and on...

share|improve this question

migrated from stackoverflow.com Mar 5 '11 at 1:40

This question came from our site for professional and enthusiast programmers.

3 Answers 3

up vote 1 down vote accepted

The reason that Googlebot is still trying to hit the page is that there is probably a link to it somewhere. When Google finds a link, it follows it without checking whether it has previous seen a 404 for that page or not. So if this page keeps getting hit, it's probably because of a link.

I agree with Crayon Violet and suggest that you include the 301 redirect from the old page to the new one.

share|improve this answer

put this in .htaccess file (change the path/to/files to your file locations)

Redirect 301 /extraNeus.php http://www.yoursite.com/new_page.php

you can leave it in there forever or next time google indexes the page you should be good to remove it.

share|improve this answer

You should use a permanent redirect to tell GoogleBot and others where the page has moved to.

If the page no longer exists at all then you can add entries in robots.txt to tell bots not to access the missing page:

User-agent: *     
Disallow: /extraNeus.php
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.