4

For two months now, googlebot is trying to get a file which does not exist anymore. This is just one example out of many: I had renamed the file to a better name and removed the old file. Now, why does Google insist on getting a file which it already has seen does not exist for months? doesn't it just give up and get on being a happy bot?

My Error log file is filled with these repeating lines all wanting to get that one file, although they know for already long time that its not there:

  1. What to do in these situations?
  2. Are there automatic redirection rules that handle via 301 to the home page?

My error log:

[Sat Mar 05 01:55:41 2011] [error] [client 66.249.66.177] File does not exist: 
/var/www/vhosts/website.org/httpdocs/extraNeus.php 

[Sat Mar 05 01:58:20 2011] [error] [client 66.249.66.177] File does not exist: 
/var/www/vhosts/website.org/httpdocs/extraNeus.php

[Sat Mar 05 02:03:57 2011] [error] [client 66.249.66.177] File does not exist: 
/var/www/vhosts/website.org/httpdocs/extraNeus.php

on and on ... and on...

1

The reason that Googlebot is still trying to hit the page is that there is probably a link to it somewhere. When Google finds a link, it follows it without checking whether it has previous seen a 404 for that page or not. So if this page keeps getting hit, it's probably because of a link.

I agree with Crayon Violet and suggest that you include the 301 redirect from the old page to the new one.

| improve this answer | |
  • Also look for where the link into that page is and update it. If it's offsite, send a message to the webmaster of the site linking to yours. – GeoffAtkins Nov 28 '19 at 8:00
4

put this in .htaccess file (change the path/to/files to your file locations)

Redirect 301 /extraNeus.php http://www.yoursite.com/new_page.php

you can leave it in there forever or next time google indexes the page you should be good to remove it.

| improve this answer | |
4

You should use a permanent redirect to tell GoogleBot and others where the page has moved to.

If the page no longer exists at all then you can add entries in robots.txt to tell bots not to access the missing page:

User-agent: *     
Disallow: /extraNeus.php
| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.