Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

Webmaster Tools is showing thousands of 404 errors, where pages on the site are referring to another incorrect URL. For example, URL not found www.example.com/shop/=, linked from www.example.com/shop/gift-voucher and www.example.com/shop/special-plant-offers.

I obviously have checked the source and cannot find any references to this link on any page.

The only consistent issue is that it only seems to report this error on pages with two section i.e. www.example.com/shop does not report any error whilst all pages with www.example.com/shop/xxx (where xxx can be several different pages such as gift-voucher) all report this.

I cannot seem to duplicate this error. I have run a link checker (we use Screaming Frog) and it does not report this error. I have fetched these pages as a bot, and these do not report this error.

I am at a total loss. I cannot even duplicate the issue, but it is most definitely an issue, as Webmaster Tools is reporting new errors every day.

Is this perhaps Google bot doing its own thing?

share|improve this question
    
Take a look at this question from Google's forum's may have an answer support.google.com/webmasters/bin/… –  Anagio Dec 5 '12 at 17:10
    
Unfortunately the generic google page on 404's does not really help (and this issue is way more complex than that addressed in the google help page). I could repost this question on google's forum, but I prefer stack exchange –  plantify Dec 5 '12 at 17:30
    
Couldn't be Googlebot trying to execute some JavaScript, could it? I've had similar woes and that's occasionally looked like the culprit. –  GDav Dec 5 '12 at 18:08
1  
@plantify i'd repost it there as well, it's not in the SE network so it's perfectly fine to post it there as well. Especially since there are Googlers chiming in on the forums once in a blue. –  Anagio Dec 5 '12 at 18:15

1 Answer 1

I cannot verify this authoritatively, but based on my own observations I'm going to hypothesise that once Google has cached a page, it will continue for some time to follow links from its own cached version of the page rather than from the live version of the page.

As far as I can see, the two quickest ways to get around this are:

1) to re-fetch the live version of the page in Google Webmaster Tools, so that the Google cache has an up-to-date copy of the live page; or

2) to update mod_rewrite rules in .htaccess, so that when Google tries to follow an outdated link to a non-existent location, it is automatically redirected by the server to an existing location.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.