Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

In our error logs we have quite a few (~ 10s / day) internal recursion errors caused by googlebot. Here is a typical example:

[Sun Jul 03 10:58:22 2011] [error] [client goo.gle.ipa.ddr] Request exceeded the limit of 10 internal redirects due to probable configuration error. Use 'LimitInternalRecursion' to increase the limit if necessary. Use 'LogLevel debug' to get a backtrace.

Looking in the access log, the offending GET is always looking for a broken URL with this format:

http://ourdomain.com/wp-content/uploads/2007/11/image-name.jpg%22%20width=%2239%22%20height=%2250%22%20alt=%22image%22/%3E%3C/a%3E%3Ca%20href=%22/m/imgres?q=some+query+here

Undoing the URL encoding to make it more readable:

http://ourdomain.com/wp-content/uploads/2007/11/image-name.jpg" width="39" height="50" alt="image"/></a>< href="/m/imgres?q=some+query+here

This looks like a broken fragment of some HTML that was displaying the image as part of a hyperlink. Looks like it might have been part of some sort of search result.

We don't have any links of this format anywhere in our site, so I can only assume this is a reference from somewhere else that is either misinterpreted by googlebot, or is just broken in the page markup.

There is no entry for the referrer in the Apache access_log. I have tried to find the source in Google webmaster tools, but although the unreachable is noted, there is no mention of the source. I've also tried to use Google to find the reference without success. I do notice that the "imgres" in the href fragment is used in Google image search, but the rest of the fragment isn't consistent with this.

My questions:

  • Should I bother with this or just let these recursion errors happen (10s of them / day)?
  • If I need to bother, any suggestions on how to find their source?

Thanks.

share|improve this question

migrated from serverfault.com Jul 3 '11 at 21:55

This question came from our site for professional system and network administrators.

    
Do you have any redirects related to images in your htaccess file? Do you have any Wordpress plugins related to images? –  paulmorriss Jul 4 '11 at 10:10
    
Only the standard rules, and no plugins that have anything to do with the images attached to posts. –  Greg Jul 5 '11 at 9:04
    
I installed a plugin called "Monkeyman Rewrite Analyzer" a couple of days ago to help me write a rewrite rule. You could install it and see if some combination of a bad link to your site out there somewhere and WPs rewrite rules is causing this. –  paulmorriss Jul 5 '11 at 9:16
    
Thanks - I'll check that out. –  Greg Jul 5 '11 at 9:47
add comment

3 Answers 3

Try doing a search on Google of type:

link:<url of image>

That 'should' help you locate what is linking to the image.

share|improve this answer
    
This is a good suggestion, but unfortunately it doesn't find the offending link. Thanks. –  Greg Jul 5 '11 at 9:01
add comment

This happens a lot. I get hundreds of links with HTML entities replacing chars too.

Google Webmaster Tools will let you know where and when the link was found 99% of the time. Just look under Diagnostics -> Crawl Errors

There will be a table listing URLs and the number of pages (3rd column) that links to them. Click on where it says 'X pages' and that will list you the X pages where that link is found and when the link was encountered last by Googlebot.

share|improve this answer
    
Webmaster Tools works pretty effectively at finding them if they're html pages. For these, if the referred to link still exists on your website, 301 permanent redirect the incoming busted link to the proper page to make it quit annoying Google and anyone who tries to come in on the link. –  Fiasco Labs Mar 3 '12 at 17:11
add comment

Maybe you need to run HTTrack on your website with it set to download everything and follow all links within the site:

http://www.httrack.com/

You will then be able to determine from your access log whether your site (although not having the offending link) is actually generating the problem links. The problem links not necessarily being the one Google Image Search is wanting, but some weird recursion going on in your Wordpress when it 404s on them.

You might also want to put in some .htaccess rules to tidy up your GoogleBot problem. If you send 410 or 301 instead of 404 then the Google Image search results for your site should get tidier next time it crawls your site.

share|improve this answer
    
Thanks. I'll check that out. The links actually cause a 500 error (the recursion I mentioned), but I might need to catch them in the .htaccess as you suggest. –  Greg Jul 5 '11 at 9:02
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.