Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

I'm noticing that I get accesses

66.249.66.198 - - [01/Jul/2011:17:13:46 +0200] "GET /img/clip.incubus.torrent.phtml HTTP/1.1" 404 143 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.198 - - [01/Jul/2011:17:13:48 +0200] "GET /img/clip.global.deejays.download.phtml HTTP/1.1" 404 143 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

that files don't exist and there is no file on my site that has this content (I hope).

Why is googlebot trying out these links?

reverse dns and whois state that 66.249.66.198 is really googlebot.

share|improve this question

migrated from stackoverflow.com Jul 1 '11 at 15:23

This question came from our site for professional and enthusiast programmers.

    
Two guesses: 1) It's linked from elsewhere 2) Those files are common on sites infected with some malware and google tries to check for that malware so it can warn users. –  CodesInChaos Jul 1 '11 at 15:30
    
I was just suspicious because we are talking about >280000 accesses and it's still going –  ithkuil Jul 4 '11 at 7:35

3 Answers 3

up vote 3 down vote accepted
  1. If domain name is new (you recently bought it) then Google may still try to get old URLs (not because it remembers those links (it will de-index such not-working-for-a-long-time links) but because there is still a site that has links to this domain)

  2. Wherever those links were published there maybe a typo in URL (domain name) and they are pointing to your domain (e.g. hellokitten.com instead of hellkitten.com).

  3. I very doubt that it is Google checking for malware using SO many different URLs which look very similarly.

  4. The URLs you showed so far suggest they are probably related to a torrent site. As we know Google does not like torrent sites any more. Maybe it had some suspicion (don't know what it may be) and checks your site for some common URLs that it founds on other sites.

share|improve this answer

Presumably because someone linked there once.

share|improve this answer

I don't know much about how those spiders work, but google is very sophisticated. It could be any number of reasons. My guess is googlebot is trying to establish some relationship between your site and one with those files. Maybe it suspects they are related, and is testing that out.

share|improve this answer
    
Who on earth is linking tons (I see thousands of accesses) of links (like /img/clon.hunter.crack.phtml, /img/clone.dvd.3.6.1.0..serial.number.user.name.crack.phtml) on random sites like mine? perhaps some people exploiting googlebot to crawl the world for them? –  ithkuil Jul 1 '11 at 15:25

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.