Webmasters Stack Exchange is a question and answer site for pro webmasters. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

My site was running well for the past three years. Recently Googlebot's request my server got increased. and at one time I got network unreachable error. When I contacted my hosting company they asked suggestions from me and I gave a list of Google ips to white list. They said that their server will block IPs which request huge numbers of pages.

After that my site appeared well in Google. Now after a month, I get "network unreachable" reported in Google Webmaster Tools. When I inquired to my host, they said I have to change my plan from shared hosting to a VPS account or a dedicated hosting.

I have 300 links indexed by Google. Previously the crawl rate was default and now I reduced to the one shown in the picture. Will this affect my indexing in Google?

enter image description here

Will my host block Google IPs for the new lowered crawl rate? I have 300 links indexed and they irresponsible say that I have to switch over to vps.

share|improve this question
up vote 1 down vote accepted

300 Links at 0.004 requests per second is next to nothing. I cannot imagine any hosting plan (even free ones) having a problem with this. If your host is throttling/blocking this then I would recommend getting a new host! (which is it by the way?) This is definitely not a reason to switch to VPS.

share|improve this answer

The way it will affect your indexing is that they will check a given page on your site less often. If the content on that page has changed then

  • they will still list the old content when they show extracts from the pages in the search results
  • if it's got "better" as far as Google is concerned, in other words, better ranking, it will take longer before your site goes up in the rankings
share|improve this answer
That's not my understanding. In my opinion it does not chance the time between 'site crawls' but just moderates the bots activity during a particular crawl (e.g. the delay between each page request). – UpTheCreek May 6 '11 at 10:29
However that means they won't look at as many pages in each crawl. I've tweaked my answer as the "less often" is a bit imprecise. – paulmorriss May 6 '11 at 10:32
Yeah, you're probably right there. – UpTheCreek May 6 '11 at 10:33

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.