Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

My site was running well for the past three years. Recently googlebot's request my server got increased. and at one time i got network unreachable error. When i contacted my hosting company they asked suggestions from me and i gave a list of google ips to white list. They said that their server will block ip's which are of huge traffic.

After that my site appeared well in google. Now after a month again i get network unreachable. when i inquired them they said i have to change my plan from shared hosting to a vps account or a dedicated hosting.

I have 300 links indexed by google. will that affect much? So i thought of reducing the crawl rate. Please see the image below and give me suggestions.enter image description here

Previously it was default and now i reduced to the one shown in the picture. Will this affect my indexing in google.

Besides will a server block google ip's for this kind of crawl rate like i have 300 links indexed. and they irresponsible say that i have to switch over to vps. confused.

share|improve this question
add comment

2 Answers 2

up vote 1 down vote accepted

300 Links at 0.004 requests per second is next to nothing. I cannot imagine any hosting plan (even free ones) having a problem with this. If your host is throttling/blocking this then I would recommend getting a new host! (which is it by the way?) This is definitely not a reason to switch to VPS.

share|improve this answer
add comment

The way it will affect your indexing is that they will check a given page on your site less often. If the content on that page has changed then

  • they will still list the old content when they show extracts from the pages in the search results
  • if it's got "better" as far as Google is concerned, in other words, better ranking, it will take longer before your site goes up in the rankings
share|improve this answer
    
That's not my understanding. In my opinion it does not chance the time between 'site crawls' but just moderates the bots activity during a particular crawl (e.g. the delay between each page request). –  UpTheCreek May 6 '11 at 10:29
    
However that means they won't look at as many pages in each crawl. I've tweaked my answer as the "less often" is a bit imprecise. –  paulmorriss May 6 '11 at 10:32
    
Yeah, you're probably right there. –  UpTheCreek May 6 '11 at 10:33
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.