Webmasters Stack Exchange is a question and answer site for pro webmasters. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Currently my site is being crawled by the Googlebot on a daily basis. Is a daily crawl from the Googlebot a good thing or bad thing?

share|improve this question

migrated from stackoverflow.com Jan 18 '11 at 18:31

This question came from our site for professional and enthusiast programmers.

Uh... It's good if you want it to be searchable, and bad if you dont. This is not a programming question. – asawyer Jan 18 '11 at 18:30
send us the url, and we have a look first and determine if its good or bad. – c2h2 Jan 18 '11 at 19:22
@c2h2 I hope that's a joke – abel Jan 18 '11 at 20:19
If you don't want to show up in Google search, then it's a really bad thing... It's how developers prelaunch alpha code level websites in the ugly duckling, golly we didn't want the competitors to see it stage. – Fiasco Labs Apr 26 '15 at 4:35

It's a good thing. This means that Google will find your new content faster which means it can be included in their index sooner. It's got to be in the index to be found in searches.

Keep in mind that this doesn't mean your pages will rank well. It doesn't even necessarily mean your pages will be indexed. But under normal circumstances being visited regularly by Googlebot means your content is being found and indexed relatively quickly. And that's a good thing.

share|improve this answer

It's a good thing because it means Google is indexing your site so that it can show up in search results.

share|improve this answer

Google crawling your site is the first step to being indexed. But, crawling in itself is more of an indicator of what is going on rather then good or bad. IE if Google is crawling your site real hard every day but none of your pages show in Google search it's probably a bad sign and you need to figure out why they are not including your site even though they know about it. On the other hand if your site is new and large and they are crawling a lot of deep pages on your site it's probably a good sign that Google likes what you have to offer and wants it in their index ASAP. Looking at your logs to see crawl patterns will also help you find issues with your site structure in terms of crawling and indexing. So I'd say rather then good or bad think of it as an opportunity.

share|improve this answer

It is a very important factor for your website. When Google crawls your site daily it means your site matters a lot or it contains a very good information or content.

When the Google crawler does not crawl your site daily. There are mainly two reasons:

  1. The site does not contain new and unique content
  2. Your server is slow. Please check it and solve the issue.
share|improve this answer

Is a daily crawl from the Googlebot a good thing or bad thing?

Good/Bad will depend on ...

  • Does the content change daily ?
  • Can you afford the bandwidth/transfer bill ?
  • Is it the whole site or specific pages ?
  • Have you solved any canonicl and duplicate issues ?

And of course, far more importantly ... is it translating into more visitors and more sales ?

share|improve this answer
If you can't afford the extra bandwidth of the Google bots then you have bigger things to worry about :) – dhaupin Apr 26 '15 at 19:16

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.