Is it possible tell Google to only index my site between certain hours? If it is, how can I manage it?
-
I don't think that's possible unfortunately– Michael dMar 13, 2018 at 16:46
-
I could have sworn I had seen that, turns out it was Bing that lets you do that, so the crawl traffic can avoid your regular peak traffic. :-|– Ulrich SchwarzMar 13, 2018 at 17:01
-
Simple... you don't. Be grateful that its visits your site when it chooses.– Simon HayterMar 13, 2018 at 17:09
-
1You mean crawl (i.e., when Google’s bots visit your pages) instead of index (what Google does with the crawled content), right?– unorMar 13, 2018 at 21:25
-
What is the reason for this?– MrWhiteMar 13, 2018 at 23:58
|
Show 1 more comment
2 Answers
It is not possible to get Google to index your web site only during certain hours of the day. If your web site isn't "always on" and "always available", Google doesn't want to index it at all.
I can no longer find a reference to it in the Google Webmaster Guidelines, but Google used to specifically state that your site had to be always available. They gave an example of a site hosted on a server at your workplace that was only turned on during business hours. The guidelines said that Google wouldn't index such a site.
At the very least you should make the information on your website available all the time. If you need to shut down some functionality at certain times of the day you may do so and put a message up to users explaining why. I've seen that before with Jewish owned eCommerce sites. You can't order anything from them one day a week on their holy day.
answered Mar 13, 2018 at 15:02
You simply can't tell Google when to crawl your site. What you can do is limit the crawl rate in order to avoid bandwidth problems on your server.
If Google is making too many requests per second to your site and slowing down your server, you can limit the crawl rate for root-level sites—for example, www.example.com and http://subdomain.example.com. The crawl rate that you set is the maximum crawl rate that Googlebot should make. Note that it does not guarantee that Googlebot will reach this maximum rate.
You can set the crawl rate at Google Seach Console (site settings).
