All Questions
2 questions
2
votes
3
answers
2k
views
How to decrease the frequency of Google/Search-engine bots crawling my site?
My server handles my visitors ok, but recently I found there are many search engine bots crawling my site and my server gets quite busy serving these bots.
My site generates over 1000 posts each day, ...
- 183
1
vote
1
answer
815
views
Why does bingbot access my sites the most even if I set crawl rate to the minmum?
Recently my sites got overloaded and it turned out bingbot was the culprit. I set the crawl rate for the minimum for my sites in bing webmaster tools, but regardless bingbot has the most access (about ...
- 498