Hot answers tagged web-crawlers
2
I know you previously asked to block the bots and now you just want to slow them down.
The only way to force them to slow down is not to serve them content if they are going too fast. The problem with not serving content is it will surely have negative consequences to your search engine rankings.
In case you decide to block some of the less important bots, ...
Only top voted, non community-wiki answers of a minimum length are eligible
