At a question asked previously at this site regarding Baidu's overactive crawling tendencies, one user comments: I'd only block a search engine spider if you never want to receive any search ...
Most of my visits are from baidu spiders. I don't think it helps search engines at all so I'm thinking of how to block them. Could this be done via iptables? I'm using nginx as my webserver.
When i'm checking my apache log I can see that baidu has been crawling my website 10 times a day for the last 2 weeks. It's not that i care much about it but I'm really curious about why he's doing ...