I moved my website from one VPS to another on 16th April. The A record of the domain name has a TTL of 86400. I made the IP address change at 19:00 GMT 16th April. I do not want the site to face ...
At a question asked previously at this site regarding Baidu's overactive crawling tendencies, one user comments: I'd only block a search engine spider if you never want to receive any search ...
Most of my visits are from baidu spiders. I don't think it helps search engines at all so I'm thinking of how to block them. Could this be done via iptables? I'm using nginx as my webserver.
When i'm checking my apache log I can see that baidu has been crawling my website 10 times a day for the last 2 weeks. It's not that i care much about it but I'm really curious about why he's doing ...