My server handles my visitors ok, but recently I found there are many search engine bots crawling my site and my server gets quite busy serving these bots. My site generates over 1000 posts each day, ...
I have an aggregate site on a linux server that pulls in feeds from a universe of about 2,000 blogs. It's in Wordpress 3.4.2 and I have a cron job that is staggered to run five times an hour on ...
At work we are in a discussion with team members who seem to think bots will cause us problems relating to performance when running on our services website. Our setup: Lets say I have site ...