Most of the bandwidth of one site is being consumed by an unidentified bot. According to AWSTATS it says: Unknown robot (identified by 'bot*') consumed 164 GB this month.
By comparison, Googlebot consumed 10 GB and visitors (viewed traffic) consumed 25 GB. This means rogue bots are consumming over 6X the bandwidth of visitors. For other sites which I run (about a dozen) the normal ratio is 25%, so for 25GB of viewed traffic, bots take about 6GB in TOTAL.
The question therefore is: How to identify which bot(s) are causing this huge amount of request and how to stop them or slow them down if they are useful?
Obviously, most bots that visit the site are important including the Googlebot, Yahoo Slurp, MSNBot, etc, including the AdSense/DoubleClick bots, so I cannot simply block all bots.
The reason I am investigating this is that I am reaching the limit of bandwidth and exceeded CPU usage for my host, so I was sent a notice.