What is the best way to detect bot generated activity (clicks/page visits) etc on a asp.net website? We have a website where we track leads generated to external websites, we track the IP of the user that generates the lead however we are seeing a lot of leads generated by Google and other search bots. What is the best way to filter this activity. I have heard of testing user agent strings and filtering based on know IP addresses, both available as data dumps from various sources, I am not sure which is the best one to use.

Thank you.

2 Answers 2


Like @Kinopiko said, Bots, esepcialy the top ones like google, or bing leave a clear UserAgent.

I don't know what your code looks like so I can't tell you what to do but to find the UserAgent in Asp.Net you look at Request.UserAgent in any WebForm, CodeBehind, or MVC Controller.

  • Thank you, my code in asp.net looks very similar to the check shown below. The reason I was doubtful about the reliability of my check is that some sites are selling lists of known bot user agents that contains more than a few thousand bots, however based on replies here I feel checking for the top bots should suffice.
    – user1081
    Commented Jul 28, 2010 at 8:00

Sorry but I have no idea about asp.net websites, but any reputable search engine will be sending you a user agent string which tells you it's a bot, and that should be available in your log file. Another giveaway is that they look for /robots.txt.

My checker looks like this (Perl):

sub is_bot
    my ($user_agent) = @_;
    if ($user_agent =~
    |princeton crawler
    /x) {
    return 1;

If you look in the source code of awstats you'll find something better than the above.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.