Tagged Questions

2
votes
1answer
109 views

Is there a good way to filter all the robots and domains they are on?

I have been working more on the bot filter for my website, but by no means is it complete. So far I have the main ones: Google, Yahoo, MSN, Baidu, Amazon, and a few others... Right now, I am ...
1
vote
1answer
105 views

What is USH.HTML and why do some robot try to access it?

I have been getting quite a few hits on my webservers for the resource /USH.HTML. I have never hosted this file, and do not know what it is. Is this a type of robots.txt file or something specific ...