24
votes
1answer
5k views

Bingbot request for trafficbasedsspsitemap.xml which does not exist

The logs for a website I manage show a request for a non-existent file by Bingbot. The details of the request are Path: /trafficbasedsspsitemap.xml Useragent: "Mozilla/5.0 (compatible; bingbot/2.0; ...
1
vote
1answer
615 views

Why are nonexistant files listed in the TRANSFER log?

This snippit from my apache log reflects some piece of malware casually browsing through my site for some unknown purpose. The most puzzling thing to me is that the GET address doesn't correspond ...
2
votes
1answer
135 views

Is there a good way to filter all the robots and domains they are on?

I have been working more on the bot filter for my website, but by no means is it complete. So far I have the main ones: Google, Yahoo, MSN, Baidu, Amazon, and a few others... Right now, I am ...
1
vote
1answer
108 views

What is USH.HTML and why do some robot try to access it?

I have been getting quite a few hits on my webservers for the resource /USH.HTML. I have never hosted this file, and do not know what it is. Is this a type of robots.txt file or something specific ...