I have a HTTP sniffer application which runs on C++ , when I am getting the user agent field I get something like : Mozilla/5.0 (Windows NT 6.1; WOW655970812; ...
I run a small web crawler and had to decide on what user agent to use for it. Lists of crawler agents as well as Wikipedia suggest the following format: examplebot/1.2 ...
How should I interpret a empty User-agent? I have some custom analytics code and that code has to analyze only human traffic. I have got a working list of User-agents denoting human traffic, and bot ...