An http header field that identifies the browser and operating system.

learn more… | top users | synonyms

3
votes
1answer
243 views

What browser is the “Apache-HttpClient/UNAVAILABLE (java 1.4)” user agent?

Some of my users are logging in with this browser: Apache-HttpClient/UNAVAILABLE (java 1.4). What browser is this?
4
votes
1answer
80 views

Can I block requests that have no user agent because they cause 404 errors?

Is it OK to block requests that contain no user agent? I am getting random 404 errors on URLs I have never used. These requests have no user agent. If this is the case, will it affect any of my ...
0
votes
1answer
36 views

What percent of all active IP + User Agent combinations are not unique?

It's fairly common practice to define a unique user by a 1-to-1 hash of IP + User Agent. But not all unique users have a unique IP + User Agent combination. Thus: collisions - events misattributed to ...
7
votes
1answer
107 views

Combine user-agents in robots.txt

Can user-agents be listed together, followed by their common rules like so in a robots.txt? User-agent: Googlebot User-agent: ia_archiver #Alexa User-agent: BingPreview #Microsoft User-agent: bingbot ...
2
votes
2answers
343 views

How to block user agent from accessing my WordPress site with htaccess?

I have noticed large amount of traffic coming to my site from different IP's with user agent. Mozilla/5.0 (compatible; SiteExplorer/1.0b; +http://siteexplorer.info/) I don't have much experience ...
1
vote
1answer
142 views

Identify this user agent - claims to be IE7 on Windows 7

We have a tracking pixel in one of our emails and our the most frequent user agent is this strange one: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR ...
1
vote
2answers
243 views

Finding different Windows XP Service Packs via User-Agent string

I am planning to get rid of the SHA1 certificate signing algorithm in our certificates and switch over to SHA-256. Problem is, that Windows XP (before SP3) is not supporting this. Therefore I need to ...
1
vote
2answers
64 views

How can we find real source of suspected user entries?

I have a website where I keep logging the user entries. I found some anonymous entries in my table and I suspect they are using some kind of proxy switchers. How can I find the real source of these ...
1
vote
3answers
228 views

Can web crawlers that visit my site fake their user agent?

I want to write a robots.txt file for my website and allow the famous bots (Google, Bing, and Yahoo) to crawl my website, but deny the rest. I want to know if I add User-agent: Googlebot, will fake ...
2
votes
1answer
119 views

404 Hits from “PHPCrawl” on WP site

Forgive me if this is in the wrong area - I thought considering the topic, it belonged here (as opposed to in WP). Correct me or move it if I am wrong. I've been looking around one of the sites I ...
0
votes
1answer
37 views

Long user agent field in HTTP header

I have a HTTP sniffer application which runs on C++ , when I am getting the user agent field I get something like : Mozilla/5.0 (Windows NT 6.1; WOW655970812; ...
2
votes
2answers
908 views

Correct usage of robots.txt file (“Googlebot-Image” and “Mediapartners-Google”)

This is the current robots.txt file I am using on a site of mine. I have excluded large parts of the IP.Board forum in order to try and cut down on duplicate content. I've also excluded some WordPress ...
3
votes
2answers
156 views

Plus sign in front of URLs in user agents

I run a small web crawler and had to decide on what user agent to use for it. Lists of crawler agents as well as Wikipedia suggest the following format: examplebot/1.2 ...
2
votes
2answers
45 views

Question about duplicate groups (agents) in robots.txt

I'm looking at a corner case with robots.txt, and am curious if you have any guidance. Basically, I'm in a situation where multiple people will be making edits to the same robots.txt file. I've ...
-2
votes
2answers
269 views

How do I prevent duplicate indexing by Googlebot-Mobile?

I have only one version of my web-site, and it is optimised for both the desktop and the smartphone. I've noticed that Google crawls each page exactly twice: once with Googlebot/2.1, and again with ...
4
votes
3answers
1k views

Filtering desktop users by user agent string

In order to determine whether to serve the mobile or full version of my site by default, I use a service that examines requests to determine the type of device they originated from. However, calls to ...
8
votes
2answers
2k views

Should I block bots from my site and why?

My logs are full of bot visitors, often from Eastern Europe and China. The bots are identified as Ahrefs, Seznam, LSSRocketCrawler, Yandex, Sogou and so on. Should I block these bots from my site and ...
1
vote
1answer
636 views

Which browser contains “com.google.GooglePlus” in the agent string?

I noticed the following agent user strings hitting my website and wondered which browser is this user string for? Mozilla/5.0 (iPad2; iPad; U; CPU OS 5_1_1 like Mac OS X; en_GB) ...
1
vote
1answer
158 views

Java/1.7.0_07 as user agent

almost every day something (it should be a bot) visits all of my pages on my website (amongst other things it doesn't correctly handle spaces in urls) with Java/1.7.0_07 (or different java versions) ...
2
votes
3answers
2k views

Interpretation of empty User-agent

How should I interpret a empty User-agent? I have some custom analytics code and that code has to analyze only human traffic. I have got a working list of User-agents denoting human traffic, and bot ...
5
votes
3answers
974 views

What does it mean when a User-Agent has another User-Agent inside it?

Basically, sometimes the user-agent will have its normal user-agent displayed, then at the end it will have the "User-Agent: " tag displayed, and right after it another user-agent is shown. Sometimes, ...
6
votes
2answers
762 views

Why do some user agents have spam urls in them (and why are they always Opera/Presto User-Agents)?

If you go to (say) the last 100 entries (visits) to the botsvsbrowsers.com website (exact link, feel free to take a look: http://www.botsvsbrowsers.com/recent/listings/index.html ), you'd notice that ...
3
votes
1answer
525 views

Disallow all user agents except one using .htaccess?

I've been struggling to get this .htaccess working. The aim is to disallow all user agents besides my app. The app sends a GET request with a user agent of lets say 'AcmeUpdater'. Whenever I try to ...
0
votes
1answer
356 views

How to secure robots.txt file?

I would like for User-agents to index my relative pages only without accessing any directory on my server. As initial thought, i had this version in mind: User-agent: * Disallow: */* Sitemap: ...
3
votes
1answer
1k views

Preventing high bandwidth usage from Yandex

Although I put a row for Yandex into my robots.txt file, sometimes Yandex indexes my website aggressively. So I hard coded a part and check for user agent, and serve cached file if user agent is like ...
1
vote
1answer
458 views

Block access to robots.txt file .htaccess

I read in this forum that this .htaccess code should block access to the robot.txt file: RewriteCond %{HTTP_USER_AGENT} ^Mozilla RewriteCond %{HTTP_USER_AGENT} !(Slurp|surfsafely) RewriteRule ...
3
votes
2answers
3k views

Why would an IE8 in a desktop has a 'Tablet PC 2.0' in its user-agent string?

I am just curious, why would a windows 7 desktop, installed with ie8, have Tablet PC 2.0 in its user agent string. Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR ...
0
votes
1answer
1k views

BingPreview user agent

I'm getting several invalid requests from a "BingPreview" user agent. The complete user agent header is: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/534+ (KHTML, like Gecko) BingPreview/1.0b The ...
2
votes
4answers
2k views

How should I deal with user agent parsing in logs?

My web app project includes logging functionality so we can see where visitors are coming from (referrer URL), what the popular user agents are, what pages are most popular, etc. The log is stored in ...
3
votes
2answers
1k views

What is a “SiteCon” user agent?

I've found some unusual activity in my site's Apache logs and every hit has this UserAgent string: "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-gb; SiteCon/8.8.14)" What is SiteCon?
1
vote
1answer
94 views

In User Agent strings, what does “AskTbTRL” denote?

Here are a bunch of examples: http://www.botsvsbrowsers.com/listings.asp?search=asktbtrl I would've guessed an Ask.com-related bot, but apparently not according to their FAQ.
0
votes
2answers
414 views

SSL Certificate is Untrusted how to fix it

How can i fix the untrusted ? And why it appears ? What may cause this problem.This is in FF but in Chrome is red https with x marked on it.
8
votes
3answers
604 views

Tactics for dealing with misbehaving robots

I have a site that, for regulatory reasons, may not be indexed or searched automatically. This means that we need to keep all robots away and prevent them from spidering the site. Obviously we've had ...
4
votes
2answers
532 views

Is collecting IP address and user agents on form submission illegal in the U.S.?

I'm creating a website where I collect the name, email, state, and an optional photo of a user. Before the user submits the form, they have to agree to the terms of the site. For testing and ...
1
vote
1answer
98 views

what is user agent Whirlpool BD2; H010818

what is this user agent Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; Whirlpool BD2; H010818) this came from a ip address tracing to the organisation Websense the question also here thanks
1
vote
2answers
231 views

Untrusted user-agent?

My logs recently showed the following user-agent: samsung-gt-s5620 UNTRUSTED/1.0 Why would a user-agent have untrusted in caps in its name?
0
votes
1answer
84 views

What browser or bot has the user agent of “SD-1.30.1”?

I have tried checking on Google but to no avail. Does anyone recognize this user agent?
2
votes
1answer
475 views

Apache logging based on user agent

I'd like to update my apache logging so that its destination is based on the user agent string or file type. Specifically, I would like to send my googlebot requests to their own log file, and I ...
1
vote
1answer
109 views

What is user agent Issbot

I am seeing a lot of traffic to my website from user agent Issbot. This certainly looks like a bot, but is not listed in the www.user-agents.org listing, and I have not been able to find anything ...
2
votes
1answer
166 views

A single request appears to have come from all the browsers? Should I be worried?

I was looking over my site access logs when I noticed a request with the following user agent string: "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.12) Gecko/20101026 ...
0
votes
1answer
319 views

Website won't load in Firefox 3.6.13

After the latest version of Firefox came out, I've been having problems with my website. The site is: http://mojheroj.com It works fine in all other browsers, as well as older versions of Firefox and ...
7
votes
3answers
2k views

What user agent should I set?

There is Ask bot, which sets this header: Mozilla/2.0 (compatible; Ask Jeeves/Teoma) Considering this, I have the following questions: If I'm writing a web crawler named Goofy, what user agent ...
6
votes
3answers
306 views

Does the user agent in any regular browser contain 'bot' or 'crawl'?

Does the user agent in any regular browser contain 'bot' or 'crawl'? I check the user agent on my site to see if it is coming from a bot or not. If it is, I can do some little optimizations since ...
1
vote
2answers
271 views

What browser is sending user agent beginning mozilla/5.0+, tramslates & into &

We've got a website which has been running for a few years now. One of our customers has just started having an intermittent problem. Looking at our iis6.0 logs the service works correctly when they ...
8
votes
4answers
7k views

Mobile phone detection (brand, model, browser etc)

What do you use to detect visitor's mobile phone, down to the model if possible? Currently we maintain our own database but it's really getting behind due to lack of manpower to maintain it, so we ...
23
votes
4answers
852 views

Is Yahoo Search the same as Bing Search now?

We had some serious issues with Yahoo's (incredibly) badly written spider in the past, and as a result we blocked them. Marco Arment of Tumblr also shared his frustrations with us on 8/31/2009, which ...
7
votes
2answers
667 views

How do you direct iPhone/Android browser to m.example.com?

I have some special coding that only works for HTML5 browsers (simple geolocation stuff). I want to redirect iPhone and Android users to the m.example.com version of my website. How is the best way ...