Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

If you go to (say) the last 100 entries (visits) to the botsvsbrowsers.com website (exact link, feel free to take a look: http://www.botsvsbrowsers.com/recent/listings/index.html ), you'd notice that almost every User Agent that has the keywords "Opera" and "Presto" inside them, will almost certainly have a web link (URL/Web Address) inside it, and it won't just be a normal web address, but a HTML anchor tag/link to that address. Why is this so, I could not even find a single discussion about it on the internet, nowhere, I tried varying my search terms many times.

If the user agent contains the words "Opera" and "Presto" it doesnt mean it will have this weblink, but it means there is about an 80% change that it will. A typical anchor tag/link inside a user agent will look like this:

Mozilla/4.0 <a href="http://osis-uk.co.uk/disabled-equipment">disability equipment</a> (Windows NT 5.1; U; en) Presto/2.10.229 Version/11.60

If you check it out at the website, http://www.botsvsbrowsers.com/recent/listings/index.html you will notice that the back and forward arrows are in there unescaped format.

This isn't just true for botsvsbrowsers, but several other user agent listing sites. I'm really confused and feel line I'm in a room full of 10,000 people and am the only one seeing this ghost :).

If I'm doing statistical analysis, should I include or exclude this type of user agent from my listing (ie: are these just normal users who've set their user agents to attempt to drive some traffic to their sites as they browser the web), or is there something else going on? The fact that it is so consistent in terms of its format leads me to believe that it is an automated process (the setting or alteration of the user agent) so I cannot decide or understand the process by which this change is made (I know how to change a user agent), but unsure which program or facility is doing this, especially since it is exclusive to Opera (Presto) user agents that are beyond I think an 8 or 9 point something browser version.

I've run some statistical tests, parsing entries from all over the place, writing custom programs, to get a better understanding of this. Keep in mind that I see normal URL's in user agents infrequently, they are just text such as +http://www.someSite.com appended to a user agent normally, especially if its a crawler or bot it provided its service URL, this is normal and isnt done with an embedded link (A HREF=) etc, so I'm not talking about "those".

share|improve this question
    
I noticed a lot of this UA strings in my log. They use Mozilla 4.0 or 5.0 as well. Nearly all - 928 from 934 - UA strings contain "Presto". I assume it is a variant of referer spam. –  user27928 May 11 '13 at 23:13
add comment

2 Answers

Looking at bots vs. browsers, they display every user agent that's ever visited their page. Some clever spammer realised that this would be a clever way to drive traffic to their sites, because webmasters/anyone looking at the site is probably going to wonder why there's a url in the user agent, think it's a new specification or something, and visit the url trying to learn more.

Opera's user agent has no url in it, btw. And bots vs browsers lists a lot of other user agents supposedly from other browsers that also have urls in them.

These can also show up in your server logs too.

Keep in mind, though, that a lot of bots list urls in their user agent (Google does this, for example), so that people can read more about their crawler.

share|improve this answer
1  
+1 for GoogleBot note –  Adam Lynch Sep 24 '12 at 18:18
    
Thanks for your reply (upvoted), my secondary question is, why it is always Opera/Presto User-Agents with the live anchor tagged link in them? I never see this in in User-Agents that are not Opera/Presto. Iv analysed over 44,000 user agents now. –  Erx_VB.NExT.Coder Sep 27 '12 at 16:32
    
@Erx_VB.NExT.Coder: They are not always Opera/Presto. Search for href= on botsvsbrowsers.com and it returns the top 500 results with HTML anchor-like user agents - only a very small number of these include the words "opera" and/or "presto". –  w3d Sep 27 '12 at 17:40
    
@Erx_VB.NExT.Coder did this answer your question? If it did, could you accept it so that we know you solved your problem, and if it didn't, could you leave a comment explaining how it could be improved? –  Christofian Oct 7 '12 at 19:27
add comment

I think that by including a complete HTML anchor in the user agent the spammer is simply hoping that the target website is going to display the complete user agent unencoded (so the HTML is rendered), possibly in unprotected stats pages, and thus benefit from some free linkage.

This is a similar principal to referer spam.

Displaying an unencoded user agent string in the webpage is an obvious coding mistake (security vulnerability), but there might be enough mistakes in the millions of spammed sites for this trick to pay off!?

I don't think they expect to benefit from the curious developer (where the link appears as an HTML anchor) who is casually flipping through their logs. "disability equipment" anyone?!

share|improve this answer
    
I've also seen links to hosting sellers, and it's feasible that someone could make a fake bot user agent and add a spam url that people would visit. –  Christofian Sep 23 '12 at 13:44
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.