Tell me more ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.
up vote 5 down vote favorite
1

We periodically (more frequently recently) have very agressive crawling activity coming from EC2 instances that give us a user agent that looks like "AF_ID=". I've looked around for common User-Agent formats and I cannot seem to find any references to these User Agents. Do any of you know what these would be, or is it just a generic user-agent used by EC2 instances?

share|improve this question
Do you have sofware on your server that accesses the pages internally? Its possible to set user-agents to anything melodramatically. Makes it possible that its a bot or software. – Frank Jun 26 '12 at 7:19

1 Answer

up vote 2 down vote

It's just some random scam/spam crawler hosted on EC2. My only advice, add this to your .htaccess :

RewriteCond %{HTTP_USER_AGENT} .*AF_ID=* [NC]
RewriteRule ^.*$ http://english-1335426938.spampoison.com/English [R=303,L] 
#http://tinyurl.com/troll-face-bot :)
share|improve this answer
And you're basing that "random spam" assessment on what? – Su' Jul 31 '12 at 18:34
1  
On the fact that it's not an usual user agent, and agressive crawling activity. It's the usual flag to ban... – Dark Aug 6 '12 at 9:24
2  
@Su' respectable bots link to their policy page in a url in the user-agent – ZJR Aug 31 '12 at 4:18

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.