We periodically (more frequently recently) have very agressive crawling activity coming from EC2 instances that give us a user agent that looks like AF_ID=<xxxxxx>.

I've looked around for common User-Agent formats and I cannot seem to find any references to these User Agents. Do any of you know what these would be, or is it just a generic user-agent used by EC2 instances?

  • Do you have sofware on your server that accesses the pages internally? Its possible to set user-agents to anything melodramatically. Makes it possible that its a bot or software. – Frank Jun 26 '12 at 7:19

It's just some random scam/spam crawler hosted on EC2. My only advice, add this to your .htaccess :

RewriteCond %{HTTP_USER_AGENT} .*AF_ID=* [NC]
RewriteRule ^.*$ http://english-1335426938.spampoison.com/English [R=303,L] 
#http://tinyurl.com/troll-face-bot :)
| improve this answer | |
  • And you're basing that "random spam" assessment on what? – Su' Jul 31 '12 at 18:34
  • 1
    On the fact that it's not an usual user agent, and agressive crawling activity. It's the usual flag to ban... – Dark Aug 6 '12 at 9:24
  • 2
    @Su' respectable bots link to their policy page in a url in the user-agent – ZJR Aug 31 '12 at 4:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.