4

I have a WordPress website with AWS on use, namely the Cloudfront service, to serve CSS, images and JS from the cloud.

Lately, I noticed a lot of hits from bots coming from :

  • IP: 54.236.71.87 Hostname: ec2-54-236-71-87.compute-1.amazonaws.com
  • IP: 54.147.229.75 Hostname: ec2-54-147-229-75.compute-1.amazonaws.com
  • IP: 34.207.96.105 Hostname: ec2-34-207-96-105.compute-1.amazonaws.com
  • IP: 52.202.239.36 Hostname: ec2-52-202-239-36.compute-1.amazonaws.com
  • IP: 34.203.222.34 Hostname: ec2-34-203-222-34.compute-1.amazonaws.com
  • ...

Goeip trace them to Ashburn, USA. They crawl all the RSS feeds of my website (posts, categories), almost every minute, coming from https://www.google.com/.

Their user-agent are a bit random :

Browser: undefined
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9a3pre) Gecko/20070330

or :

Browser: Chrome version 0.0 running on MacOSX
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1309.0 Safari/537.17

or :

Browser: undefined
Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.13; ) Gecko/20101203

How can I find out what they want ? Are they related to the cache generation for Cloudfront ?

I wonder because in the last 9 days, more than half of the bandwidth of my hosting has been consumed by unidentified bots.

3

These look to me like bots/crawlers running on EC2 instances, rather than anything to do with Cloudfront. AWS publish a list of IP ranges used by Cloudfront here : https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/LocationsOfEdgeServers.html

They're probably harvesting content or email addresses, looking for vulnerable apps, open comment forms where they can post comment spam, or something else not to your benefit. Sadly it's not uncommon for a lot of your traffic to come from bots like this.

You might want to block those IP addresses in the security group for your instance or ELB. If those IP's are hitting your origin servers rather than coming through cloudfront, and you only use cloudfront to access those servers, you might consider using something like this which blocks everything except the cloudfront IPs https://github.com/awslabs/aws-cloudfront-samples/tree/master/update_security_groups_lambda

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.