2

Amid increasing complaints of excessive resource usage from a shared webhost, a detailed check on the Raw Access Log files reveals repeat instances of the following from various IP addresses:

18.232.53.251 - - [04/Oct/2022:13:32:09 -0400] "GET /folder1/folder2/t-shirts-xl.php HTTP/2.0" 301 278 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/69.0.3497.81 Safari/537.36"
34.239.154.216 - - [04/Oct/2022:13:32:08 -0400] "GET /folder1/folder2/skirts-xl.php HTTP/2.0" 301 278 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/69.0.3497.81 Safari/537.36"

44.201.68.39 - - [05/Oct/2022:13:32:10 -0400] "GET /folder1/file.php HTTP/2.0" 301 284 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/69.0.3497.81 Safari/537.36"
44.202.197.233 - - [05/Oct/2022:13:32:10 -0400] "GET / HTTP/2.0" 301 225 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/69.0.3497.81 Safari/537.36"
54.234.97.69 - - [05/Oct/2022:13:32:10 -0400] "GET /folder1/file.php HTTP/2.0" 508 288 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/69.0.3497.81 Safari/537.36"
34.229.153.218 - - [05/Oct/2022:13:32:10 -0400] "GET /folder1/file.php HTTP/2.0" 508 288 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/69.0.3497.81 Safari/537.36"
18.207.251.140 - - [05/Oct/2022:13:32:10 -0400] "GET /folder1/file.php HTTP/2.0" 301 284 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/69.0.3497.81 Safari/537.36"
3.85.143.170 - - [05/Oct/2022:13:32:10 -0400] "GET /folder1/file.php HTTP/2.0" 507 389 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/69.0.3497.81 Safari/537.36"
34.228.165.128 - - [05/Oct/2022:13:32:10 -0400] "GET /folder1/file.php HTTP/2.0" 507 389 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/69.0.3497.81 Safari/537.36"
3.238.27.211 - - [05/Oct/2022:13:32:10 -0400] "GET /folder1/file.php HTTP/2.0" 200 26242 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/69.0.3497.81 Safari/537.36"
54.160.17.73 - - [05/Oct/2022:13:32:10 -0400] "GET /folder1/file.php HTTP/2.0" 301 276 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/69.0.3497.81 Safari/537.36"
54.221.4.72 - - [05/Oct/2022:13:32:10 -0400] "GET /folder1/file.php HTTP/2.0" 200 24711 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/69.0.3497.81 Safari/537.36"
54.167.45.247 - - [05/Oct/2022:13:32:10 -0400] "GET /folder1/file.php HTTP/2.0" 200 27012 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/69.0.3497.81 Safari/537.36"
3.238.24.9 - - [05/Oct/2022:13:32:10 -0400] "GET /folder1/file.php HTTP/2.0" 301 260 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/69.0.3497.81 Safari/537.36"
3.93.220.155 - - [05/Oct/2022:13:32:10 -0400] "GET /folder1/file.php HTTP/2.0" 301 269 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/69.0.3497.81 Safari/537.36"
54.224.92.26 - - [05/Oct/2022:13:32:10 -0400] "GET /folder1/file.php HTTP/2.0" 301 261 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/69.0.3497.81 Safari/537.36"
100.26.255.39 - - [05/Oct/2022:13:32:10 -0400] "GET /folder1/file.php HTTP/2.0" 301 277 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/69.0.3497.81 Safari/537.36"

Only the first entry (IP address like 18.232.53.251 in the first record) and the GET php filename is different. Otherwise, everything else seems to be repeating very frequently with huge no. of similar records within a minute's timeframe.

Any idea what these entries are for, and if they are contributing to excessive resource usage (Entry Processes in particular), and what can be done about them if they are causing high usage.

Addendum: If I do an IP check on the first field (IP addresses like 34.204.203.38 or 3.238.24.9), they all point to AmzonAWS (like ec2-54-160-17-73.compute-1.amazonaws.com). Obviously they are not from any genuine search engines like Googlebot or Bingbot. Any idea what is going on here, and more importantly - what can I do to prevent these hits from amazonaws?

3
  • "Headless" indicates that there is no monitor attached to the system, so it is some sort of bot that is rendering for some other purpose such as creating images or a DOM tree. Are the any more details you can include? I don't think I can say more than that based on just the information you provided. Oct 6, 2022 at 12:00
  • Thanks @StephenOstermiller - these are entries from the Raw Access Log files. Have included more sample records in the question above. Let me know if you are looking for anything else, or What more might help you please?
    – Aquaholic
    Oct 6, 2022 at 13:49
  • If I do an IP check on the first field (IP addresses like 34.204.203.38 or 3.238.24.9), they all point to AmzonAWS like ec2-54-160-17-73.compute-1.amazonaws.com. Obviously they are not from genuine search engines like Googlebot or Bingbot. Any idea what is going on here, and what can I do to prevent these hits from amazonaws?
    – Aquaholic
    Oct 6, 2022 at 14:59

1 Answer 1

1

You can take several approaches:

  • If using Cloudflare, or similar, you can block everything coming from AWS, by enabling a WAF rule to provide a challenge or directly block anything coming from ASIN 14618.

  • To block it on your Apache, you can set up a rule in .htaccess to block the offending IPs, User Agents, or a combination of both patterns. For example, block every User Agent that contains the word: HeadlessChrome

  • In addition, you can contact the abuse department on AWS and file a complaint for them to track down the offending user and shut him down.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.