0

Updated:

I used Chrome inspect and got this,

enter image description here

What I did on my EC2 is to extend AWS EBS volumes from 10G to 13G, and rebooted my machine. Is it relevant to my issue?

My disk usage:

$ df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            965M     0  965M   0% /dev
tmpfs           200M  620K  199M   1% /run
/dev/xvda1       13G  7.5G  4.6G  62% /
tmpfs           996M     0  996M   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           996M     0  996M   0% /sys/fs/cgroup
tmpfs           200M     0  200M   0% /run/user/1000

I hosted a WordPress site on Amazon EC2. I visited my homepage. It is processing but no response (no error message) after waiting for a long time.

I also tried wget example.com, and got this,

$ wget example.com
--2020-04-25 16:17:20--  http://example.com/
Resolving example.com (example.com)... 52.68.57.153
Connecting to example.com (example.com)|52.68.57.153|:80... connected.
HTTP request sent, awaiting response... 

Read error (Operation timed out) in headers.
Retrying.

--2020-04-25 16:32:23--  (try: 2)  http://example.com/
Connecting to example.com (example.com)|52.68.57.153|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: 'index.html'

index.html                                  [  <=>                                                                         ] 174.08K   511KB/s    in 0.3s    

2020-04-25 16:32:23 (511 KB/s) - 'index.html' saved [178262]

index.html doesn't exist in my wordpress directory. Below is my .htaccess.

$ cat .htaccess 
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
DirectoryIndex index.php
</IfModule>

It took around 15 minutes to respond. I also checked log files under /var/log/. mysql.err, mysql.log, kern.log and php_errors.log are empty. My webserver is apache2.

The tail of /var/log/apache2/access.log is as the followings.

$ tail access.log
116.179.32.154 - - [25/Apr/2020:12:05:42 +0000] "GET /zh/?s=%E3%80%90%E5%BC%80%E4%B8%AA%E8%B5%9B%E8%BD%A6%E7%BE%A4%E8%A6%81%E5%A4%9A%E5%B0%91%E9%92%B1%E8%96%87%E4%BF%A1%EF%BC%96%EF%BC%93%EF%BC%99%EF%BC%99%EF%BC%99%EF%BC%90%EF%BC%96%E3%80%91%E6%98%AF%EF%BD%82%EF%BD%8A%EF%BD%81%EF%BD%8C%EF%BD%97%EF%BD%85%EF%BD%96%EF%BD%82%EF%BD%9A%E7%9A%84D18a39 HTTP/1.1" 302 759 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
115.202.67.140 - - [25/Apr/2020:12:06:10 +0000] "GET /robots.txt HTTP/1.1" 200 564 "-" "Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML like Gecko) Chrome/47.0.2526.108 Safari/537.36 2345Explorer/7.2.0.13379"
115.239.24.161 - - [25/Apr/2020:12:06:11 +0000] "GET /robots.txt HTTP/1.1" 200 564 "-" "Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML like Gecko) Chrome/47.0.2526.108 Safari/537.36 2345Explorer/7.2.0.13379"
117.83.201.8 - - [25/Apr/2020:12:06:11 +0000] "GET /robots.txt HTTP/1.1" 200 564 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36"
125.123.142.154 - - [25/Apr/2020:12:06:12 +0000] "GET /robots.txt HTTP/1.1" 200 564 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36"
46.229.168.131 - - [25/Apr/2020:12:06:12 +0000] "GET /zh/?s=%ED%95%98%EB%82%A8%EC%B6%9C%EC%9E%A5%EC%98%A4%EC%93%B0%ED%94%BC%EA%B1%B8%EF%BB%BF%5B%EC%B9%B4%ED%86%A1-%2BPo%2B3%2B4%5D%E3%80%8EPoo3%2B4.c%EF%BC%90M%E3%80%8F%EC%BD%9C%EA%B1%B8%EC%97%85%EC%86%8C%EC%98%A4%ED%94%BC%EA%B1%B8Y%E2%95%B3%E2%9E%B72019-04-05-14-54%ED%95%98%EB%82%A8%E2%9C%94AIJ%E2%96%BD%EC%BD%9C%EA%B1%B8%EC%B6%9C%EC%9E%A5%EC%95%88%EB%A7%88%EC%B6%9C%EC%9E%A5%EC%95%84%EA%B0%80%EC%94%A8%EC%98%A4%ED%94%BC%EA%B1%B8%EF%B9%83%EC%B6%9C%EC%9E%A5%EC%83%B5%EC%BD%9C%EA%B1%B8%E2%99%A1%EC%BD%9C%EA%B1%B8%EC%B6%9C%EC%9E%A5%EB%A7%88%EC%82%AC%EC%A7%80%E2%94%82%ED%95%98%EB%82%A8 HTTP/1.1" 302 1047 "-" "Mozilla/5.0 (compatible; SemrushBot/6~bl; +http://www.semrush.com/bot.html)"
222.68.129.199 - - [25/Apr/2020:12:08:08 +0000] "GET /favicon.ico HTTP/1.1" 200 236 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.81 Safari/537.36 SE 2.X MetaSr 1.0"
115.159.200.27 - - [25/Apr/2020:12:04:46 +0000] "GET / HTTP/1.1" 200 33024 "-" "DNSPod-Monitor/2.0"
220.181.108.153 - - [25/Apr/2020:12:11:04 +0000] "GET /en/?s=%E6%8B%93%E6%89%91 HTTP/1.1" 200 35788 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
46.229.168.161 - - [25/Apr/2020:12:11:06 +0000] "GET /?s=%E3%80%90%EA%B2%BD%EC%82%B0%EC%B6%9C%EC%9E%A5%EC%95%88%EB%A7%88%E3%80%91%E2%98%AA%EC%B6%9C%EC%9E%A5%EB%B6%80%EB%A5%B4%EB%8A%94%EB%B2%95%E2%98%80%EA%B2%80%EB%B9%9B%2B%EA%B2%BD%EB%A7%88%2B%EA%B2%80%EC%83%89%E2%86%95%E3%80%96%EC%B9%B4%ED%86%A1%3A%2Bmxm33%2B%E3%80%97%E2%97%8B%EF%BC%88%D1%88%D1%80f636.%D1%81%D0%BE%D0%BC%EF%BC%89%EC%95%84%EB%A7%88%2B%EC%8B%9C%EC%95%882019-04-19-23-06%EA%B2%BD%EC%82%B0%EA%B2%BD%EC%82%B0%5B%5D%EB%AA%A8%ED%85%94%2B%EC%A0%A4%E2%95%8B%EA%B2%BD%EC%82%B0%E3%83%AA%5B%5DaRr%EC%B6%9C%EC%9E%A5%EC%95%84%EA%B0%80%EC%94%A8%EC%B6%9C%EC%9E%A5%2B%EB%A7%8C%EB%82%A8%2B%EC%B9%B4%ED%86%A1%EA%B2%BD%EC%82%B0IoG%EC%BD%9C%EA%B1%B8%EA%B2%BD%EC%82%B0%EA%B2%BD%EC%82%B0%EC%97%AC%EC%9E%90%2B%EB%AA%A8%ED%85%94 HTTP/1.1" 200 33906 "-" "Mozilla/5.0 (compatible; SemrushBot/6~bl; +http://www.semrush.com/bot.html)"

Here is error.log,

$ cat error.log
[Sat Apr 25 06:25:03.436228 2020] [mpm_prefork:notice] [pid 1185] AH00163: Apache/2.4.29 (Ubuntu) mod_wsgi/4.5.17 Python/2.7 configured -- resuming normal operations
[Sat Apr 25 06:25:03.436249 2020] [core:notice] [pid 1185] AH00094: Command line: '/usr/sbin/apache2'

The result of ping,

$ ping example.com
PING example.com (52.68.57.153): 56 data bytes
64 bytes from 52.68.57.153: icmp_seq=0 ttl=37 time=103.808 ms
64 bytes from 52.68.57.153: icmp_seq=1 ttl=37 time=98.609 ms
64 bytes from 52.68.57.153: icmp_seq=2 ttl=37 time=99.717 ms
64 bytes from 52.68.57.153: icmp_seq=3 ttl=37 time=99.243 ms
64 bytes from 52.68.57.153: icmp_seq=4 ttl=37 time=97.699 ms
64 bytes from 52.68.57.153: icmp_seq=5 ttl=37 time=111.710 ms
64 bytes from 52.68.57.153: icmp_seq=6 ttl=37 time=99.019 ms
64 bytes from 52.68.57.153: icmp_seq=7 ttl=37 time=97.801 ms
64 bytes from 52.68.57.153: icmp_seq=8 ttl=37 time=98.201 ms
64 bytes from 52.68.57.153: icmp_seq=9 ttl=37 time=99.747 ms
64 bytes from 52.68.57.153: icmp_seq=10 ttl=37 time=101.267 ms
64 bytes from 52.68.57.153: icmp_seq=11 ttl=37 time=98.445 ms
64 bytes from 52.68.57.153: icmp_seq=12 ttl=37 time=101.986 ms
^C
--- qiankun.su ping statistics ---
13 packets transmitted, 13 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 97.699/100.558/111.710/3.643 ms

What else can I do to find out the reason?

  • What webserver are you using? What is in the log files for the webserver? The syslog is usually irrelevant for web serer requests. – Stephen Ostermiller Apr 25 at 12:05
  • @StephenOstermiller, apache2 – SparkAndShine Apr 25 at 12:08
  • What is in the apache acces_log and error_log files? – Stephen Ostermiller Apr 25 at 12:09
  • @StephenOstermiller, I just updated my question. Added the content of access.log and error.log. – SparkAndShine Apr 25 at 12:13
  • I notice your wget is saving to index.html, does that file exist in your wordpress directory? The index file (the file processed by default) should be index.php. If index.html exists, see what happens if you rename it to index.html.old – Steve Apr 25 at 22:37
0

I found the main reason: Bytespider insanely crawls my site, leading to a resource crunch.

60.8.123.241 - - [03/May/2020:00:02:30 +0800] "GET /?s=%E3%80%90%E2%94%8B%E6%80%BB%E4%BB%A32%E2%92%8F7%E2%92%8F01705%5B%E6%89%A3%5D%E3%80%93%E3%80%91%E5%8D%8E%E5%AE%87%E5%9B%BD%E9%99%85%E7%99%BB%E5%BD%95%E7%94%B5%E8%84%91%E7%89%88 HTTP/1.1" 200 33329 "-" "Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; Bytespider; https://zhanzhang.toutiao.com/)"
110.249.201.182 - - [03/May/2020:00:03:41 +0800] "GET /robots.txt HTTP/1.1" 200 487 "-" "Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; Bytespider; https://zhanzhang.toutiao.com/)"
110.249.202.183 - - [03/May/2020:01:08:50 +0800] "GET / HTTP/1.1" 200 36288 "-" "Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; Bytespider; https://zhanzhang.toutiao.com/)"

Refer to: What is the “Bytespider” user agent? [closed]

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.