3

I currently have apache timeout incomplete requests after 21 seconds because I don't want bad bots holding up the available connection slots, and I feel people would rather wait 21 seconds for a slot to free up than (apache timeout default of) 300 seconds.

After log research, I see entries like this:

(remote ip address) - - [17/Mar/2016:10:29:13 -0400] "-" 408 - "-" "-"

Most of the time, they come from mobile devices connected to mobile networks, and a 408 code suggests to me to increase the timeout, but I thought 21 seconds is plenty for a local mobile device to connect to a local server. By local, I mean within about 1000 km (from central ontario to central quebec). The funny thing is when I tested from various locations from California USA via webpagetest.org, the entire webpage, ads and all load on all devices in well under 21 seconds.

Am I still specifying too low of a timeout? if so? why? or is it just some networks that are trying to connect to my site are poorly configured? or is it just that some mobile devices themselves are that terrible with connecting to websites?

These lines appear between 10% and 30% of the time.

  • I've noticed my CDN causing these kinds of issues but only with bots - such as Google. Do you use a CDN? – L Martin Mar 17 '16 at 21:43
  • No CDN for me because my site is updated at least 3x a week if not more, and visitors want fresh content right away (not wait till the CDN cached it) – Mike Mar 17 '16 at 22:00
  • I guess the best you can do is keep simulating high latency in Chrome Dev with a mobile, and seeing if there's any issues while the server is under high load. If users can't get to your site in 20 seconds they probably don't have a sufficient connection to. – L Martin Mar 18 '16 at 6:50
  • 2
    "No CDN for me because my site is updated at least 3x a week if not more, and visitors want fresh content right away (not wait till the CDN cached it)" <-- This is an incorrect assumption, any modern CDN detects changes at the origin and serves the fresh content. This is a solved problem, it's how they work. You can update your site every minute and the CDN will easily keep up. (You'll get a lot more cache misses though, as each new origin change is fetched and then cached for subsequent requests). – Tom Brossman Mar 18 '16 at 8:54
  • I'm not sure if the free ones would do that. I tried a CDN once on one subdomain of my site (when I had subdomains) and the load time for me was slower (probably because the CDN is in USA and my computer and the server are both in canada). – Mike Mar 19 '16 at 1:43
1

Ok I figured it out from hard research. The answer is 21 seconds is WAY TOO HIGH! This is because an attack called Slowloris exists. What it does in a nutshell is hold up apache slots for about as long as the timeout value specified in apache config. This in turn prevents legit users from even connecting to the website. Because connections now need less than 1 second to setup, a timeout over 3 seconds for me is bad. (of course I'm trying to help some mobile clients with somewhat weak connections which is why I'm not going to a flat 1 second timeout)

Here's an article on slowloris.

https://www.incapsula.com/ddos/attack-glossary/slowloris.html

  • You shouldn't need to adjust the time outs... if your visitors are getting time outs then invest in good hosting with multiple redundancies and fall-overs. – Simon Hayter Mar 28 '16 at 17:45
  • After lowering timeouts my income went up again. I bet anything google is measuring speed to the extreme. Heck. I might get downvotes now but if this makes me even better money, ill follow through – Mike Mar 29 '16 at 2:30
  • Ads will always vary from month to month and can, and regularly does vary a lot from time to time for no valid reason. It's the same why some shops get unexplained increase in sales some weeks than the previous year. – Simon Hayter Mar 29 '16 at 7:56
  • True, but I noticed consistent patterns. About 1/2 a year back I was making at most 6 cents a day in adsense RPM. It turned out because of a readahead setting on my server. Now I'm usually making more than 6 cents a day. I think google is taking server speed very seriously – Mike Mar 29 '16 at 17:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.