http request terminating early

Question

I noticed that on some of my sites, images were occasionally not getting downloaded fully. After a bit of investigation it appears that it is not restricted to images - .css, .js etc were also occasionally terminating early. The faults appear to be random.

When I use the debug/proxy tool Fiddler2 reports that fewer bytes have been received than were requested. Firebug reports "Image corrupt or truncated".

Obviously this is mainly a concern between me and my hosting company. However despite many emails they have not been able to get to the bottom of it. Transfer to another hosting company is obviously an option but is really a last resort.

Has anyone seen this kind of thing before or can anyone suggest what might be causing it. Or any apache setting or something that I can ask them to check out. Will apache log this kind of error - they havent been able to provide me with any logs, but if I know exactly where things have been logged, maybe I can prompt them in to action.

EDIT: More data as requested by commenters..

I created a test page. It is pure HTML and embeds 9 images each of size between 4 and 6MB. This is a very large amount of data. I found having so much data speeded up the occurrence of faults which makes benchmarking easier. As a control, I run the same test on the server of another hosting company that I have access to - I never get faults when running on this other host.

It is on shared hosting.

I have tested it on FF, IE & Chrome.

EDIT - DETAILS OF HEADERS ETC ...

(In case your wondering, this version of the test is not pure HTML but has a small amount of PHP added to to prevent caching.)

LiveHeaders o/p... https://docs.google.com/file/d/0B93lksnTC7_cYVhQT2lMbms5Mm8/edit

What you see in Firefox... https://docs.google.com/file/d/0B93lksnTC7_ceHBSbmhXWHhXdlk/edit

What Firebug logs... https://docs.google.com/file/d/0B93lksnTC7_cOEdmeU1lMkh4UE0/edit

If you have difficulty getting into google docs - alternative... https://skydrive.live.com/redir?resid=9F8DE44BC9AC7FD6!107&authkey=!AAXurFZleJE5okQ

As you can see the images that failed were: DSC_0046_0.JPG DSC_0232_0.JPG

EDIT - UPDATE

I've noticed that it gets worse if I simultaneously run several copies of the test. (And running several copies of the test on the control host still passes.) Further suggesting that if the server gets too many requests it might be terminating some requests early or something.

Answer 1

As far as I can see, there seems to be a problem with caching and the freshness of the information, in this case, the images, but I still don't have a clear answer.

In the headers docs you posted, you can see a response header with a 206 code, that means that the server is responding to a request for partial content, the question is why is it happening.

The first thing you should try, is add an etag to all files, that should help, you can do that with this code in your .htaccess

FileETag All

If you can also set the cache-control, it will be of help also, you can do it in your .htaccess with a code like this:

<IfModule mod_headers.c>
    Header unset Cache-Control
    <FilesMatch "\.(ico|jpg|png|gif)$">
        Header set Cache-Control "max-age=31536000, public, s-maxage=31536000"
        Header set Pragma "cache"
    </FilesMatch>
</IfModule>

Try those changes to see if the situation improves, then if you can log the headers again, it and paste it, we can check the differences.

If you don't like those values, you can change them, but hose should be good for some tests.