2

I bet google is checking web pages via HTTP HEAD and HTTP GET request methods. So I'm thinking I should also allow proper header output for downloadable files from my website via HTTP HEAD request but no documentation tells me the proper way to go.

Currently this is my HTTP header for a downloadable file when one requests it from my server with either a GET or HEAD request method: 

HTTP/1.1 200 OK
Date: Sun, 06 Sep 2015 06:11:29 GMT
Server: Apache
Set-Cookie: CC=something;path=/;domain=example.com;expires=Tue, 06 Oct 2015 06:11:29 GMT
X-Robots-Tag: noindex
cache-control: max-age=864000,must-revalidate
Last-Modified: Sun, 06 Sep 2015 02:39:19 GMT
content-disposition: attachment; filename="HQ_Photo8.jpg.gz"
Content-Length: 251085
X-Frame-Options: DENY
Accept-Ranges: bytes
Content-Type: application/gzip

What headers should I add or remove to comply to standards when someone requests the download via a HEAD request method?

I read with a standard html document that when one requests it with a HEAD method that the "content-length" header should be removed, but since this is a downloadable file I'm dealing with, I think its more involved.

Any ideas?

Improve this question
3
  • 1
    "I bet google is checking web pages via HTTP HEAD" - do you have any evidence of this?
    – MrWhite
    Sep 6, 2015 at 12:32
  • at this time, no, but google checks webpages to make sure they meet their quality guidelines before being indexed. I'm just trying to be one step ahead of google so that I can make more than 5 cents a day on adsense. In the past I made a lot more and the real visitor count has been consistent.
    – Mike -- No longer here
    Sep 6, 2015 at 17:14
  • @w3d You will hear/read people argue the notion of Google and a HEADrequest back and forth with people saying they have never seen it. This includes a statement from Cutts that Google never uses HEAD requests, however, I remember reading in their own documentation some years ago that a HEAD request can be made to check things like modification dates to see if a resource has changed. I believe this is a true statement, however, I rather suspect that now that Google has more bandwidth, a HEAD versus a GET is more redundant that useful. I seem to remember HEAD requests from Google in the past.
    – closetnoc
    Sep 6, 2015 at 23:03

1 Answer 1

Reset to default
2

The HTTP response headers for HEAD and GET requests should be identical. The only difference is that the response body is missing from the HEAD response.

From RFC 2616 - HTTP/1.1 Section 9.4 HEAD:

9.4 HEAD

The HEAD method is identical to GET except that the server MUST NOT return a message-body in the response. The metainformation contained in the HTTP headers in response to a HEAD request SHOULD be identical to the information sent in response to a GET request. This method can be used for obtaining metainformation about the entity implied by the request without transferring the entity-body itself. This method is often used for testing hypertext links for validity, accessibility, and recent modification.

The response to a HEAD request MAY be cacheable in the sense that the information contained in the response MAY be used to update a previously cached entity from that resource. If the new field values indicate that the cached entity differs from the current entity (as would be indicated by a change in Content-Length, Content-MD5, ETag or Last-Modified), then the cache MUST treat the cache entry as stale.

There would not seem to be any reason to omit the Content-Length header from the response (unless it is perhaps computationally expensive to include it). The spec specifically mentions the inclusion of this header and if it was omitted it could reduce functionality.

I read with a standard html document that when one requests it with a HEAD method that the "content-length" header should be removed, but since this is a downloadable file I'm dealing with, I think its more involved.

Where did you read this?

There isn't really any difference between an "HTML document" and a "downloadable file".

Improve this answer
4
  • I probably misread, but I looked here: stackoverflow.com/questions/3854842/…
    – Mike -- No longer here
    Sep 6, 2015 at 17:21
  • The accepted answer there quite clearly indicates that the content length header should be present for responses to HEAD requests. One of the uses for HEADs is to allow useragents to see the size of a resource to see whether they want to download it.
    – Tim Fountain
    Sep 6, 2015 at 20:26
  • One of the other answers on that question does suggest a valid reason for perhaps omitting the Content-Length header on a HEAD response... if it is computationally expensive to calculate it. (For example, if your page requires 100 DB lookups - or something - in order to generate a response to simply calculate its size for a HEAD response then it's up to you whether you think it's "worth it".) Otherwise, the spec and all the answers to that question do state that it should be included.
    – MrWhite
    Sep 6, 2015 at 21:56
  • It is also worth noting (as mentioned in the linked question), the definition of "SHOULD" in terms of the RFCs. It is a (strong) recommendation, not a strict requirement.
    – MrWhite
    Sep 6, 2015 at 22:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.