In the Hypertext Transfer Protocol (HTTP), HTTP header fields contain the operating parameters of an HTTP request or response. With the request or response line (first line of message), they form the message header.

learn more… | top users | synonyms

1
vote
1answer
59 views

When using resources from cache Firefox show a “304” response but Chrome says “220 (from cache)”

While working on some optimizations for my site, I noticed some things in Networking Timings for Chrome Developer Tools differ from those in Firefox Developer Tools. Very consistently Chrome loads a ...
2
votes
2answers
37 views

Is there a way to block wget from crawling and downloading my website?

I do not want ANY of my alleged website to be crawled and downloaded (css, images, html, etc). Is there a way to block wget completely?
2
votes
0answers
17 views

Handling bots that request URLs in paths

In my server log, I found at least one IP address to be requesting a full URL in an awkward place. For example, the header the client sends to my server is this: GET ...
1
vote
0answers
36 views

Is a Server and Date HTTP header really necessary for error or redirecting pages?

On practically every URL I visit using the command-line tool CURL, there is always a server identification header and date header outputted regardless of the status of the request. For example, on my ...
2
votes
1answer
328 views

Can a past Expires header value cause a site to not be indexed by Google?

I'm using this: <filesMatch "\.(html|htm|js|css)$"> FileETag None <ifModule mod_headers.c> Header unset ETag Header set Cache-Control "max-age=0, no-cache, no-store, ...
1
vote
2answers
39 views

Is it a good idea to close the connection on a web hacker / spammer?

When I receive a request from a web spammer, such as a GET /wp-login.php, I am thinking of replying with an error (404) and closing the HTTP connection immediately with this header: Connection: close ...
2
votes
3answers
230 views

When do browsers clear cache and how can I make more use of 304 responses?

I have a website with some some content which changes frequently. I would like to benefit from good caching when this doesn't change (so would ideally use a longer Cache-Control header), but would ...
9
votes
1answer
33k views

Font blocked from loading by Cross-Origin Resource Sharing policy: No 'Access-Control-Allow-Origin'

we are experiencing this error in Google Chrome. We thought everything was set-up correct. But maybe not? Font from origin http://skin.cdn.com has been blocked from loading by Cross-Origin ...
5
votes
4answers
2k views

What does it mean when a User-Agent has another User-Agent inside it?

Basically, sometimes the user-agent will have its normal user-agent displayed, then at the end it will have the "User-Agent: " tag displayed, and right after it another user-agent is shown. Sometimes, ...
2
votes
0answers
52 views

What strings do %{HTTP_HOST} and other mod_rewrite variables return? [closed]

If I have a htaccess file that has RewriteCond %{HTTP_HOST} in it, what is the HTTP_HOST? This is something I've been struggling to find out. To help me understand, could you tell me, in each example ...
2
votes
2answers
1k views

How do I modify the HTTP headers?

Google page speed rank demands me to set an expiry date or a maximum age in the HTTP headers, by changing the "Expires" and "Cache-Control: max-age". The site is hosted on a hosting company (not in ...
36
votes
2answers
33k views

Difference between the Accept and Content-Type HTTP headers

So the Accept header tells the server the MIME-type of the resource the browser is looking for. For example, the server can send plain text, HTML, JSON, etc. OK, that makes sense. But then I look at ...
2
votes
1answer
334 views

Trailing slash on Root folder - Is it possible to remove it without 301?

I have noticed something very strange on the site I'm working on. We currently only show http://www.example.com. The URL http://www.example.com/ (slash at the end) gets 301'd to ...
2
votes
1answer
31 views

requesting for downloadable files via HEAD

I bet google is checking web pages via HTTP HEAD and HTTP GET request methods. So I'm thinking I should also allow proper header output for downloadable files from my website via HTTP HEAD request but ...
2
votes
2answers
56 views

Caching a changing site

I run a website where the static content changes at least 4 times a week and I'm trying to figure out the best header arrangement to use. Currently I have all caching related headers in use. I have ...
1
vote
1answer
28 views

Can servers ever return headers not beginning with HTTP?

When I checked URLs for headers, they always start with HTTP. For example, HTTP/1.1 301 Moved Permanently. If I remember, I think in the past, headers started with Status: followed by a code. ...
1
vote
1answer
115 views

How to serve for International Googlebots if we redirect users to subdomains for language matching?

So, we have a site and a few sub-domains with content in different languages and we're using Hreflang and canonical implementations for SEO. We do detect the language that users use in their browsers ...
1
vote
2answers
72 views

HTTP Header broken? Browsers don't open our pdf, but open all other pdfs

My Firefox and Chrome usually open PDF files very nicely, without asking and within the browser window. When clicking on a link to a PDF on our own website: Firefox asks if it should download them, ...
0
votes
1answer
52 views

Eliminating 'cookie' from 'vary:' without breaking website functionality

I read that some middle-man servers won't cache websites that have cookie set in the vary: header. For example, If my website issues a vary:accept-encoding,cookie as opposed to vary:accept-encoding ...
0
votes
1answer
34 views

Is it okay for url to end .html even if stored gzipped?

I have some html files stored on Amazon s3, some are quite large so the plan was to store them gzipped and return that as the encoding type when receive a request for them, is that how it works rather ...
7
votes
4answers
3k views

How to tell the Browser the character encoding of a HTML website regardless of Server Content-Type Header?

I have a HTML page that correctly (the encoding of the physical on disk matches it) announces it's Content-Type: <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html> ...
1
vote
1answer
61 views

Vary: User-Agent in case of redirect?

Should we use Vary: User-Agent in our headers when the regular url www.mydomain.com is forwarded to m.mydomain.com? Or is it not needed for Google due the redirect?
1
vote
1answer
166 views

Earliest browsers that support if-modified-since or if-none-match headers

Based on my observations, google and probably many others are in love with websites that include the Etag and Last-modified headers along with the if-modified-since and if-none-match header checking ...
2
votes
1answer
86 views

Trying to add cache header to 301 pages made by ModRewrite

I have several URLs on my website that get redirected to other URLs using Mod-Rewrite. For example: RewriteRule ^section$ /newsection [NC,L,R=301] The above rule redirects ...
8
votes
1answer
378 views

If I only serve gzipped version of my content, should I add the vary accept encoding header?

I just moved a static site from a VPS to Amazon S3. I have decided to serve only gzipped version of my pages, since S3 is not a webserver I can't have logic based on headers. I also use Cloudfront as ...
0
votes
0answers
150 views

X-Robots-Tag: noindex on an HTTP 302 response to remove subdomain

This very related to this question "X-Robots-Tag: noindex" on an HTTP 301 response, however I don't see a definitive answer. We have subdomains we use for our affiliate links, the ...
0
votes
2answers
53 views

What is the proper response header when an address is not available but its parent is available and is displayed?

My website addresses has a hierarchy structure. When one page is requested, and it is not a valid address, it tries to find its parent, and if the later exists, it will show it. Now, my question is ...
3
votes
2answers
1k views

CloudFlare Not Sending If-Modified-Since Headers

I'm trying to reduce my server load by providing 304 Not Modified responses for content where appropriate. CloudFlare is my middle man, so they should be sending If-Modified-Since headers whenever a ...
6
votes
3answers
464 views

How should an expired content be handled?

Let's imagine this simplified case: An auction website has "auction detail" pages. A few weeks after the auction is over, the "auction detail" page is no more available. We are simply serving a ...
18
votes
2answers
29k views

Headers to prevent 304/If-modified-since/HEAD requests

What headers should I be sending to outright stop all requests to the server after the content has been cached? We have a very high latency server (Sigh, VMWare) so even sending a HEAD request to the ...
1
vote
0answers
356 views

Apache 404 Issue after upgrading from Mavericks to Yosemites

Good afternoon fellow StachExachangers, Background story: I have recently upgraded my macbook pro from Mavericks to Yosemites, everything went well except my local development environment got ...
1
vote
1answer
2k views

Caching of JS and CSS files – Apache mod_headers doesn’t take effect

As suggested by an answer on SO, I'm trying using a version number on my JS and CSS files so that the browser will only make a request when the JS or CSS version number is changed in my PHP file. To ...
0
votes
1answer
36 views

Does referrer header checking offer any real world security improvement?

At work we use a central portal that provides basic SSO functionality to other applications. In addition to verifying the SSO data sent, all of our existing in-house applications (used by the public) ...
1
vote
1answer
130 views

Did I use the proper way to remove pages from Google Webmaster Tools forever?

I have many pages from one year ago that were deleted and are returning a 404 error code, but two week ago after I cleared the content of my robots.txt, the errors in my Google Webmaster Tools reports ...
1
vote
2answers
500 views

Redirect to an affiliate using PHP or JavaScript redirect?

I was asked to introduce a unified self hosted URL-Shortener/Forwarder/Tracking Service for an website that earns money from affiliates. One example I found was one of the bigger german sites for ...
1
vote
1answer
216 views

Use different .htaccess file based on set header from browser

I want to have the ability to test .htaccess rules before saving them to the main .htaccess file. Is it possible to set a specific request header and have Apache use a set of rules from a different ...
0
votes
1answer
44 views

What is best practice for search engines when a website is under maintenance?

I need around a week to transition a heavily data-driven website from one back end to another. During that time I do plan to attempt to keep some pages live, but they won't all work well or look ...
1
vote
1answer
41 views

Can serving a different header code for crawlers and users be considered cloaking?

I have urls that point to pages I don't know yet if exist. Whenever reaching those, I perform a not-so-quick external sources check, and if exists generate the page on the fly. In order to avoid this ...
4
votes
1answer
2k views

How to get keywords from Google HTTP referrer?

Can the http referrer URL from Google be decoded so that I can see the specific keywords? ...
2
votes
1answer
1k views

Can I set Cache-Control in the .htaccess file for a specific part of a document (just a section of the <body>)

I noticed, by using Google Page Speed Insights, that if I structure my HTML to load the critical, above-the-fold content first, my page will improve performance - and it woks (with now current perf. ...
2
votes
1answer
73 views

Why was 303 skipped in HTTP/1.0?

While looking back over HTTP response codes today I noticed that the 301, 302, and 304 response codes were defined in HTTP/1.0 but not the 303 response code. Does anyone know why the number 303 was ...
1
vote
2answers
2k views

Lots of HTTP HEAD requests originating from porn sites

The access log on my web server has a ton of HTTP HEAD requests coming from porn sites. What are HEAD requests and are they doing something bad with my site? Here is an excerpt from my log: (valid ...
1
vote
3answers
461 views

302 Moved Temporarily or 301?

HTTP status code checker tool shows HTTP/1.1 302 Moved Temporarily for the home page URL http://someurl.com (just a namesake URL). Also, this URL opens up http://www.someurl.com/general/index. As you ...
4
votes
1answer
3k views

Is it necessary to specify content-type in http response header?

Except for the html document, currently all the files (css, javascript and images) served from my web server do not specify the content-type in the http response header. I happen to notice this as I ...
4
votes
2answers
424 views

Why doesn't this web crawler obey requests to stop crawling my pages?

I have a hidden page that verifies internal payments from PayPal. The page has the following in its HTTP headers: X-Robots-Tag: noindex, nofollow But once in a while I see this in the page logs: ...
12
votes
1answer
18k views

Character set not specified in HTTP headers error

When testing this page with Page Speed, I get the a Specify a character set error: The following resources have no character set specified in their HTTP headers. Specifying a character set in HTTP ...
7
votes
1answer
332 views

Why is “29030400” seconds used as a common value for cache expiration?

I noticed that 29030400 is used very often in the Expires directives for static files. Google recommends to cache this kind of files up to 1 year (1 month at least). I did the math: 29030400 seconds ...
2
votes
1answer
116 views

How to configure apache2 to just save certain POST requests without even passing them to application?

I'm running Apache in front of glassfish server using BalancerMember. For performance reasons I would like that POST requests on certain endpoint are just saved to a file without passing them to ...
3
votes
1answer
361 views

Should we allow HEAD requests?

Our web application is predominantly read-only. As such we only allow GET requests. However, we regularly get errors in our logs because of user agents making HEAD requests. I've seen these come from ...
2
votes
1answer
154 views

What's the best HTTP code for dynamically deleted pages?

I've built my own CMS and I'm looking for the best way to handle pages / items deletion. Actually, pages and items were deleted and cannot be restored. But since I have implemented a bin, I don't ...