2

The use of 'to cache or no-cache depends on the end performance sort: 1) Affiliate Marketers - need to 'cache tags' when pages etc go to landing pages as the cookies are needed for allocation of funds. 2) Informative/ transformational posts - use of 'non-cache tags' no-cookies as social trust factors are being built.


1

The content-type is used by browsers to figure out how to display the content. Browsers have to choose the rendering method based on the content type. Rendering plain text is very different than rendering HTML. wget performs the same action for all files: it saves them to disk. In my experience it doesn't pay attention to the Content-Type header, it ...


1

You may not be explicitly setting HTTP headers but wget certainly will be. If you run the same command with the -d (debug) flag you'll see more verbose output including the request and response headers: $ wget -d https://webmasters.stackexchange.com/robots.txt [...] ---request begin--- GET /robots.txt HTTP/1.1 User-Agent: Wget/1.20.1 (linux-gnu) Accept: */*...


Only top voted, non community-wiki answers of a minimum length are eligible