Request Missing a User Agent Header

Question

I activated ModSecurity on a cheap web hosting but I have no control over it: that is, the only thing the service provider lets you do is reading logs but, unfortunately, you can't really configure ModSecurity yourself.

This morning I found this log:

[Sun Sep 16 16:50:31 2018] 
[error] 
[client 214.250.181.113] ModSecurity: Access denied with code 403 (phase 2). Operator EQ matched 0 at REQUEST_HEADERS. 
[file "/usr/local/apache2/conf/modsecurity/base_rules/modsecurity_crs_21_protocol_anomalies.conf"] 
[line "65"] 
[id "960009"] 
[rev "2.1.1"] 
[msg "Request Missing a User Agent Header"] 
[severity "NOTICE"] 
[tag "PROTOCOL_VIOLATION/MISSING_HEADER_UA"] 
[tag "WASCTC/WASC-21"] 
[tag "OWASP_TOP_10/A7"] 
[tag "PCI/6.5.10"] 
[hostname "www.example.com"] 
[uri "/firepro/newfile.txt"] 

I verified this happens when my PHP code simply tries to read and echo the contents of newfile.txt using file_get_contents. This error breaks a Wordpress widget and renders it useless (it doesn't display any content, since the content is provided by the txt file overwritten every hour by PHP with the help of a cronjob).

What header is ModSecurity expecting? Why do I need to specify a User Agent header?

Answer 1

I verified this happens when my PHP code simply tries to read and echo the contents of newfile.txt using file_get_contents.

It sounds like you are requesting newfile.txt over HTTP instead of making a direct request on the filesystem. If you use the HTTP protocol then PHP makes an entirely separate HTTP request, using the user_agent value in php.ini - which is NULL by default. (You can set this in your script with a call like ini_set('user_agent','my-user-agent')).

However, if this file is on your local filesystem then you should be making a local filesystem call instead, using an absolute filesystem path. For example:

file_get_contents($_SERVER['DOCUMENT_ROOT'].'/path/to/newfile.txt');

What header is ModSecurity expecting?

Any request from a real end-user should have a non-empty User-Agent header, hence the rule set by ModSecurity. (However, the User-Agent header isn't strictly necessary for a valid HTTP request.)