Hot answers tagged

44

.dev domains are HTTPS only. It isn't a redirect. It is HSTS preload. HSTS is a technology that lets domains declare that they are HTTPS only. It is meant to mitigate against protocol downgrade attacks. The first time you visit a site that wants to use HSTS, you get a header that prevents you from ever visiting that domain on HTTP. The HSTS preload ...


11

See http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.9.3: The max-age directive on a response implies that the response is cacheable (i.e., "public") unless some other, more restrictive cache directive is also present. It's conceivable (likely?) that there are proxies in the wild which break this but since the only failure mode could be ...


10

You need to drop the 'http:' prefix from all image (and any other) links on your site. When you do so, your browser will default to whatever protocol the page loads with. For example this is best: <a href="//www.example.com/"> And this is not: <a href="http://www.example.com/"> The top example is perfectly valid markup and you will load all ...


8

In your robots.txt add #Baiduspider User-agent: Baiduspider Disallow: / #Yandex User-agent: Yandex Disallow: /


8

Great question! This can actually be done, but it's relatively tricky. Tumblr has made it very clear that they don't want you to do this. They have silently broken CloudFlare integration a while back, where everyone who did this reverse proxy stuff with CloudFlare has basically gotten booted off of their custom domain name without any warning. (Keep in ...


7

Try to make one setting default and other one turned on by condition: expires max; valid_referers none blocked something.com; if ($invalid_referer) { rewrite ^ /leech.jpg; expires epoch; }


7

I have just successfully blocked the Chinese searchbot Baiduspider from accessing any content on my site. I made the decision to do so because of the following reasons. Reasons for deciding to block Approximately every 20th request to my server was from a baidu bot. This is unpolite behavior. Baidubot accounts for 5% of my sites bandwidth usage. I make a ...


6

Turns out, the WordPress install was indeed compromised. I did a diff of my existing install vs a new WordPress install, and diff reported new file: # diff -qr wordpress_installed/ wordpress_new/ Only in wordpress_installed/wp-includes: class-wp-init.php I also found wp-config.php had been edited to include this @include_once(ABSPATH . 'wp-includes/class-...


6

Although the question is 2 years old, I would like to keep on answering to it. The page linked by the accepted answer (https://www.nginx.com/resources/wiki/modules/auth_digest/) is 11 years old and states itself that "... (it) is in need of broader testing before it can be considered secure enough for use in production." A GitHub page (https://github.com/...


6

rewrite ^/$ ? permanent; This would only match the document root (a valid request) - this isn't a URL that ends with a slash, but a URL that starts with a slash. Try something like the following instead, to match any request URL that ends with a slash: rewrite ^(.+)/$ $1 redirect; $1 is a backreference to the original URL-path, less the trailing slash. ...


5

You can use the following directive in robots.txt to disallow the crawling of your site. # robots.txt User-agent: Baiduspider Disallow: / However, crawlers may decide to ignore the content of your robots.txt. Moreover, the file can be cached by search engines and it takes time before changes are reflected. The most effective approach is to use your server ...


5

The 301 redirect that you are doing is the correct way to recover. The canonical tag that you are considering and 301 redirects can't be used at the same time. The canonical tag is only for pages that can't be redirected. Recovery should be in a week to a month. If it is longer than that, it is likely that something else has happened.


4

Your first log snippet suggests that the file may actually exist, since nginx response is 200, with 731 bytes of payload data, so, don't be too fast to have it blocked. However, if you want to have such requests omitted from logs, you can use the following in all servers affected: location = /wpad.dat { access_log off; log_not_found off; } Or, ...


4

You need to set an alias. location /coolapp/ { alias /var/www/apps/coolapp/; } Read more nginx Update after comment: Unfortunately, it's not working for me. I'm getting a 404 response when trying to request www.mysite.com/coolapp/test.php when /var/www/apps/coolapp/test.php exists To understand why alias is not working, you have to take a ...


4

The main reason is: HTTP requests include the domain name. You could have stackoverflow.com and askubuntu.com all served off the same front end machine. When the server gets a request it needs to know what content it has to send. That's the main reason server configuration speaks about host name at all. Why do you have to set it if you're only serving one ...


4

Those are the backend servers that will handle the incoming request. So your domain's DNS should point to this nginx server, which will then use your upstream configuration to pass the request to one of the specified servers. Your config should look something like this (inside http {} block)... upstream backend { server backend1.example.com; server ...


4

Add this block to your NGINX config: server { listen 80; server_name here.your.ip.address; rewrite ^ http://example.com$request_uri? permanent; }


4

To direct the URL domain.tld/image/filename to the local path /imageshare/images/extension/filename, use the alias directive (see this for details): location /image/ { alias /path/to/imageshare/images/extension/; } If you really do mean to rewrite the URL to another URL, use: location /image/ { rewrite ^/image(.*)$ /imageshare/images/extension$1 ...


4

NGINX has a digest authentication module: https://www.nginx.com/resources/wiki/modules/auth_digest/ Unlike basic authentication, digest authentication does not send user names and passwords in plain text over the internet. If your site is SSL only, then basic authentication is probably fine. the SSL encrypts the entire session including the user names and ...


4

Here's the Nginx Conf which works for me, placed at the path /etc/nginx/conf.d/example.com.conf. server { listen 80; listen [::]:80; server_name example.com; proxy_set_header Host $http_host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; location / { proxy_pass "...


4

If the underlying filename is script.js.php then it doesn't make sense to rewrite this to script.js - as that would result in a 404. However, the directives you posted would seem to result in a 404 for a different reason, as they rewrite script.js.php to script.js.php.js. Presumably, you want to link to script.js?c=21 and internally rewrite this to script....


3

Just decided to block Baidu as the amount of traffic it was giving us was too negligible for their aggressive scanning. In addition, they now run an agent that impersonates a browser and launches JavaScript code (such as Google Analytics) and messed up our statistics. The nice version is updating your robots.txt with the following User-agent: Baiduspider ...


3

Yes, you can run vBulletin with nginx. The rewrite rules you need will be specific to your setup and any modules you are using (you do not need any rewrite rules to run a standard vBulletin installation unless you use SEO Friendly URLs - see this vBulletin forum post). Many popular mods have already had their rules translated, (e.g. vbSEO) however, you can ...


3

No, php-fpm user does NOT need execute permissions on PHP files. It only needs read permission, as PHP files are parsed by PHP preprocessor and not executed directly.


3

I think that you want to filter traffic by user agent and do something like this, in order to prevent access on some specific user agents: if ($http_user_agent ~* ^EirGrabber|PHPCrawl|PHP/5|^EmailSiphon|^$) { return 410; } Solution Create a separate file that will contain the above code or any other filtering common code, and include this file on each ...


3

User agent wget Blocking the user agent wget will block operating systems that use wget to download files within the terminal i.e Linux, MacOS, Arm etc. This will block every file from being downloaded using wget example.pdf. But why block wget? There's no right or wrong to this question but generally webmasters want site visitors actually visiting their ...


3

I will say right from the start I do not know Amazon Linux EC2, but I did do a bit of research to try and help. You should be able to stop nginx with: nginx -s stop You should be able to stop it from starting on reboot with: chkconfig nginx off It may be that you need to use sudo to issue both of these commands: sudo nginx -s stop sudo chkconfig nginx ...


3

There is no difference. There are different ways to set the same PHP configuration parameter. It's useful - for example - if you have a generic php.ini which gets loaded in all PHP-FPM pools, but in one of them you want to override a certain value - like memory_limit See http://php.net/manual/en/configuration.changes.php


3

The HTTP status code 301 is named "Moved Permanently": The requested resource has been assigned a new permanent URI and any future references to this resource SHOULD use one of the returned URIs. So the resource (i.e., your document) would stay the same, it just gets a new URI. As your front page http://example.com/ is (usually) not the same resource as ...


3

The space is a delimiter (ie. a special character) in .htaccess so must be backslash escaped if you want to match a literal space in the regex. Eg. DV\ CRAWLER. (Otherwise you are likely to get a less than helpful 500 Internal Server error.) Or, you can use the shorthand character class \s which matches any white space character (space, tab or new line / ...


Only top voted, non community-wiki answers of a minimum length are eligible