7

The robots.txt standard controls whether the bot can view foo.html under any circumstances. Just because the bot sees a link to it from a different site doesn't give the bot permission to sneak a peek at it! Suppose I have a URL /foo.html for which crawling is blocked using robots.txt. Then the bot will not crawl the page, but it might still be indexed if ...


3

Apache Tomcat has gzip filter and it starts to zip from 2kb, my quick test tells that it's the lowest boundary, and you can increase it at least to 3-4kb. Because for 2kb you will get a similar size at the output + time penalty for zipping.


1

Content Security Policy (CSP) Quick Reference Guide CSP-Reporting on Google Developers If you're just starting out with CSP, it makes sense to evaluate the current state of your application before rolling out a draconian policy to your users. As a stepping stone to a complete deployment, you can ask the browser to monitor a policy, reporting violations but ...


Only top voted, non community-wiki answers of a minimum length are eligible