Timeline for Tactics for dealing with misbehaving robots
Current License: CC BY-SA 3.0
7 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jan 6, 2014 at 19:03 | comment | added | John Conde♦ | @JW01 All you have to do to avoid that is not visit the page that handles this. Since there is no content on it that should be simple to do. | |
| Jan 6, 2014 at 19:02 | comment | added | JW01 | Nice idea. But, If I implemented it, I bet I'd keep accidentally hitting the honeypot myself and keep getting blocked from my own site. | |
| Jan 6, 2013 at 20:24 | comment | added | Free Radical | You may use the described method from casually stopping bad bots from crawling your site. But 1) It is possible to bypass (bad bots - and their masters - may learn how to identify honeypots and know how to avoid them); and 2) This method also may also block legitimiate human users that have been re-allocated IPs that has been blacklisted as belonging to misbehaving bots. If you have a legal or regulatory obligation to not have your site indexed or automatically seached, you must use proper authentication and only give authenticated users access. Everything else is not secure. | |
| Jan 6, 2013 at 19:35 | vote | accept | Kris | ||
| Sep 2, 2011 at 13:06 | comment | added | John Conde♦ | I would stick to IP addresses then. Plus if you use I addresses and see a pattern from a block of IPs you can then easily block all of them with one simple rule instead of maintaining a long list of individual IPs. | |
| Sep 2, 2011 at 12:43 | comment | added | Kris | That is a very interesting idea, although in case they are using a very generic user-agent, you wouldn't want to blanket lockout any user-agent automatically. | |
| Sep 2, 2011 at 11:56 | history | answered | John Conde♦ | CC BY-SA 3.0 |