Tagged Questions

2
votes
1answer
34 views

Which tools can help limit maximum page view per ip to limit scrapers and bots?

I would like to prevent scrapers grabbing all my content except Google, Bing and other search engines. I Am thinking of going with Fail2ban and limiting hits from an ip maybe at around 1000 per day. ...
0
votes
0answers
17 views

Is this Anti-Scraping technique viable with Crawl-Delay? [duplicate]

Possible Duplicate: How do spambots work? I want to prevent web scrapers from abusing 1,000,000 on my website. I'd like to do this by returning a "503 Service Unavailable" error code for ...