Tagged Questions

4

votes

4answers

401 views

How to protect SHTML pages from crawlers/spiders/scrapers?

I have A LOT of SHTML pages I want to protect from crawlers, spiders & scrapers. I understand the limitations of SSIs. An implementation of the following can be suggested in conjunction with any ...

security scraper-sites

asked May 13 '11 at 19:40

Adam Lynch
23118

newest scraper-sites security questions feed

Technology			Life / Arts	Culture / Recreation	Science	Other
Stack Overflow Server Fault Super User Web Applications Ask Ubuntu Webmasters Game Development TeX - LaTeX	Programmers Unix & Linux Ask Different (Apple) WordPress Answers Geographic Information Systems Electrical Engineering Android Enthusiasts IT Security	Database Administrators Drupal Answers SharePoint User Experience Mathematica more (14)	Photography Science Fiction & Fantasy Seasoned Advice (cooking) Home Improvement more (13)	English Language & Usage Skeptics Mi Yodeya (Judaism) Travel Arqade (gaming) Bicycles Role-playing Games more (22)	Mathematics Cross Validated (stats) Theoretical Computer Science Physics more (7)	Stack Apps Meta Stack Overflow Area 51 Stack Overflow Careers

rev 2013.5.17.690

Tagged Questions

How to protect SHTML pages from crawlers/spiders/scrapers?

Community Bulletin