We’re rewarding the question askers & reputations are being recalculated! Read more.
2 added 168 characters in body
source | link

Update 2017

Archive bot now does not care about your robots.txt.

If you really want to block it, send them a email according to this page, or block their IP address via htaccess.

Update 2017

Archive bot now does not care about your robots.txt.

If you really want to block it, send them a email according to this page, or block their IP address via htaccess.

Update 2017

Archive bot now does not care about your robots.txt.

If you really want to block it, send them a email according to this page, or block their IP address via htaccess.

1
source | link

Update 2017

Archive bot now does not care about your robots.txt.

If you really want to block it, send them a email according to this page, or block their IP address via htaccess.