up vote 1 down vote favorite
share [g+] share [fb]

Do Facebook's webcrawling bots (for example facebookexternalhit/*) respect the Crawl-delay: directive in robots.txt files?

link|improve this question
92% accept rate
3  
Don't dupe questions across sites. You got your answer over there, by the way. (See Jeff's bio.) – Su' Oct 19 '11 at 5:00
Boo... thanks @Su' - now how do I withdraw my bounty??? – Mike Hudson Oct 19 '11 at 6:59
feedback

Know someone who can answer? Share a link to this question via email, Google+, Twitter, or Facebook.

Your Answer

 
or
required, but never shown
discard

Browse other questions tagged or ask your own question.