Tagged Questions
0
votes
0answers
65 views
Facebook crawling [closed]
Does a normal person is allowed to crawl all the publically visible links of facebook for research purpose with the help of automated bots?
And whether there is any API provided by ...
1
vote
1answer
285 views
Facebook and Crawl-delay in Robots.txt?
Do Facebook's webcrawling bots (for example facebookexternalhit/*) respect the Crawl-delay: directive in robots.txt files?
