I want to allow all bots to crawl everything, do I need to create a robots.txt?
If the file is missing will it have a negative influence on some crawlers?
If I need to create it, what should it look like?
Webmasters Stack Exchange is a question and answer site for pro webmasters. It only takes a minute to sign up.Sign up to join this community