I don't like that I see a lot of 404 errors in the access.log of my web server. I'm getting those errors because crawlers try to open a robots.txt file, but couldn't find any. So I want to place a simple robots.txt file that will prevent the 404 errors from appearing in my log file.
What is a minimum valid robots.txt file that will allow everything on the site to be crawled?