After reading up on this Google Webmasters article on creating a robots.txt file, I noticed that they mentioned that the
robots.txt file should always be at the root directory (
/). However, I've been getting accustomed to putting all of my resource files (CSS, JS, images, PDFs, etc) in a static files sub-directory (
/static/...) and perma-redirecting (
301) associated links to them (in the case of meta-tags, etc).
Does my solution of redirecting
/static/file/robots.txtfunction as expected?
Even if my solution does work, should I still bite the bullet and do the proper thing and just serve the file as the spec tells me to do?