Possible Duplicate:
Do I need robots.txt?

From my understanding robots.txt is only needed to do something different from the norm like exclude a certain file directory and such, but if this is true why do so many people think it is important on every website? Should I just put it on every website to be sure?

| improve this question | | | | |

If you don't have anything that you wan't to tell the searchengines you don't wan't indexed (or crawled) don't. It doesn't impact your indexing.

See this question answered by Google.

| improve this answer | | | | |

Not the answer you're looking for? Browse other questions tagged or ask your own question.