Webmasters Stack Exchange is a question and answer site for pro webmasters. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Possible Duplicate:
Do I need robots.txt?

From my understanding robots.txt is only needed to do something different from the norm like exclude a certain file directory and such, but if this is true why do so many people think it is important on every website? Should I just put it on every website to be sure?

share|improve this question

marked as duplicate by John Conde Feb 20 '12 at 20:37

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

up vote 2 down vote accepted

If you don't have anything that you wan't to tell the searchengines you don't wan't indexed (or crawled) don't. It doesn't impact your indexing.

See this question answered by Google.

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.