I want stop my beta.example.com being indexed by search engines, because it can generate duplication issue.

  • tillinberlin is right about stopping. To remove them from the index you may also use noindex meta robots or canonical. – Marian Popovych Apr 28 '15 at 9:00

After trying different methods I came to the conclusion that you can never be really sure if a search engine respects robots.txt or no-index or anything the like.

So if you really want to be sure your test site is not indexed, I's recommend using password protection through a .htaccess / .htpasswd file combi similar to the following code snippet:

AuthType Basic
AuthName "Password Protected Area"
AuthUserFile /path/to/.htpasswd
Require valid-user

The .htpasswd file should then contain something like this:


(username followed by md5 encrypted password)

| improve this answer | |

Not the answer you're looking for? Browse other questions tagged or ask your own question.