We host several sites on a Cloud server. It constantly seems to be getting picked up by Google, but our clients need to be able to access there in progress builds to add content. This brings with it several problems because we can't assign ips to block as most clients would be on a dynamic ip.

Is there any recommendations for any sort of portal system that would allow us to have quite alot of sites based off the portal domain and accessed seperately via a username and password. Even if with the one password you could still view someone elses site?

Our setup is currently used for Magento / Wordpress sites and as such obviously needs to be php5.

  • Can you just use HTTP Authentication (password protected folder/site -- everything is done by web server, no custom coding is required). Should be enough to protect from any kind of search bots (no valid user name & password = no content). There are plenty of examples for Apache. If you already have some sort of Control Panel to manage your sites (e.g. cPanel), it should have such functionality built in.
    – LazyOne
    Sep 2 '11 at 15:21

If you only problem is that the test sites is being index by Google bot, then you can use the X-Robots-Tag to tell Google bot to stop indexing.

With PHP:

header("X-Robots-Tag: noindex, nofollow", true);

Or in the .htaccess or httpd.conf:

<IfModule mod_headers.c>
Header set X-Robots-Tag: "noindex"

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.