Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

Let me explain my situation, I have an azure website (with azurewebsites.net sub domain), and a custom domain.com, built with asp.net MVC

Both are being indexed by Google, but I've noticed the custom domain is being penalized and it doesn't show up in results, it only shows when I search for "site:domain.com" I want to remove and block the azurewebsites.net subdomain from Google. I've read the "possible" solutions:

  • Adding robots.txt: won't work, because the subdomain and the domain are the exact same content, so subdomain.azures.net/robots.txt will lead to domain.com/robots.txt, removing the domain as well.
  • Adding the tag, is the same situation as the previous point.
  • I'm using a CNAME register to redirect the domain to the subdomain, so I can't redirect to a sub directory.

Do you have any other ideas?

share|improve this question
add comment

1 Answer

If you don't want to separate robots.txt or you don't want to add metatags noindex then your options are very limited; one could be to password protect your azure site, but that will lead to restricted visitors access.

Some about this here.

On the other hand, is your robots.txt too complex? does it have tons of rules? why do you give it too much importance? on regular basis it is just something like this:

User-agent: *
Disallow:

Just delete it from your local folder, and never upload it via ftp or whatever you are using, create an independent robots.txt file for each site and let each one grow apart. Put this simple robots.txt into your azure site:

User-agent: *
Disallow: /

And put your "complex" robots.txt on the robots allowed site:

User-agent: *
Disallow: /cgi-bin/
Disallow: /tmp/
Disallow: /~joe/
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.