2

I have a web based proxy and web filters eventually blacklist the domain. Because of this, I am thinking of creating alternate domains for the same website. Each domain would host 100% same content. I will prevent alternate domains from being indexed by Google to prevent duplicate content.

Is this approach acceptable to Google?

Can I place Google ads on the alternate domains as well?

Edit :

The main domain would function as normally and is indexed and crawl-able by Google.Only the alternate domain would not be indexed.Alternate domain will be used by users who can't access the main domain (due to being blocked by web filters )

Improve this question
5
  • He said he would block them from being indexed in Google, so it should't be an issue, least in Google.
    – Max
    May 26, 2015 at 7:42
  • It might help to clarify how you plan to do this: I will prevent alternate domains from being indexed by google
    – dan ♦
    May 26, 2015 at 7:43
  • @dan robots.txt or nginx
    – user2650277
    May 26, 2015 at 7:52
  • If you're running a proxy, the proxied content shouldn't be crawlable or indexable at all. How you shuffle your homepage around (assuming you use a rel=canonical to a single indexable version) is up to you.
    – John Mueller
    May 28, 2015 at 11:13
  • @JohnMueller The proxy homepage is crawlable while the proxified content is not.
    – user2650277
    May 29, 2015 at 16:36

2 Answers 2

Reset to default
2

If you prevent robots from indexing content, these websites are not considered as duplicate versions, and Google will ignore them.

In fact, noindex is usually applied to "print version" of web articles. That's the most similar scenario. In other scenarios, I would suggest to use redirection or canonicalization.

For the ads: it depends. You need to read the ToS and see if some sort of limitation is present.

Improve this answer
7
  • Can i place ads on the alternate domains
    – user2650277
    May 26, 2015 at 8:38
  • What's the point to place ads on alternate domains if they are not public? Just keep them "dormient", and awake one of them when you need.
    – lucgenti
    May 26, 2015 at 8:42
  • Users who cannot access main domain because its blocked , will use alternate domain and proxy hosting cost a lot of money.Others will keep using using the main domain as usual
    – user2650277
    May 26, 2015 at 8:45
  • @lucgenti The OP didn't say they weren't public; just not indexed.
    – MrWhite
    May 26, 2015 at 8:47
  • Ok now I understand. I edit the answer, because in that case you also need redirections. About the ads: you need to read the ToS of the service provider.
    – lucgenti
    May 26, 2015 at 8:50
1

You don't want to block duplicate content with robots.txt. Google's guidelines say:

Google does not recommend blocking crawler access to duplicate content on your website, whether with a robots.txt file or other methods. If search engines can't crawl pages with duplicate content, they can't automatically detect that these URLs point to the same content and will therefore effectively have to treat them as separate, unique pages. A better solution is to allow search engines to crawl these URLs, but mark them as duplicates by using the rel="canonical" link element, the URL parameter handling tool, or 301 redirects.

The rel canonical link tag is what you want to use. It is for cases where users need to be able to use various URLs but where search engines need to know which one in the correct one to index. Every page on your site should get a tag in the head like:

<link rel="canonical" href="http://mainsite.example.com/this-page.html" />

As for Ads, the AdSense TOS says:

Publishers are not allowed to create multiple pages, sub-domains, or domains with substantially duplicate content.

The "duplicate content" is linked to the document that says that you can use rel canonical links to solve the problem. So implement the rel canonical links and you should be fine for both search and AdSense.

Improve this answer
7
  • you are sure that this won't break adsense TOS since the same content will be available on many domain:)
    – user2650277
    May 27, 2015 at 9:59
  • I looked up the AdSense TOS and updated my answer based on it. Thank you for pointing that out.
    – Stephen Ostermiller ♦
    May 27, 2015 at 10:14
  • Thanks for explaining it so nicely:)....I will leave the question open for 24 hours just in case i get an official response from adsense team
    – user2650277
    May 27, 2015 at 16:46
  • do i need to use noindex as well or just canonical will do.I don't want alternate domain to be registered
    – user2650277
    May 27, 2015 at 17:51
  • Just canonical will do. Only the canonical domain will be indexed if all the pages have the tag pointing to it.
    – Stephen Ostermiller ♦
    May 27, 2015 at 17:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.