2

I have a web based proxy and web filters eventually blacklist the domain. Because of this, I am thinking of creating alternate domains for the same website. Each domain would host 100% same content. I will prevent alternate domains from being indexed by Google to prevent duplicate content.

Is this approach acceptable to Google?

Can I place Google ads on the alternate domains as well?

Edit :

The main domain would function as normally and is indexed and crawl-able by Google.Only the alternate domain would not be indexed.Alternate domain will be used by users who can't access the main domain (due to being blocked by web filters )

|improve this question
  • He said he would block them from being indexed in Google, so it should't be an issue, least in Google. – Max May 26 '15 at 7:42
  • It might help to clarify how you plan to do this: I will prevent alternate domains from being indexed by google – dan♦ May 26 '15 at 7:43
  • @dan robots.txt or nginx – user2650277 May 26 '15 at 7:52
  • If you're running a proxy, the proxied content shouldn't be crawlable or indexable at all. How you shuffle your homepage around (assuming you use a rel=canonical to a single indexable version) is up to you. – John Mueller May 28 '15 at 11:13
  • @JohnMueller The proxy homepage is crawlable while the proxified content is not. – user2650277 May 29 '15 at 16:36
1

You don't want to block duplicate content with robots.txt. Google's guidelines say:

Google does not recommend blocking crawler access to duplicate content on your website, whether with a robots.txt file or other methods. If search engines can't crawl pages with duplicate content, they can't automatically detect that these URLs point to the same content and will therefore effectively have to treat them as separate, unique pages. A better solution is to allow search engines to crawl these URLs, but mark them as duplicates by using the rel="canonical" link element, the URL parameter handling tool, or 301 redirects.

The rel canonical link tag is what you want to use. It is for cases where users need to be able to use various URLs but where search engines need to know which one in the correct one to index. Every page on your site should get a tag in the head like:

<link rel="canonical" href="http://mainsite.example.com/this-page.html" />

As for Ads, the AdSense TOS says:

Publishers are not allowed to create multiple pages, sub-domains, or domains with substantially duplicate content.

The "duplicate content" is linked to the document that says that you can use rel canonical links to solve the problem. So implement the rel canonical links and you should be fine for both search and AdSense.

|improve this answer
  • you are sure that this won't break adsense TOS since the same content will be available on many domain:) – user2650277 May 27 '15 at 9:59
  • I looked up the AdSense TOS and updated my answer based on it. Thank you for pointing that out. – Stephen Ostermiller♦ May 27 '15 at 10:14
  • Thanks for explaining it so nicely:)....I will leave the question open for 24 hours just in case i get an official response from adsense team – user2650277 May 27 '15 at 16:46
  • do i need to use noindex as well or just canonical will do.I don't want alternate domain to be registered – user2650277 May 27 '15 at 17:51
  • Just canonical will do. Only the canonical domain will be indexed if all the pages have the tag pointing to it. – Stephen Ostermiller♦ May 27 '15 at 17:54
2

If you prevent robots from indexing content, these websites are not considered as duplicate versions, and Google will ignore them.

In fact, noindex is usually applied to "print version" of web articles. That's the most similar scenario. In other scenarios, I would suggest to use redirection or canonicalization.

For the ads: it depends. You need to read the ToS and see if some sort of limitation is present.

|improve this answer
  • Can i place ads on the alternate domains – user2650277 May 26 '15 at 8:38
  • What's the point to place ads on alternate domains if they are not public? Just keep them "dormient", and awake one of them when you need. – lucgenti May 26 '15 at 8:42
  • Users who cannot access main domain because its blocked , will use alternate domain and proxy hosting cost a lot of money.Others will keep using using the main domain as usual – user2650277 May 26 '15 at 8:45
  • @lucgenti The OP didn't say they weren't public; just not indexed. – MrWhite May 26 '15 at 8:47
  • Ok now I understand. I edit the answer, because in that case you also need redirections. About the ads: you need to read the ToS of the service provider. – lucgenti May 26 '15 at 8:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.