0

I have a website with several thousand pages and I generate a small version of it (100 pages) which I upload to a temporary testing.mywebsite.com URL for testing purposes.

Yet, it could be seen as duplicate content considering the live website on a different URL. Should I be worried about this? Can I get a penalty for this?

I was considering setting canonical links to the main website or setting no index on the pages generated for testing. Is it necessary to do so? Would it help?

  • Why would you even want this to be crawled? – John Conde Sep 7 '13 at 17:47
  • I don't want it to be crawled, but one way or the other, Google could learn about it. – Jérôme Verstrynge Sep 7 '13 at 17:50
  • 1
    Block it every way possible. Basic authentication, robots.txt, x-robots-tag, etc. Make sure they never see it. – John Conde Sep 7 '13 at 17:55
  • Google could learn about it Learn from what exactly? – Simon Hayter Sep 7 '13 at 20:31
  • I use adsense, so adsense may notice the ad call from my testing website, and inform Google robots may be... – Jérôme Verstrynge Sep 8 '13 at 12:44
2

Search engines might …

  1. … find your testing website.
  2. … index pages from your testing website.
  3. … decide that a page from the real site and the testing site are duplicate copies, so they might only index one of those pages.

If you are unlucky, they might decide to index the testing page instead of the real page in some cases (especially, if not all of your real pages are indexed yet).

I was considering setting canonical links to the main website or setting no index on the pages generated for testing. Is it necessary to do so? Would it help?

Yes, it would help (for those search engines that support canonical resp. robots-noindex).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.