Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

I was making a site for a friend of mine and he wanted to be able to see my progress as I worked on the site, so I decided to put the site on a server on my computer and enable access by a domain name registered to me.

It turns out that I forgot to set up a robots.txt file for the site and somehow Google indexed the site.

My question is: What do I do now?

As I understand it, Google doesn't like duplicate content and my friend could have problems when I upload the new site to his server.

Right now his current site, which only has a work in progress page, is first on Google when searching for relevant keywords and I really really don't want to damage that.

Is there anything else I need to be concerned about?

share|improve this question
2  
+1 for getting the site on top for relevant keywords without any efforts! :) lol i wish that would happen with me sometime. :( –  Jasdeep Singh Mar 8 '11 at 6:53

4 Answers 4

up vote 13 down vote accepted

Add the robots.txt file to your server then request the pages to be removed from Google's index using Google Webmaster Tools. You should then set up 301 redirects to the new site when it goes live.

share|improve this answer
    
Do you have any idea how long it takes for removal request to be processes using Google Webmaster Tools? I placed request several hours ago and it is still waiting. Is that normal? –  AndrejaKo Mar 8 '11 at 14:17
    
When I used it before on a hacked site it took a couple of days for all the links to be removed. It will be more than a few hours anyway. –  Ewan Heming Mar 9 '11 at 2:23

1) Immediately block the site you have up now using robots.txt

2) On the development site use canonical URLs to point to the new site so that site gets credit for the content. They do work cross-domain. This meta tag may also help.

3) after you launch use 301 redirects from the development site to the new site so Google knows those pages have moved to new URLs

share|improve this answer
    
Good to know about the canonical tag working across domains as I thought it was just within the same domain - Is that a change from the past? –  Ewan Heming Mar 9 '11 at 2:22

In the future, you should set up Basic Authentication in Apache or IIS for test pages. This will not only prevent indexing (even from spiders that don't respect robots.txt) but will also prevent random users from getting to your site and seeing what you don't want to be public yet.

share|improve this answer
    
Probably something we all should do more often with development sites... –  Ewan Heming Mar 9 '11 at 2:28

Remove a page or site from Google's search results - http://www.google.com/support/webmasters/bin/answer.py?answer=164734

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.