2

I have two domains: example.com and dev.example.net. example.com is the active one and dev.example.net should be invisible to Google.

dev.example.net has 301 redirects to example.com and it has its own robots.txt that doesn't allow Google to index pages.

Still I got dev.example.net results showing in Google. How can I convince Google that it shouldn't show any results from dev.example.net?

  • 2
    Be careful. Blocking access to Google in your robots.txt file will ensure that any 301 redirect is not seen. As well, it can take weeks or months before Google has fetched your pages to know that they are redirected. This is done one page at a time according to Google's schedule and not your expectations. – closetnoc Jun 25 '17 at 18:37
  • 301 is there just as patch because google still shows dev.otherdomain.com results. As soon as I get dev.otherdomain.com out of google that domain will cease to exist. – sarte Jun 25 '17 at 19:35
  • Ceasing to exist by removing it from DNS? That will get it out of the search results faster than using robots.txt. – Stephen Ostermiller Jun 26 '17 at 6:46
  • How long you got dev.example.com crawled for? When you said you got more ranking on dev.example.com domain, can you explain a little bit more about it? It can be managed in different ways, let me know these and will let you know the solution. – TopQnA Jun 26 '17 at 12:49
  • 1
    Is the content identical? If yes, do you redirect all pages to their counterparts, or only to the homepage? – unor Jun 27 '17 at 8:00
1

First, you should check that you've properly set up 301 redirects from your old domain to corresponding pages on the new one, as you say you've done. You can use your browser's developer tools (e.g. like this on Chrome or like this on Firefox) to check that your redirects are working properly, and are really using the status code 301 and not 302.

You also need to allow Google to access all pages in the robots.txt file on your old domain. Otherwise Google cannot tell that the pages have been redirected, and will continue to show them in search results.

Once you've done both of these things, the problem should eventually resolve itself over time, as Google recrawls your old domain and finds the redirects. However, to speed up the process, you can use the change of address tool on Google's Search Console. (Of course, you'll also need to verify your ownership of both domains with Google first.) If you want, you can also do the same thing with other search engines such as Bing.


The process described above should be sufficient to get your new domain indexed and your old domain dropped from Google's index promptly, and it will preserve any "link juice" your old domain may have acquired.

If you have some pages on your old domain that you can't (or don't want to) meaningfully redirect to the new domain, e.g. because no corresponding page exists there any more, you should configure your webserver to return a 410 "Gone" status code for them. You can also use Google's URL removal tool to speed up the process, just like with redirects and the change of address tool.

Finally, this should almost go without saying, but you should also make sure that none of the pages on the new domain still link to URLs on the old domain, or (worse!) have the old domain still included in their canonical URL.

2

I would suggest that you follow Google webmaster guidelines for permanent URL removal.

To remove content or a URL from Google search permanently you must take one or more of the following additional actions:

  • Remove or update the actual content from your site (images, pages, directories) and make sure that your web server returns either a 404 (Not Found) or 410 (Gone) HTTP status code. Non-HTML files (like PDFs) should be completely removed from your server.
  • Block access to the content, for example by requiring a password.
  • Indicate that the page should not to be indexed using the noindex meta tag. This is less secure than the other methods.

You can read more about it here: https://support.google.com/webmasters/answer/1663419?hl=en

  • 1
    You need to note that for your password or noindex solutions to work, they need to allow Googlebot to crawl the domain in robots.txt. – Stephen Ostermiller Jun 26 '17 at 6:41
  • So there is no way to tell google that some domain is off limits for google? That's disappointing. The basic problem is that dev.otherdomain has higher google ranking than the actual page. So giving the 404 or password protection is kind of bad solution for my point of view – sarte Jun 26 '17 at 6:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.