2

There are two subdomains for my website. The subdomain for the production site is www.example.com. The subdomain for the test site is test.example.com.

test.example.com is showing up in Google search results and needs to be removed. I have added a robot.txt file and set up basic authentication for test.example.com to keep Google from crawling the site in the future.

Is it possible to remove URLS that contain test.example.com without removing www.example.com? Google search console states "All URL variations (www/non-www and http/https) will be affected". Does this mean if I remove https://test.example.com, https://test.example.com/page1.html, https://test.example.com/page2.html then https://www.example.com, https://www.example.com/page1.html, https://www.example.com/page2.html will also be removed?

UPDATE: I decided not to add basic authentication. I'm redirecting any test pages in the Google search to the production site. robot.txt contains the following:

User-agent: * Disallow: / 

It is my understanding that this can take weeks to remove the sight from Google search. Is there a way to make this happen sooner without affecting SEO on the production site?

4
  • Is the robots.txt behind the basic authentication? If so, it isn't doing anything (which is probably a good thing.) If not, it is preventing Googlebot from crawling the URLs on the test subdomain and seeing that they require authentication now. May 17, 2023 at 18:51
  • I decided not top add basic authentication. I'm redirecting any test pages in the google search to the production site. Robot.txt contains the following: I decided not top add basic authentication. I'm redirecting any test pages in the google search to the production site. Robot.txt contains the following: User-agent: * Disallow: / It is my understanding that this can take weeks to remove the sight from google search. Is there a way to make this happen sooner without effecting SEO on the production site?
    – Anthony
    May 18, 2023 at 14:15
  • 2
    If you are redirecting, users who click from search will get to the right place, so I wouldn't worry too much about it. Again, your robots.txt file is preventing Googlebot from seeing your redirects, so it should probably be removed. May 18, 2023 at 15:10
  • 2
    For your comment question "Is there a way to make this happen sooner". I'm new to using GSC, but have you tried the "Removals" feature (in the "Indexing" group)? This is what I did recently for subdomains. I'm still wondering how the subdomains got indexed in the first place.
    – Tim R
    May 19, 2023 at 4:18

1 Answer 1

0

Suggestions

  • Use <meta name='robots' content='noindex'> on all pages of your test site
  • Remove any links from your production site to the test site
  • Create a login page, for example test.example.com/login and dont have a link to it anywhere
  • On your login page, add a token (or more) in the SESSION variable
  • On all other pages, if the token is not there, redirect to the main site
  • Remove your pages at Remove Outdated Content

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.