I've just looked up my website in Google, to see if the sitemap had changed to reflect some new content and pages, and I have a bit of a problem. I use a third-party hosted toolset, which is linked via a CNAME record on my host server.

So lets say...

My Site = www.mysite.co.uk
Toolkit = www.clients.mysite.co.uk
The CNAME on my host says: clients.mysite.co.uk > thirdparty.toolset.com 

But when I look at the Google sitemap, I see several of the clients.mysite.co.uk pages listed as well - many of which I don't actually use, and so don't want the public to see.

I can't make any changes to the third party site, so can't put a robots.txt file in the root of clients.mysite.co.uk (as it's actually thirdparty.toolset.com), or make any changes to the .htaccess there either. Of course I can make changes to anything in the normal www.mysite.co.uk domain...

Any idea what I should do to stop the indexing happening in the subdomain, and also preferably to quickly remove what has already appeared in Google?

  • 1
    "the Google sitemap" - What do you mean by "sitemap" in this context? It sounds like you are using "sitemap" to refer to the search results (SERPs)? – MrWhite Mar 7 '16 at 0:25
  • When I say "Sitemap", I mean when you see the Google results page, you get the main site link at the top, followed by 2 columns of page results from the site. – WeddingDJ Mar 8 '16 at 7:57
  • Those are called "Sitelinks" (this is very different to a "sitemap"). You can "demote" sitelinks for specific searches - if that will help. However, this doesn't influence the indexing of those pages, just how the sitelinks are returned. – MrWhite Mar 8 '16 at 8:37
  • That sounds like something I need to do - can you explain a little more please w3dk? – WeddingDJ Mar 8 '16 at 11:03
  • More information is given in the page I linked to. – MrWhite Mar 8 '16 at 11:27

Without any access to the sites, you aren't going to be able to permanently block crawling or indexing. Even Google's help help articles on this only really provides options that would require some form of access, if not to the code then to the server in some way. For example, see: https://support.google.com/webmasters/answer/6062602?hl=en&authuser=4&ref_topic=4598466

However, one option that might exist is to temporarily remove the URLs provided you can verify the site and gain access to Google Search Console. The REmove URLs tool gives you the opportunity to remove URLs for ~90 days. More about that here: https://support.google.com/webmasters/answer/1663419/?hl=en&authuser=4

The one problem with that route might be verifying ownership. However, you should be able to verify via DNS: https://support.google.com/webmasters/answer/35179/?hl=en&authuser=4

Not the answer you're looking for? Browse other questions tagged or ask your own question.