I have recently created a subdomain for testing my changes on a live server. Today I have found that Google has indexed nearly a third of this test domain. How did Google find this test subdomain? There are no links to it, there's not wordpress blog (so no pinging of Google when a blog post is made), and it's not been registered in Webmaster Tools.

The only explanation I can think of is Google's free DNS has captured the subdomain address when it propagated around the 'net and is using that to index new websites.

I've added order deny,allow to my .htaccess file to block Google (and any visitors who come from the search results), but my question is: Is Google using the DNS records to find new websites? If not, how did Google find my website?

  • Yes they are using DNS information. – Jérôme Verstrynge Oct 7 '14 at 16:40
  • "Google's free DNS has captured the subdomain address when it propagated around the 'net " the DNS does not work exactly like this, it is not top down. Anyway, technically, they can look at all DNS queries they get and infer new domains from that. THey could also, like anyone, download the list of all domain names in gTLD, daily, and use it. THere are a myriad other ways to find domains. Do you use Chrome? Any antivirus or plugin to protect you against phishing or scams ? Or Skype? All of these tools, and many more, are known to "phone home" and collect and forward DNS names. – Patrick Mevzek Nov 1 '19 at 5:59

Google finds websites lots of different ways. They definitely don't wait for a webmaster to submit the site to them.

They could be, and probably are, using DNS. Also, if anyone with something that reports back to Google (ie analytics if it is installed, possibly Chrome, Google toolbar, etc) visits a URL on that subdomain Google will know about it.

|improve this answer|||||

I dont think so, Google says

Google DNS is not used in web ranking, data is not shared with 'other parts of google'


|improve this answer|||||
  • I'm not asking if Google uses the DNS as a ranking factor, just as a method of locating websites that do not have external links pointing to it. – Richard Parnaby-King Oct 8 '14 at 9:20
  • I understand you didnt ask whether Google uses its Public DNS in ranking, the post I made contained a quote to give context, in that : Google doesnt use its dns in web ranking.. and/as data is not shared with other parts of google. (ie - not for ranking, not for indexing and not for discovery) – user29671 Oct 8 '14 at 12:54

They are registered to sell domains. I'm sure they use a wide variety of crawling methods including keeping track of currently registered domainscalong with DNS data.

|improve this answer|||||

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.