1

A few years ago I had couple of sites. Then I closed them. Back then, the search engines had added those sites of mine into their databases. Specifically Google and Chinese Baidu know about them. Now their bots query my DNS server for non-existent sites.

The complication is that now I've got a working website. Let's call it example.com. Those non-existent sites are subdomains like forum.example.com.

The traffic is not significant. But why to have it on a daily basis?

As far as I see it, there are two ways to possibly deal with it:

  1. An administrative way, ie., to contact somehow the search engines and ask them nicely to remove those sites from their databases
  2. A technical way, ie., to deal with it myself with some BIND9 configuration.

But I just don't know how to approach the issue, frankly.

Improve this question
1
  • 2
    If the sites in question are sub-domains that are no longer used, just simply delete them from your DNS if you have not already. Your DNS will be queried from time to time, however, your web server will not be. In time, it will all stop. It is nearly impossible to make contact with a search engine that will get you anywhere.
    – closetnoc
    Feb 4, 2016 at 4:42

4 Answers 4

Reset to default
2

You could set up a single page on the subdomain with a message to users that the subdomain is no longer active, and they should access www.example.com instead. Provide a link for users to click through to the main site, and include <meta name="robots" content="noindex"> in the header for search engines to know that it's no longer available.

This option is the most helpful to both users and search engines.

Improve this answer
2

If you have access to those sites, add a hard 404-header. It won't get more obvious than that. The meta's (noindex and nofollow) still indicate that some content exists, you just don;t want it indexed (right now as far as they care).

If those sites dont exists, use a htaccess and some CNAMES to do that same trick. (CNAME forum.site.com as site, and htaccess it to 404).

Improve this answer
1
  • It sounds like the problem isn't that the site is indexed, search engine bots still try to visit it occasionally.
    – Stephen Ostermiller ♦
    Feb 5, 2016 at 15:09
1

Google never forgets a site that it had once indexed. Googlebot will continue to try to visit your old sites for years. There is no way that I know of to get the bots to totally stop trying to visit the old URLs.

If the sites are truly down, Google won't include them in the search index. You won't get real users visiting those sites from search engines. You will only see occasional hits from the search engine bots trying to see if your site is back up yet.

It sounds like you have token the DNS entries for the sub-domains away. The only activity you will see is occasional DNS queries for the subdomains. This is the least amount of activity from search engine bots that you can hope for.

If you want to serve an error page, use the "410 Gone" HTTP status for it. That status makes it explicit that you removed the content. With that status, search engine bots will come back less often than with other statuses such as "404 Not Found." (They will still come check occasionally, though.)

Improve this answer
0

If you still have control on your websites before, just add this on the head

<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">

That will instruct the Search Engine spiders to stop indexing your website and stop following links. Those entries will be flushed from their database. Google on the other hand uses more than 1 database and it may take around 7 days before all copies of the database gets purged.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.