I have a couple of subdomains on my website that I use for various purposes and I don't want/need them to be indexed by search engines. From the very beginning, since I created them, I added a robots.txt
file to disallow search engines from crawling those pages.
Now, I know that this does not enforce anything, it's just a "guideline" for search engines. But in respect to Google, they have lots of help documents suggesting the use of robots.txt, so I believe they respect that file.
Anyway, one of those sub domains is still indexed and I can't understand why. Like I said, this has been created with the disallow rules in robots.txt since the very beginning that sub domain went life and it's been running like this for 2 years or so. Still, I just noticed this:
How can I permanently get rid of this indexed result on Google? For the record, Bing and Yahoo! don't have a result for the same query...