Sign up ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

I have a client that has 3 domain names. For the sake of this argument I will call them, and

He used to have a website package with VistaPrint and was his main business website and website2 and website3 used to redirect to the main website.

However recently I created a new WordPress website for him and now his main domain name is and website1 and website2 simply redirect to the main website.

But when I go on Google and search for the domain name (any of the three) it does not show me the description of the website and it says: A description for this result is not available because of this site's robots.txt – learn more.

I used to have a robots.txt file but have deleted it since the migration to the client's host however Google seems to have indexed it in the meanwhile. I got in touch with the hosting company and they said that it is a virtual file and that it doesn't physically exist - and that the issue with the description in Google is happening because it has indexed the website while the robots.txt was there.

I want all references to website1 and website2 to vanish from Google and I only want it to index, which is the main business website - with the correct description of the site

What are my options? I logged into Google Webmaster Tools and it still says that the file is there.

I hope you understand what I want to achieve but feel free to ask for clarification

Any advice will be greatly appreciated as I want to solve this issue ASAP before it starts damaging the client's online presence and reputation

share|improve this question
just checked webmaster tools and this is what it says: Googlebot is blocked from – user28690 Jun 10 '13 at 13:17

1 Answer 1

up vote 1 down vote accepted

Rather than delete your robots.txt file, you should upload one that explicitly allows all crawling.

User-agent: *

When you delete a robots.txt file, Google often uses the old one for some time period (maybe even weeks) to make sure that you didn't delete it by accident.

share|improve this answer
you are not going to believe it! I seem to have left an option ticked that says something like "Discourage crawlers to index this site" - so naturally I unticked it and what Wordpress did was modify the content of my htaccess file and the website was all of a sudden not available :error 500. Never thought Wordpress actually modifies the htaccess file – user28690 Jun 10 '13 at 14:24
we learn something new everyday - I now know that messing with that WP option will update my htaccess file!! – user28690 Jun 10 '13 at 14:34

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.