1

I have a client that has 3 domain names. For the sake of this argument I will call them www.website1.com, www.website2.com and www.website3.com.

He used to have a website package with VistaPrint and www.website1.com was his main business website and website2 and website3 used to redirect to the main website.

However recently I created a new WordPress website for him and now his main domain name is www.website3.com and website1 and website2 simply redirect to the main website.

But when I go on Google and search for the domain name (any of the three) it does not show me the description of the website and it says: A description for this result is not available because of this site's robots.txt – learn more.

I used to have a robots.txt file but have deleted it since the migration to the client's host however Google seems to have indexed it in the meanwhile. I got in touch with the hosting company and they said that it is a virtual file and that it doesn't physically exist - and that the issue with the description in Google is happening because it has indexed the website while the robots.txt was there.

I want all references to website1 and website2 to vanish from Google and I only want it to index www.website3.com, which is the main business website - with the correct description of the site

What are my options? I logged into Google Webmaster Tools and it still says that the file is there.

I hope you understand what I want to achieve but feel free to ask for clarification

Any advice will be greatly appreciated as I want to solve this issue ASAP before it starts damaging the client's online presence and reputation

Improve this question
1
  • just checked webmaster tools and this is what it says: Googlebot is blocked from http://www.mydomain.com/
    – user28690
    Commented Jun 10, 2013 at 13:17

1 Answer 1

Reset to default
1

Rather than delete your robots.txt file, you should upload one that explicitly allows all crawling. 

User-agent: *
Disallow:

When you delete a robots.txt file, Google often uses the old one for some time period (maybe even weeks) to make sure that you didn't delete it by accident.

Improve this answer
2
  • you are not going to believe it! I seem to have left an option ticked that says something like "Discourage crawlers to index this site" - so naturally I unticked it and what Wordpress did was modify the content of my htaccess file and the website was all of a sudden not available :error 500. Never thought Wordpress actually modifies the htaccess file
    – user28690
    Commented Jun 10, 2013 at 14:24
  • we learn something new everyday - I now know that messing with that WP option will update my htaccess file!!
    – user28690
    Commented Jun 10, 2013 at 14:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.