1

My website is using WordPress with the following robots.txt

User-agent: *
Disallow: /cgi-bin
Disallow: /wp-admin
Disallow: /wp-includes    
Disallow: /wp-content/cache
Disallow: /trackback
Disallow: /*trackback
Disallow: /*trackback*
Disallow: /*/trackback
Disallow: /*?*
Disallow: /*.html/$
Disallow: /*feed*

# Google Image
User-agent: Googlebot-Image
Disallow:
Allow: /*

# Google AdSense
User-agent: Mediapartners-Google*
Disallow:
Allow: /*

I updated the robots.txt file 3 days ago and the problem I'm having is when I search for the site on Google or Bing it shows all sub pages but not the home page. Google says the following for the homepage:

A description for this result is not available because of this site's robots.txt – learn more.

I tested using Google webmaster tools and it shows no errors. It shows description of home page when is search with site:domainname.com. Is this a WordPress issue or does my robots.txt file have errors.

1

We have also faced similar issue on our website. This happened because initially you must have disallowed your home page in robots.txt. Now even if you allow, it takes sometime around a week-two to reflect in google indexes.

I have tested your robots.txt and it looks fine.

| improve this answer | |

Not the answer you're looking for? Browse other questions tagged or ask your own question.