0

For the last few days I haven't seen the title and meta description in Google search results. When I inspected URL in search console, I got this error message:

URL is on Google, but has issues.

Coverage: Indexed, though blocked by robots.txt

It isn't blocked by robots and it's also in the sitemap.

I have checked the CMS, meta data is updated.

URL inspection in Search Console

URL inspection in Search Console

Live testing in search console

Live testing in search console

Title and Description is not Appearing in SERP

Title and Description is not Appearing in SERP

  • 1
    When did you last update robots.txt? The robots testing tool shows that the home page should be allowed now. You might still get that message if you had disallowed it any time in the past couple months. – Stephen Ostermiller May 18 at 13:33
  • 1
    There are some problems with your robots.txt file that you should fix, but they probably aren't related to this. 1. Remove blank lines between User-Agent, Allow,` and Disallow. Blank links are not allowed there per the spec. 2. Remove the Allow: / rule. It is not needed because allowing everything is already the default. It could only confuse some crawlers. 3. Remove all the rules that start with Disallow: http or remove the URL portions except the path. Disallowed items must start with /. 4. Remove duplicate rules. – Stephen Ostermiller May 18 at 13:38
  • 1
    5. Remove redundant rules like Disallow: ?source=, Disallow: /customer/account/, and Disallow: /index.php/vendorform/view/index/ which are covered by other rules. 6. Add a space after the disallow: in Disallow:*--*. – Stephen Ostermiller May 18 at 13:39
  • @StephenOstermiller Thanks for helping out. I tried robots testing tool and got this error robots.txt fetch failed You have a robots.txt file that we are currently unable to fetch. In such cases we stop crawling your site until we get hold of a robots.txt, or fall back to the last known good robots.txt file. Learn more. – Ali Silat May 18 at 23:53
  • It was last updated 9 months ago. – Ali Silat May 19 at 0:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.