2

Two days ago I've got an email with a warning from Google Search Console, telling me that 3 URLs from my site where indexed, but were blocked by robots.txt.

Since I first deployed my website, I haven't changed the robots.txt file.

robots.txt

User-agent: *
Disallow: /admin
Disallow: /admin/
Disallow: /blog/search
Disallow: /blog/search/

Sitemap: https://www.example.com/sitemap.xml

And these are the pages that I got the warning from:

https://www.example.com/blog/slug-1
https://www.example.com/blog/slug-2
https://www.example.com/blog/slug-3

I think everyone agrees that those pages are NOT being blocked by my robots.txt file, right?

Note: I have others blogposts with other slugs and they're just fine. No warnings beyond those 3 urls.

Anyway, I clicked the Fix Issues button on Google Search Console and they're currently in pending state.

Has this happened to anyone before? What could be the cause of this?


UPDATE:

Infact, I just inspected one of the 3 urls and got this:

enter image description here

But when I clicked on Test Live URL, this is what I get:

enter image description here

And I'm 100% my robots.txt file hasn't changed since I first deployed this website. I.e: this is 100% a bug from the Googlebot crawling process.

2

This exact bug is being reported by others in the Google webmaster forum. It may be an actual bug & you might want to report it there.

|improve this answer|||||
  • Thank you! Could you give me link of the discussion? – cbdeveloper Mar 3 at 10:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.