Google Search is indexing my robots.txt file. I know this because when I search site:example.com on Google, my robots.txt shows in the list of results. I don't want my robots.txt to show in Google.

The content of my robots.txt is

User-agent: *

Sitemap: https://example.com/sitemap.xml

1 Answer 1


Why is Google indexing my robots.txt file?

It is normal behavior for Google to index your robots.txt file. It's a URL on your website just like any other.

I don't want my robots.txt to show in Google.

Your robots.txt file already does not show on Google, in the general sense that Google Search users could stumble upon it without specifically looking for it. Google is smart enough to know that it's generally irrelevant to searchers. Nobody will see it in search results unless they are trying to comb through your site's index using the site: operator.

To answer the question as written: To prevent your robots.txt file from being indexed, you can configure your web server to serve it up with a X-Robots-Tag: noindex header. However there is no practical benefit to doing so, and depending on your server software it may involve some extra setup/configuration.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.