This question already has an answer here:

In my robots.txt I disallowed some of the "thank you" pages on my site that I don't want to indexed by Google or be in Google search results. But those pages have recently become searchable on Google. What should I do to get them deleted?

marked as duplicate by John Conde Dec 15 '18 at 21:11

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.


The robots.txt file cannot and should not be used to disallow search engine indexing. It only prevents pages from being crawled, not from being indexed.

To prevent pages from being indexed, either serve them with this HTML meta tag in the head:

<meta name="robots" content="noindex">

Or with this HTTP header:

X-Robots-Tag: noindex

Once you are serving the noindex directive, the pages will fall out of Google Search over time. If you need to remove them urgently, submit them using the Google Remove Outdated Content Tool. This will only work if the pages are serving the noindex directive or are gone altogether.

Not the answer you're looking for? Browse other questions tagged or ask your own question.