In my robots.txt I disallowed some of the "thank you" pages on my site that I don't want to indexed by Google or be in Google search results. But those pages have recently become searchable on Google. What should I do to get them deleted?


The robots.txt file cannot and should not be used to disallow search engine indexing. It only prevents pages from being crawled, not from being indexed.

To prevent pages from being indexed, either serve them with this HTML meta tag in the head:

<meta name="robots" content="noindex">

Or with this HTTP header:

X-Robots-Tag: noindex

Once you are serving the noindex directive, the pages will fall out of Google Search over time. If you need to remove them urgently, submit them using the Google Remove Outdated Content Tool. This will only work if the pages are serving the noindex directive or are gone altogether.

| improve this answer | |

Not the answer you're looking for? Browse other questions tagged or ask your own question.