robots.txt I disallowed some of the "thank you" pages on my site that I don't want to indexed by Google or be in Google search results. But those pages have recently become searchable on Google. What should I do to get them deleted?
robots.txt file cannot and should not be used to disallow search engine indexing. It only prevents pages from being crawled, not from being indexed.
To prevent pages from being indexed, either serve them with this HTML meta tag in the head:
<meta name="robots" content="noindex">
Or with this HTTP header:
Once you are serving the
noindex directive, the pages will fall out of Google Search over time. If you need to remove them urgently, submit them using the Google Remove Outdated Content Tool. This will only work if the pages are serving the
noindex directive or are gone altogether.