Google is still indexing 410 pages which was implemented 2-3 month ago. These pages should have gone away at this time.
So, Should it be logical to implement X-Robots-Tag: Noindex,Noarchive while having 410 http status?
What's your suggestion?
3 Answers
No, there is no need for that. As John Mueller said in Webmasters central
From our point of view, in the mid term/long term, a 404 is the same as a 410 for us. So in both of these cases, we drop those URLs from our index.
It is normal if Google stills crawl those URLs from time to time:
We’ll still go back and recheck and make sure those pages are really gone or maybe the pages have come back alive again.
If those pages are still indexed it could be because they don't have much popularity and Googlebot doesn't crawl them very often. Just wait or use the Remove URLs tool to speed up the process.
There is no need to use X-Robots-Tag: Noindex,Noarchive. Google automatically removes 410 pages from its index within hours of crawling them. (Compared to 404 pages which it will give a 24 grace period to see if they come back.)
Google would usually have crawled and de-indexed a 410 page within 2-3 months.
- Check to ensure that these pages are NOT disallowed in robots.txt. If Google can't crawl the page, it can't see the 410 status. (Some people disallow pages in robots.txt to try to get them removed, but robots.txt is for only for blocking crawling. Google may still index disallowed pages if it crawled them before, or based on external links.)
- Use the Remove URLs tool from Google Search Console to individually remove remaining URLs. (documentation)
- If there are too many of the pages to submit one at a time, create a temporary sitemap of them and add that to Search Console.
- Otherwise, just try waiting a few more months. Googlebot will eventually crawl all your pages again but if the pages don't have any links from anywhere it could be six months between crawls.
answered Oct 18, 2019 at 10:20
Look at your log-files - is googlebot visited 410 pages since they become 410? If yes - just wait, if not - make a sitemap containing only 410 pages and upload it into search console.
