18

Does it make sense to return a 410 instead of 404 when some page has been permanently removed? In this video, Matt Cutts says that Google treats them the same, but are there other reasons to implement a 410 (except for technical correctness)?

1
  • I have wondered about this myself. It seems like 410 just isn't used anywhere anymore. – wogsland Mar 3 '17 at 14:54
7

On our website we have recently implemented 410 errors for pages that have been removed permanently. We have around 40 million pages in the Google index and get crawled with 2 million requests per day by the Googlebot.

After cleaning up our database we found a large number of 404 errors kept showing up in the Crawl Errors on Webmaster Tools. When we switched to 410 statuses instead of 404 the number of errors per day cut in half while the total number of requests done kept around the same. So that's more successfully crawled pages per day in the end. It looks like pages with a 404 status get crawled multiple times before being removed from the index fully.

So I would say, yes definitely worth using 410 where appropriate in order to maintain your Google Crawl budget.

1
  • Thanks for the info @stefan. Would like to know 1) how much time it took for the change to get implemented and those pages to be removed from the index 2) Did you see increase in traffic because of this change? – Saurabh Goel Jul 14 '20 at 10:22
12

There is more than Google in this world. A 410 unambiguously tells a bot that the file is gone. A 404 does not. A persistent bot might keep trying to find a 404 indefinitely whereas they might stop trying to find a 410 immediately which would make your server very happy.

3
  • 2
    It goes both ways, though: a really crappy bot might only recognise 200 and 404. – Peter Taylor Feb 7 '12 at 21:56
  • 4
    @Peter not sure that matters - if they are crappy and don't understand HTTP codes then what harm would it do? If you use 404 then both crappy and good bots will keep requesting the page, whereas with 410 only the crappy one will. – DisgruntledGoat Feb 7 '12 at 22:47
  • 1
    What @DisgruntledGoat said. See Postel's Law. Crappy implementations simply can't be accounted for; what are you going to do, try and account for all of them and each of their particular quirks, all at the same time? There are rare exceptions, like say IE6's long-standing browser domination, but in general it's simply not worth the effort. – Su' Feb 8 '12 at 5:03
6

Besides there being more search engines than Google out there, there's also no reason to assume that Google won't ever change the way they treat 410 responses. Indeed, it seems that's already happened: the information Matt Cutts quotes in the video is from 2007, whereas this post from 2009 by John Mu on Google's Webmaster Central forums says otherwise:

"I followed up on the 404 vs 410 thing with the team here. As mentioned by some others here & elsewhere, we have generally been treating them the same in the past.

However, after looking at how webmasters use them in practice we are now treating the 410 HTTP result code as a bit "more permanent" than a 404. So if you're absolutely sure that a page no longer exists and will never exist again, using a 410 would likely be a good thing."

So it looks like Google is now indeed treating 410 responses differently from 404s.

4

In my experience, Google doesn't understand 410 error codes.

Some time ago, I changed a full site so all the old URLs are "410 Gone". I can't use 301 redirects because there is no direct relationship to a new URL.

I used a .htaccess to tell Google the old URLs are gone, but it keeps telling me there are a HUGE amount of 404 errors on my site. I checked the URLs in the crawl error report and they are all are 410, not 404 as it states.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.