Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

Does it make sense to return a 410 instead of 404 when some page has been permanently removed? In this video, Matt Cutts says that Google treats them the same, but are there other reasons to implement a 410 (except for technical correctness)?

share|improve this question
add comment

5 Answers 5

up vote 2 down vote accepted

On our website we have recently implemented 410 errors for pages that have been removed permanently. We have around 40 million pages in the Google index and get crawled with 2 million requests per day by the Googlebot.

After cleaning up our database we found a large number of 404 errors kept showing up in the Crawl Errors on Webmaster Tools. When we switched to 410 statuses instead of 404 the number of errors per day cut in half while the total number of requests done kept around the same. So that's more successfully crawled pages per day in the end. It looks like pages with a 404 status get crawled multiple times before being removed from the index fully.

So I would say, yes definitely worth using 410 where appropriate in order to maintain your Google Crawl budget.

share|improve this answer
add comment

There is more than Google in this world. A 410 unambiguously tells a bot that the file is gone. A 404 does not. A persistent bot might keep trying to find a 404 indefinitely whereas they might stop trying to find a 410 immediately which would make your server very happy.

share|improve this answer
2  
It goes both ways, though: a really crappy bot might only recognise 200 and 404. –  Peter Taylor Feb 7 '12 at 21:56
1  
@Peter not sure that matters - if they are crappy and don't understand HTTP codes then what harm would it do? If you use 404 then both crappy and good bots will keep requesting the page, whereas with 410 only the crappy one will. –  DisgruntledGoat Feb 7 '12 at 22:47
    
What @DisgruntledGoat said. See Postel's Law. Crappy implementations simply can't be accounted for; what are you going to do, try and account for all of them and each of their particular quirks, all at the same time? There are rare exceptions, like say IE6's long-standing browser domination, but in general it's simply not worth the effort. –  Su' Feb 8 '12 at 5:03
add comment

Besides there being more search engines than Google out there, there's also no reason to assume that Google won't ever change the way they treat 410 responses. Indeed, it seems that's already happened: the information Matt Cutts quotes in the video is from 2007, whereas this post from 2009 by John Mu on Google's Webmaster Central forums says otherwise:

"I followed up on the 404 vs 410 thing with the team here. As mentioned by some others here & elsewhere, we have generally been treating them the same in the past.

However, after looking at how webmasters use them in practice we are now treating the 410 HTTP result code as a bit "more permanent" than a 404. So if you're absolutely sure that a page no longer exists and will never exist again, using a 410 would likely be a good thing."

So it looks like Google is now indeed treating 410 responses differently from 404s.

share|improve this answer
add comment

In my experience, Google doesn't understand 410 error codes.

Some time ago, I changed a full site so all the old URLs are "410 Gone". I can't use 301 redirects because there is no direct relationship to a new URL.

I used a .htaccess to tell Google the old URLs are gone, but it keeps telling me there are a HUGE amount of 404 errors on my site. I checked the URLs in the crawl error report and they are all are 410, not 404 as it states.

share|improve this answer
add comment

There is more than Google in this world.

Quoted for truth. Given your requirement, to tell bots (and presumably humans as well) that a page has been permanently removed, I'd actually opt for a 301 redirect to a page explaining (if necessary) why the content was removed, or take the bot / user back to the index page.

The lesser-known HTTP error codes usually aren't as well supported, understood or implemented across the board, so I generally stick to the ones that are in order to ensure the expected and hopefully seamless browsing experience.

Choosing which option to go with is dependent on how many request are still coming through for the (now) missing content.

share|improve this answer
1  
Why does the need to explain the removal mean you need to 301 to a different URL with that explanation? The explanation could be on the page itself, with a 410 status so that search engines understand that it's dead. –  IMSoP Sep 11 '12 at 12:03
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.