What is the proper metadata to set on temporary 500 errors?

Question

Context

I am building a web application with server side rendering on top of a headless CMS. Sometimes the CMS is down and the frontend has no idea what to display to the user, other than showing a generic error page. I am setting the HTTP status code to 500 and wondering what the proper metadata is to set in these cases.

Goals

• The page should not be removed from the index
• If possible, crawlers should not update their index of the page with the error page content

Possible actions

HTTP status codes

Sending 500 Internal Server Error or 503 Service Unavailable as appropriate seems like a good idea and hopefully causes the crawlers to not update the index and try again after some time.

noIndex

When Googlebot next crawls that page and sees the tag or header, Googlebot will drop that page entirely from Google Search results, regardless of whether other sites link to it.

Seems not to fit this case since the error is temporary, so I am not going to do this.

title

Seems like a good idea to set to something like Something went wrong | Sitename for the users as it shows up in their tab etc. I just hope the crawlers don't update the index to it. If this value does end up in the index even if the HTTP status code was 500 or 503, I would put Path | Sitename or something similar.

Meta description

Depending on the above I would put either what I am going to show to the user, or something more specific that would fit better in the search result such as There was a temporary error during the last indexing of this page.

Answer 1

It is my experience that Google generally does nothing for a previously indexed page when it sees a 5xx error. It may drop it from the SERPs temporarily if the 5xx error exists for too long. The page is not de-indexed.

The reason is simple.

5xx errors are seen as temporary and likely to be fixed once discovered.

I would focus on why your site is going down and advise you NOT TO change the 5xx error page. Again the reason is simple. Why confuse things? Don't fix what is not broken. Fix what is broken.

Answer 2

For SEO, the title and the meta tags of a 5xx page don't matter. Search engines bots see the HTTP status code and base their decisions only on that. They will ignore titles, meta descriptions, and noindex.

For the most part, search engines do what you want. They never index content from the error page itself. For most error statuses, they retry the request again later and give a grace period (usually 24 hours) before removing pages from the index.

There are some differences between the various error codes:

• 500 Internal Server Error is the appropriate error to show when the code powering the page fails due to an exception or error. It indicates that there is supposed to be content at that URL, but something went wrong. It may or may not be an intermittent error. There is no promise as to when it will be fixed.

• 503 Service Unavailable is the appropriate error to show when the site is down for maintenance. Typically the entire site (or a large section of the site) will show this error all at once. Ideally the message along with it will say how long the outage is expected to last. For bots you can put that information into the Retry-After HTTP header. See HTTP headers | Retry-After - GeeksforGeeks

• 404 Not Found is the appropriate error to show when the CMS does not have a rule for what content should be at that URL.

• 410 Gone is the appropriate error to show when content has been removed from a URL. This is the one error status to which Google does not give a grace period, and removes the URL from its index right away.

Answer 3

There are some good answers here. The bottom line is that a 500-series error tells the search engine "there's a temporary issue right now, come back later".

The search engine will do just that - come back later. It won't index or really even read any of the content on the error page, so don't worry about that.

In summary, just the 500-series status code is enough. Leave it up until you can serve the proper page again. You don't need to do anything else SEO-wise.