2

Could me somebody explain following topic:

On different sites i realized two scenarios, how not existing urls are handled. I'm not sure, which setup causes which scenario and which scenario should be preferred and why.

These scenarios are: on accessing of no longer existing url

  • the url remains the same, but the page has the status code 404,
  • an url changes to /404.html

Other differences are in what kind of 404 page is displayed. It can be a custom 404 page, which was setup with Error Document, otherwise it can be a default server 404 page or a default 404 page of CMS (if any in use).

The question is: which setup details cause the cases i listed above?

4

the url remains the same, but the page has the status code 404,

This is certainly the preferred option, as Martijn suggests. For a "custom" error page on Apache (ignoring the CMS option for a moment), this could be achieved with the ErrorDocument directive:

ErrorDocument 404 /errors/my404.php

it can be a default server 404 page

Using the default keyword clears any custom URL that might have been set and results in the servers default error document being served.

ErrorDocument 404 default

or a default 404 page of CMS

Since a CMS is often serving pages defined in a database, it's not possible to use the ErrorDocument directive to handle 404s for pages defined (or not) in the CMS, since whether a request results in a 404 is only known much later in the request. The CMS detects that the requested URL does not exist and manually serves a 404 Not Found header with some appropriate page content.

an url changes to /404.html

For the URL to change there must be some kind of external redirect. Most probably a 3xx status code. To me, this would suggest a misconfiguration. The user-agent first sees the 3xx status code, not a 404. The document being redirected to may or may not then serve a 404 (it should). The user-agent ends up issuing 2 requests (because of the redirect). Providing the end-point returns a 404 status it is still effectively a 404.

Other caveats with issuing a redirect to the error document is that you lose the original URL (to provide a meaningful message to your user) and other details (server variables) related to the error. (This information would need to be manually passed somehow if it was required.)

There is rarely a need to redirect to a 404. On Apache, this is possible again with the ErrorDocument directive. If you specify an absolute URL (with protocol and hostname) as the error document argument then Apache issues a 302 (temporary) redirect (regardless of whether the URL actually resides on the same server):

ErrorDocument 404 http://example.com/errors/my404.php

This is primarily for redirecting to an external resource (should that be a requirement). But the same caveats with issuing a redirect apply, as mentioned above.

  • One other option is to edit the 404 page served by the web server. I used to do this with IIS and later with Apache. It is not a redirect to a 404 page, but the exact response page that the server will issue whenever a 404 error occurs. I personally like editing this file for a custom 404. However, the CMS that I wrote allows me to use any page as a 404 so that I can tailor the 404 error for the specific URL for maximum recovery. I rarely do this these days because I simply do not care anymore. ;-) However it is a valid idea for those who can do it. Cheers!! – closetnoc Mar 24 '17 at 16:45
4

The TLDR; answer: Send out a 404 header, don't change the url.

The longer answer:
There are more options then you suggest, which I'm not going to list here (because it's not really needed).

The most basic solution is just to serve a 404 header when the page does not exist. The reason for this is simple; This is what visitors expect. You open a URL and that either works or it doesn't.

Redirects to other pages are possible, but this should only be done when there is relevent content on the new page (do not redirect everything to home!). A better solution would be a custom page, with a few suggestions for your visitor, along the lines of:

"Oops! Page not found. Maybe you're looking for X, or perhaps Y? You can also go to Home"

  • @MrWhite per your comment, do you see any issues with a 404 response on the second (redirected) request? I'm trying to determine if a page that no longer exists that has a 302 to 404 makes sense. Thanks – Rob T Nov 26 '19 at 14:49
  • Dont 302 to a 404. Just instantly do a 404. It's the first page that doesnt exist, so the first page should do that – Martijn Nov 26 '19 at 15:00
  • @RobT I covered the "issues" with this approach in my answer. – MrWhite Nov 26 '19 at 15:31
  • @Martijn I agree the straight 404 would be best but I think this site's webmaster is using this method when items go out of stock, as if trying to indicate to crawlers "hey this is a temporary 404". So just looking for a reason why this is bad practice. – Rob T Nov 26 '19 at 19:13
  • 1
    @MrWhite if I am interpreting your answer correctly, it sounds like "as long as it ends in 404 you're good", but I don't get why someone would 302 to 404 vs 301 to 404 – Rob T Nov 26 '19 at 19:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.