There are 404 pages with two kinds of response headers (copypasted in full length from Chrome DevTools, Network tab):

  1. Response headers:

    cache-control:max-age=0, no-store content-type:text/html date:Wed, 24 Jan 2018 10:55:59 GMT server:Apache/2.4.29 (Ubuntu) status:404 x-powered-by:PHP/5.5.9-1ubuntu4.22

  2. Response headers

    cache-control:max-age=0, no-store cache-control:no-cache, max-age=0, must-revalidate content-type:text/html; charset="utf-8" date:Wed, 24 Jan 2018 10:55:40 GMT expires:Thu, 19 Nov 1981 08:52:00 GMT pragma:no-cache server:Apache/2.4.29 (Ubuntu) set-cookie:bypassStaticCache=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; Max-Age=0; path=/; httponly set-cookie:bypassStaticCache=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; Max-Age=0; path=/; httponly status:404 x-powered-by:PHP/5.5.9-1ubuntu4.22

The pages with the first kind of response headers aren't recognized by Google as 404. Instead of 404 alerts the Search Console those pages as duplicate pages without canonical tag.

The pages with the second kind of response headers are correctly recognized as 404.

Not recognized means: Google calls such pages Duplicate page without canonical tag, despite the fact, that developer tools of Firefox and Chrome get correct 404 status code.

Recognized means: such pages are called Not found (404), like on the following screenshot: enter image description here

Why is it so? What prevents the correct status code recognition? Does the answer need additional information? Just say - i'll try to provide it.

PS: maybe it is a bug of the new Search Console...? @JohnMu

  • I'd recommend getting the headers from curl --head <url> or curl -D - <url> rather than use chrome. As DocRoot points out, it looks like you might be trying to set the status in a header field rather than in the first line of the response. Or it could just be Chrome mucking it up. You should use curl to see which of those is true. Apr 24 '18 at 18:33

The HTTP response status is indicated by the very first line of the response (the "Status Line") - which you aren't currently showing in the output in your question. For a 404 response, you would expect to see something like:

HTTP/1.1 404 Not Found

The Status response header is non-standard.

  • It would make sence for me, if this status would not be recognized in the second case...
    – Evgeniy
    Jan 24 '18 at 12:30
  • Sorry, I'm not sure what you mean? status:404 is probably ignored in both cases. What is the first line of the response? What does the browser think the response status is? Is Google (GSC I assume?) seeing the second response as a 404 or a "soft-404"?
    – DocRoot
    Jan 24 '18 at 12:35
  • as "real" 404 - thats it. What do you mean with "first line"? - i cited both full responses.
    – Evgeniy
    Jan 24 '18 at 12:53
  • The output in your question looks like it could be a parsed response. In Google Chrome you should have the option to "view source". The very first line of the response should be of the form HTTP/1.N <CODE> <MESSAGE>, eg. HTTP/1.1 404 Not Found (as mentioned above) or HTTP/1.1 200 OK etc. But even before viewing the headers, "What does the browser think the response status is?" (Network tab of the Developer Tools / Inspector)
    – DocRoot
    Jan 24 '18 at 15:52
  • Response headers i cited above are from Chrome developer tools, Network tab. Both kinds of page are shown in Developer tools as 404 pages. In case of page with not-recognized 404 the only content is the text "file not found". In case of recognized 404 the content is a customized (non-standard) HTML-page without any status message, like HTTP/1.1 404 Not Found.
    – Evgeniy
    Jan 24 '18 at 16:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.