0

My website addresses has a hierarchy structure. When one page is requested, and it is not a valid address, it tries to find its parent, and if the later exists, it will show it. Now, my question is that in this case, should I send a 200 header response or a 404 one? How differently do the search engines react to these headers?

1

I strongly suggest you examine the status codes and their meanings. You did not explain why a client (which could be a human or a bot) might request a URL that does not exist but the URL might have a valid "parent" URL. The reasons for unusual situation will likely guide your choices of status codes.

The information provided by closetnoc is excellent, and I only have a few things to add.

  1. 203 might make sense depending on the relationship between the requested material and the served material.
  2. Some status codes are largely for the benefit of humans (like 401), while others are largely for the benefit of bots (like 301). What is your target audience--human, bot, or both?
  3. You might want to return different status codes based on the requesting client. For example, maybe you want search engines (bots) to completely remove the now-gone URL from their database, so you return 410, but you want to keep human users on your site, so you return 201, 301, 303, or 404.
  4. You asked, "should I send a 200 header response[?]" Absolutely not.
  5. "How differently do the search engines react to these headers?" See Google's answer.
  6. Your fundamental issue seems to be link rot. If this is a common issue on your site, you might benefit from learning as much as you can about link rot.
  • 1
    One problem with at least part of your premise is that it takes a fair amount of data infrastructure and processing to know a bot from a human. There are some clues, however, it is not a snap to know bot from human anymore. I have about 25 or more criteria to know for sure and even then, that is not always possible. Due to the nature of the question, I thought it best to keep the answer simple and geared toward the main-stream without getting into too many nuances. I understand why he is doing this. Pages come and go but links do not match the effort. I still see links from 12+ years ago. – closetnoc Jan 6 '15 at 14:29
  • Excellent point, my bot list is so long that I don't even want to count how many are on it. I assume clients are human unless I know they are a bot. Since the majority of the bot traffic I care about comes from just a few sources, I don't cry to much about the scrapers. But you are right that people should keep in mind that it is more work to do it this way. – hunterhogan Jan 6 '15 at 14:42
  • I was not trying to be critical. Trust me. I gauge the question based upon the OPs knowledge and experience. This was an odd one. Clearly the OP is not a noob, but asked a fundamental question. As well, a lot of people still try and use the agent name and other criteria that can be spoofed easily. I apply 7+ years of historical data and advanced network trust analysis to determine bots from humans and even then, sometimes a bot can have a human behind it that confuses the system slightly. Also keep in mind, that Google prefers certain status codes over others such as a 404 over a 301. Sheesh! – closetnoc Jan 6 '15 at 14:50
  • I did not interpret your comment as critical. I sincerely think it is a valid point and well worth considering. Very few people would benefit from my suggesting of trying to separate bots from humans, and since I did not make that clear, your comment did helps. In short, I agree with you. – hunterhogan Jan 6 '15 at 14:54
  • 1
    Oh I know. I just want to make sure that you knew I was not wearing my waffle stompers (hiking shoes- but fluffy white rabbit ear slippers). I like the answers you give and you are going to be one of the great ones for sure. I did not want to discourage but encourage you. ;-) – closetnoc Jan 6 '15 at 14:59
1

Here is a run-down of some of the HTTP Status Codes.

If you want to know more about HTTP Status Codes, I suggest this page: http://en.wikipedia.org/wiki/List_of_HTTP_status_codes It explains things rather well.

A 202 status code is OK. This means that the request was successfully fulfilled. A 301 status code Moved Permanently is a redirect. A 303 status code See Other is similar to a 301 but rarely used. A 404 status code is Not Found and the default returned by most if not all web servers. A 410 is Gone is similar to 404 but not used enough.

Traditionally, the default that a web server returns if a page is not found is a 404 Not Found, however, this means that the page may return so search engines are likely to retry this page for a set number of times. A 410 is used when a page is removed and not expected to return but has to be issued explicitly. Technically, if a page is not expected to return, then a status code of 410 is best.

A 301 status code Moved Permanently is a redirect. However, a 303 status code See Other might be more accurate though not traditionally used. A 301 status code is more widely understood and traditional although you can easily try a 303 and I am reasonably sure it will work okay.

The same applies with a 404 status code. It is widely understood, however, the 410 should work too and seems correct for your scenario.

I would venture to guess that using a 301 and/or 404 are more browser friendly though I am sure most browsers handle 410 and 303 just fine.

Since you are trying to find and present another page, you will want to do 1 of 2 things conditionally whether you are successful in finding another page or not:

If a replacement page is found: Issue a 301 (or 303) redirect to the new page. This is easy enough to do in code.

--or--

If a replacement page is not found: Issue a 404 (or 410). This is easy enough to do in code.

  • The first half of your answer is a bit rambling and all over the place, might be simpler to have a short description of the status codes then a recommendation. – DisgruntledGoat Jan 8 '15 at 16:17
  • @DisgruntledGoat You may not have liked how my answer was written and you did the right thing to make an edit suggestion. However, the answer is technically correct. The down-vote was a bit excessive but your right to do of course. I made a comment to an answer that you made that is technically incorrect, but did not down-vote you. I gave you room to adjust what could be an excellent answer with the perfectly valid points you made with some room for what is another consideration for the OP. Perhaps I offended you which was not my intent. – closetnoc Jan 8 '15 at 16:32
  • The answer is badly written (no offence) so that makes it not a good answer in my opinion. I don't think that's "excessive". (I'm happy to up vote it later if it improves.) – DisgruntledGoat Jan 8 '15 at 16:49
  • @DisgruntledGoat Saying so is enough. It seemed retaliatory due to the timing of the remarks. No big deal. For what it is worth, I do like your answers and do not mean offense to you at all. There are two things going on here: I feel that there is too much inaccuracy in SEO advice on the web and it drives me nuts; and I was attacked here before and there was a time that retaliatory down-votes as a problem. I made a remark about giving the user time to adjust their answer and if this does not happen in time, then the down-vote makes sense. It is just polite (from a southerner's pov). – closetnoc Jan 8 '15 at 16:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.