2

There was an old website located at http://example.com. Then it has been replaced by a new website, with a totally new url configuration, leaving old domain, but switching to https-only. So now all http-requests are redirected to https://example.com.

The problem now is that there are a lot of search bot requests that try to access old-website urls and receive 301 as a response, redirecting to https site, and only then properly receiving 404.

So, it's been several months now and it looks like bots are not going to forget about old urls. Maybe they just don't understand that kind of response and consider a page still alive? If that so, how could I possibly fix the problem? Naturally, I could allow users to interact with http-site as well (disable the redirect), but that's generally unsecure (though, probably, no big deal for a short time), so I'd rather not do this.

  • Short answer? Nope. You are fine. This is how it is supposed to work. Sure there are other options, however, Google does publicly state they prefer a 404 over anything else when a resource does not exist. – closetnoc Feb 28 '15 at 1:49
1

Once you publish a URL, search engine bots such as Googlebot will request it for all time. I still get requests for URLs retired 15 years ago.

Redirecting to a 404 page isn't significantly worse than showing the 404 page directly. Googlebot and other search engine bots will be able to understand that the page is missing when you redirect to a 404.

You could consider showing a "410 Gone" status page rather than a 404 page. There isn't much difference between the two other than 410 will cause search engines to remove content from their indexes immediately as opposed to getting a grace period of a day for 404 status.

The only thing that you can do that would be better would be to redirect the URL to replacement page. If you have a replacement page, that would get all reputation and links that pointed to the old page to the new page. I wouldn't redirect URLs to your home page (or other central page). Google considers such redirects "soft 404". It treats redirects to the home page the same as if the server returned 404 status.

2

A soft-404 is a page that contains readable text that describes it as an error page but the actual status code attached to that page is not 404.

For example, creating an HTML file with the contents along the lines of "The resources you are trying to request were not found" and then saving it on the server then accessing it would qualify for a soft-404 because of the words "not found" and because the HTML headers that will return (as with any HTML page) will start with "HTTP/1.1 200 OK" or "HTTP/1.0 200 OK".

If you want to make sure you're making a true 404 error page, use any programming language and force the 404 result code. In PHP, a ridiculously simple page can be done this way:

<?php
header("HTTP/1.1 404 Not Found",true);
?>
<!-- insert full HTML here -->

If you have access to curl, use it with the -I option on the full URL to see the returned headers.

So If you must redirect to 404 error, at least redirect to a real 404 error, not a soft one. What's even better than that is to skip the redirection phase and jump straight to the 404 page (or better yet, the page with error 410).

-2

Putting up http headers which inform the crawlers about the state of a page is always a good idea. A 301 redirect to a different page is a best option so you keep your a) page rep, b) incoming links. I have had a lot of clients simply removing their pages carelessly and thus losing these two, which is deadly for SEO. If there is nowhere to redirect the page, put an 410 Gone instead of it, which is a clear signal the page has been removed. 404 is a kind of non-solution. Also you might want to scout your site for addresses which trigger 404s and put something up again, even if it is 301 redirects. It is a good way to regain your precious lost inbound links.

  • This doesn't address the question of the advantages of showing a 404 status directly vs redirecting to the 404 page. – Stephen Ostermiller Nov 14 '18 at 15:10
  • If someone asks, if he should put sand or rocks into his gas tank, you don't play his sands and rocks game, but tell him, he would do much better with diesel or unleaded. Hence my answer. – Marko36 Nov 14 '18 at 15:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.