2

I need assistance with some issues on a Big Commerce site.

To preface, it is not my site and the IT team has a complex technology stack being utilized to build the site.

The issue is that adding an extra character in different areas of the URL structure (to create a URL that doesn't exist) produces different response codes. Examples:

  • /shop/collection/women = 200 and valid page
  • /Ashop/collection/women = 404
  • /shop/Acollection/women = 500
  • /shop/collection/Awomen = 200 but no content

My suggestion is to have any URL that does not exist return a 404 not found code. Their IT department set up their staging site to work like this:

Redirect trace info

My questions are:

  1. Is this correct? Can't the server just be configured to return a 404?
  2. Would a 200 then JS redirect to a 404 still inform search engines that the content is no longer there? Or since the 404 is associated with a different URL "/Notfound" it won't be useful?
2

It is always better to have proper 404 responses. However, search engines can deal with both JS redirects to 404 pages and blank pages. They should treat them the same as a proper 404 error.

There is a long history of web servers redirecting to error pages rather than showing a proper error status directly. Search engine bots have to be programmed to handle that case or they will misclassify pages on a large number of sites. Search engines are even pretty good about JavaScript redirects these days. For the most part they treat them the same as 3xx status redirects.

Google calls 200 status error pages "soft 404" pages. It shows them in a special report in Google search console, but otherwise treats them the same as real 404 pages. That is, Google doesn't index them, but Googlebot may return to crawl them periodically to see if they have changed.

Even if search engines can deal with a blank page, it is not good for users. It makes a site look very broken to get a blank page. Users much prefer helpful error messages.

As for whether the server could be configured to return a 404 -- Yes it could. I've never worked with a site for which I couldn't eventually get proper statuses. Whether or not it is worth doing is mostly a matter of:

  • Whether anybody understands the code base well enough to make the changes.
  • How much effort (and cost) would be involved in making the changes.
  • How likely it would be to break something else while making the changes.
  • I think question 2 was the real issue here, and explaining that crawlers see it as a soft 404 makes sense. Appreciate it! – Rob T Nov 25 '19 at 14:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.