Why is Google reporting CSV files as "soft 404"?

Question

I have a few hundred soft 404 errors reported in Google Search Console. Almost all of them are for CSV files containing data. For example here is the HTTP response for one of them:

HTTP/1.1 200 
Content-Disposition: attachment; filename="fewer-bank-failures.csv"
Content-Length: 116
Content-Type: text/csv; name="fewer-bank-failures.csv";charset=UTF-8
Date: Thu, 01 Feb 2018 11:32:56 GMT
Server: Apache
Connection: keep-alive

"",Bank Failures
2000,2
2001,4
2002,11
2003,3
2004,4
2005,0
2006,0
2007,3
2008,25
2009,140
2010,157
2011,92
2012,51

Why is Google reporting that this is a soft 404? I've usually seen soft 404 because:

• You have a "200 OK" status but say "not found" in the page
• You redirect to the home page
• The page is blank

I can't figure out why Google would think that this CSV file would indicate a not found error.

I do understand other reasons that Google might not want to index this content:

• It is a download attachment rather then a page
• CSV wouldn't be the best landing page experience
• The content is duplicate -- we have the an HTML page with the same data including a graph.

I would expect Google to choose not to index the page for one of those reasons, but I am completely surprised that they call it a "soft 404".

What can I do to tell Google that the page is real? Would using a Link: <https://example.com/fewer-bank-failures.html>; rel="canonical" HTTP header help?

Answer 1

Well, it does fall within the realms of Google's definition of a soft 404 (highlighting my own):

A soft 404 means that a URL on your site returns a page telling the user that the page does not exist and also a 200-level (success) code to the browser. (In some cases, instead of a "not found" page, it might be a page with little or no usable content -- for example, a sparsely populated or empty page.)

So, from that "definition" you can't really say it's not a soft 404.

What can I do to tell Google that the page is real?

But is the .csv file is a "real page"? It is a file that is available for download from the "real page". You could try setting a rel="canonical" Link: HTTP response header (as you suggest), but it may just be ignored, as it's not duplicate nor indexed (which is what the rel="canonical" tag is usually used to resolve). However, there would be no harm in doing so.

Since Google already considers these files "soft-404" they aren't going to be indexed anyway. You could simply block crawling with robots.txt (as @closetnoc suggested in comments) to save some time and suppress the warning.

Alternatively, you could send an X-Robots-Tag: noindex HTTP response header to simply block indexing (and still allow the resource to be crawled).

At the end of the day, the "soft-404" warning is just a FYI from Google. If Google considers there to be "no usable content" then it's not going to be indexed, whether Google tells you about it or not. So there is nothing you really can do, apart from suppressing the warning in GSC.

Answer 2

It does seem like it falls under the Google's linked description of a soft 404 by @Docroot. I think in a situation like this, even though we know that these CSV files aren't worth indexing, we might not want to block Google from being able to crawl them because we don't want to hide content from Google so that Google doesn't think there's anything deceptive going on with the site.

I'm not sure of a remedy to this issue, but here are some possible solutions:

1. Maybe have the user download the csv files through Javascript rather than a followed link. Google will be less likely to crawl the Javascript URLs and will be more likely to see these URLs as server requests than actual landing pages. While it might be wise to allow Google to crawl these CSV files, it also isn't useful to pass link juice to them. Javascript could remedy this.

2. Perhaps there is an alternate header status that you can use, though I'm not sure if that is possible.

3. I don't think it's possible to set a NOINDEX NOFOLLOW status in a csv file, but it's something to think about. If there's a way to tell Google not to index these pages that would be better than them interpreting all of these links as soft 404s.