9

Our site is a private service -- that is, firewalls and user authorisation keeps both the general public and Google's indexing off it.

The site generates dynamic DocX files, which the customer downloads with their browser. It's been pretty much the same way for years.

This week we have started to intermittently see warnings in the Chrome downloads shelf:

Screenshot: "suitability-rep....docx is not commonly downloaded and may be dangerous"

I've dived into Chrome's safe browsing stuff and discovered that you can get diagnostics at chrome://safe-browsing/. There I can see that:

  • Chrome sends a safe browsing "ping" when it downloads .docx files
  • Most of the time the response to this contains "verdict": "SAFE"
  • Sometimes the response contains "verdict": "UNCOMMON", and this is when the user gets a warning.

I've also had a bit of a hunt through the Chromium source and Chrome blogs, and it doesn't look as if either Chrome's approach, nor its policy for the .docx file type has changed recently. All I can think of is that the Safe Browsing service has changed its rules.

Here's an anonymised ping payload that gave an UNCOMMON verdict:

{
   "archive_directory_count": 0,
   "archive_file_count": 0,
   "archived_binary": [  ],
   "download_type": 14,
   "file_basename": "doc.docx",
   "length": 493030,
   "referrer_chain": [ {
      "ip_addresses": [  ],
      "is_retargeting": false,
      "main_frame_url": "",
      "maybe_launched_by_external_application": false,
      "navigation_initiation": "UNDEFINED",
      "navigation_time_msec": 1.591196304544e+12,
      "referrer_main_frame_url": "",
      "referrer_url": "https://foo.bar/baz",
      "server_redirect_chain": [ "https://foo.bar/5ca70c26-9b9a-4cda-a99e-27394630910d" ],
      "type": "EVENT_URL",
      "url": "https://foo.bar/5ca70c26-9b9a-4cda-a99e-27394630910d"
   }, {
      "ip_addresses": [ "1.2.3.4" ],
      "is_retargeting": false,
      "main_frame_url": "",
      "maybe_launched_by_external_application": false,
      "navigation_initiation": "RENDERER_INITIATED_WITHOUT_USER_GESTURE",
      "navigation_time_msec": 1.591196217058e+12,
      "referrer_main_frame_url": "",
      "referrer_url": "https://foo.bar/baz/bap",
      "server_redirect_chain": [  ],
      "type": "LANDING_PAGE",
      "url": "https://foo.bar/baz"
   }, {
      "ip_addresses": [ "1.2.3.4" ],
      "is_retargeting": false,
      "main_frame_url": "",
      "maybe_launched_by_external_application": false,
      "navigation_initiation": "RENDERER_INITIATED_WITH_USER_GESTURE",
      "navigation_time_msec": 1.591196195633e+12,
      "referrer_main_frame_url": "",
      "referrer_url": "foo.bar/baz/bat",
      "server_redirect_chain": [  ],
      "type": "CLIENT_REDIRECT",
      "url": "https://foo.bar/bap"
   }, {
      "ip_addresses": [ "1.2.3.4" ],
      "is_retargeting": false,
      "main_frame_url": "",
      "maybe_launched_by_external_application": false,
      "navigation_initiation": "RENDERER_INITIATED_WITH_USER_GESTURE",
      "navigation_time_msec": 1.591196152516e+12,
      "referrer_main_frame_url": "",
      "referrer_url": "",
      "server_redirect_chain": [  ],
      "type": "LANDING_REFERRER",
      "url": "foo.bar/baz/bat"
   } ],
   "request_ap_verdicts": false,
   "url": "blob:https://foo.bar/5ca70c26-9b9a-4cda-a99e-27394630910d"
}

(The download is initiated by a user click on a React component which triggers Javascript using the file-saver library, which delivers it as a blob)

Google Search Console indicates no problems with our site -- which is to be expected because the site is private.

I understand that UNCOMMON means "we can't say this is safe because it's not a file we've seen before". It sort-of makes sense to me that our docx files could be described as uncommon, because each one is dynamically generated and therefore unique. However I don't understand why the service sometimes reaches a verdict of SAFE and sometimes UNCOMMON - at a guess there are some heuristics based on the contents of the referrer chain, but I've not been able to spot any correlations so far.

I know we can advise our users or their Enterprise admins to whitelist our site, and while this is certainly an option, I'd really like to not give them that inconvenience.

So, questions:

  • Does Google publish anything to explain their criteria for reaching an UNCOMMON verdict?
  • What can we do at the server side to prevent an UNCOMMON verdict?
  • Any other solutions to this problem?

Update: For us, this problem went away on its own after a couple of weeks. Evidently Google tweaked their heuristics and stopped classifying our downloads as UNCOMMON. The issue may remain for some people - that Google Safe Browsing is using opaque and proprietary rules to tell the browser whether content is safe.

8
  • 3
    Is there a reason why you're generating ".docx" files? Could you change to a format that's easier for Google to "like"?
    – gnicko
    Jun 12 '20 at 18:54
  • I wish 🙄. MS Word docs is what our customers want.
    – slim
    Jun 13 '20 at 7:13
  • 1
    I'd think that PDFs would be just as easy for your customers to open as word documents and would be far less likely to trigger security warnings. Aug 12 '20 at 14:53
  • 1
    @StephenOstermiller I repeat, MS Word docs is what our customers want -- the documents are a starting point that they then edit to their liking. Regardless, this problem seems to have gone away on its own -- maybe Google tweaked a parameter in their proprietary heuristics.
    – slim
    Aug 15 '20 at 13:06
  • I am curious to know if PDFs would make things better. It is not clear that the heuristic algorithm puts a high weight on the file format. It it were, I would expect the payload to include Content-Type and Content-Disposition headers.
    – Tony
    Aug 20 '20 at 8:03
1

I was encouraged in the comments to provide this as an answer:

  • "Does Google publish anything to explain their criteria for reaching an UNCOMMON verdict?"
    • Not that I was able to find. Their criteria appear to be proprietary and opaque, and are applied on Google servers where we are unable to inspect or control them.
  • "What can we do at the server side to prevent an UNCOMMON verdict?"
    • Nothing proactive that I was able to identify. However the problem ceased to occur a couple of weeks after it first appears. Because the rules are opaque and undocumented, we cannot know whether this is because Google amended their rules, or whether their AI-ish system trained itself to trust our patterns.
  • "Any other solutions to this problem?"
    • We found no solutions other that waiting it out.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.