Can basic HTTP password authentication interfere with CDN access?

Question

I'm doing my best to ask this question, I'm stepping across domains here to cover for another team that is MIA on this task. Please guide me if the question sounds stupid.

We are dealing with a website that is reportedly hosted via VHost on Apache servers. So when we go to blah.oursite.com, many of our static assets actually come via redirect/proxy/something from a CDN that points to our S3 bucket.

So if I try to make a crude visual version of this: User > VHost site > redirect/proxy to CDN > S3

The site and assets for the VHost in this case (in the specific subdomain blah) are protected by what we believe to be .htaccess controlled Basic HTTP password authentication. So when we visit blah.oursite.com we are presented by the browser with a popup requesting username and password. This is by design.

What we're seeing is that the few of our static assets that are hosted on an outside CDN are not able to be loaded on the blah subdomain. If we load www.oursite.com then the assets all load without any problems. Therefore, one of our leading theories, as novices in this particular domain, is that the .htaccess credential protection is interfering with the blah subdomain's ability to access the CDN'ed content.

Does that make any sense? Is that a possibility? What techniques might we use to test this theory (not knowing what sort of configurations are set by the team managing the Apache instances) or otherwise troubleshoot the issue? Are we out of luck without more cooperation from the other team?

I wish I could provide more detail on what is happening at the steps between User > VHost site > redirect/proxy but unfortunately my team doesn't own those steps in the process and the team that does own them is not playing nicely with us on resolving this. Any help or hints to guide or help us find what the issue might be and how to resolve it would be super helpful; I have no experience with server administration.

Answer 1

We dealt with this type of issue recently when setting up a staging copy of one of our websites. We needed to load assets through the cdn to make sure it was setup properly but the cdn was denied by the .htaccess basic password auth just like you described.

To work around this we used Apache's SetEnvIF Directive.

You can match an HTTP Request Header coming from CDN requests, set an environment variable and then allow traffic with that environment variable through the basic password protection.

To do this based on the CDN's User-Agent, your .htaccess file (for Apache 2.4) would look like the following:

SetEnvIf User-Agent ^cdnUserAgentToMatch$ cdn

AuthType Basic
AuthName "Test Site"
AuthUserFile "/path/to/passwd"
Require valid-user
Require env cdn
ErrorDocument 401 "Authorization Required"

This will allow access to either a valid-user or a request with the cdn environment variable. Note: By default, the Require Directives are evaluated as being wrapped in a <RequireAny> tag; so, we don't need to include one even though there are two options.

Since User-Agents can be spoofed, it would probably be best to match against the requesting IP Address when setting the environment variable. Or even better, some CDNs allow you to set a custom HTTP Header which you could put a key in and check against that, like so:

SetEvnIf CustomHeader ^5tfasdoqu7891435$ cdn

Answer 2

Unless the static resources you are loading contain anything that absolutely has to be protected then there should be no need to load them through a proxy. The point of having a CDN is to move content closer to the end user and distribute it over a number of servers so as to increase page load times and decrease the calls being directed to the main server.

Ideally what should be happening here is that your site itself be behind your chosen authentication mechanism (however this is done is up to you) but the static resources on the CDN are not. When you place them into the BasicAuth protected site it is no different than loading them into an unprotected site.

Now depending on the addressing scheme and if https is being used for your site this is what could be bring you undone with loading the static resources, especially if you are accessing the static resources over a standard http connection instead of https as most browsers won't load non https protected content on a site protected by https for security reasons.

You also mentioned in your question that your site is being hosted on a vHost and make the supposition that the static content is pulled in via a proxy or something similar from the CDN. Now while above I point out that this is not the best practice and actually defeats the purpose of using a CDN I will further point out here that it is unlikely that this has been set up in this way as the static content, while being added to the html on your server, would be getting pulled directly from the CDN by the end browser once the HTML page has been loaded.

When issues such as these arise the most common causes I have found have been accessing http resources from a https protected site or attempting to use static resources from CDN's with referrer restrictions (hotlink protection) on sites which are not in the accepted sites list.

If it is the https issue this is easy for you to identify as you just need to check if the site has https:// as the protocol or not and if it does check the HTML code to see if the static resources are being loaded from a https:// address or a http:// address.

If this is a referrer protection issue this is a little bit harder to identify. The easiest way to do it is to load the site in your browser, open up the Google Chrome developer tools and take a look at console at the bottom. If there was an issue with loading the resources it should show here along with the HTTP status code and the actual file which failed to load.