0

We switched to HTTPS in our legacy ASP.NET site about a month ago. We had already implemented redirection with HTTP status code 301 'Moved permanently' for outdated aspx resources in the app's Global.asax file before switching to HTTPS, so we added a few more lines to the redirect algorithm in the Application_BeginRequest method to redirect all requests to aspx pages to their HTTPS counterparts. The core part of the current version of this algorithm looks like this ("example.com" is used instead of the real domain):

Sub Application_BeginRequest(ByVal sender As Object, ByVal e As EventArgs)
    Dim host_requested As String = Request.Url.AbsoluteUri

    ' Save the original request string to know whether we need to redirect
    Dim host_to_redirect As String = host_requested

    ' Prepare for analysis - remove the protocol and domain
    For Each sRoot In m_asPossibleRoots
        If host_to_redirect.StartsWith(sRoot, StringComparison.OrdinalIgnoreCase) Then
            host_to_redirect = host_to_redirect.Substring(sRoot.Length)
            Exit For
        End If
    Next

    ' Remove "home/" if any
    Dim sHome As String = "home/"
    If host_to_redirect.StartsWith(sHome, StringComparison.OrdinalIgnoreCase) Then
        host_to_redirect = host_to_redirect.Substring(sHome.Length)
    End If

    ' Redirect old product pages and articles
    If host_to_redirect.StartsWith("outdated-resource-folder1", StringComparison.OrdinalIgnoreCase) Then
        host_to_redirect = "resource1.aspx"
    ElseIf host_to_redirect.StartsWith("outdated-resource-folder2", StringComparison.OrdinalIgnoreCase) Then
        host_to_redirect = "resource2.aspx"
    ' ElseIf
    ' ... processing other outdated resources
    End If

    ' ... other manipulations with host_to_redirect

    ' Add the required protocol and domain
    host_to_redirect = "https://example.com/" + host_to_redirect

    ' If we need to redirect, do it
    If (host_to_redirect <> host_requested) Then
        ' also provides a 301 HTTP status code in the response
        Response.RedirectPermanent(host_to_redirect)
    End If
End Sub

We also added the HTTPS version of our website to Google Webmaster Tools and similar resources working with our website. All works Ok, but recently it has turned out that one of the web-analytics tool we use determines the main protocol of our resource (HTTP or HTTPS) by sending requests to example.com/robots.txt and reading the protocol in the response. Sure, all these requests to robots.txt on our website are still processed as HTTP because we didn't (and couldn't) code redirection for this non-ASP.NET resource in Global.asax.

Having all this, the main question is the following: Do we need to implement HTTPS redirection for robots.txt if we switched to HTTPS in our ASP.NET website? May it affect SEO and similar things? We didn't bother about this as it seems this is not needed for Google & Bing WMT, Google Analytics and similar tools.

If we need to implement HTTPS redirections for robots.txt though, do we need to do this for other non-ASP.NET resources like sitemap.xml and the like? And what is the best method to implement this in our ASP.NET project? As I understand, we can do this only in web.config, but this may mean that we need to move the HTTPS redirection logic part from Global.asax to web.config too. Will this 2-stage redirection logic work seamlessly?

Improve this question

1 Answer 1

Reset to default
1

If you want to run your website over SSL you need to send everything over SSL, otherwise the security can still be compromised. That means all the static files like css, javascript, images, robots.txt and your sitemap.xml need to be loaded over SSL.

The best way to make sure is to use URL Rewrite in IIS. You might have to install it if you don't see it anywhere in IIS Manager. The redirects can be easily configured in the web config:

  <system.webServer>
    <rewrite>
      <rules>
        <rule name="HTTP to HTTPS redirect" stopProcessing="true">
          <match url="(.*)" />
          <conditions trackAllCaptures="false" logicalGrouping="MatchAll">
            <add input="{HTTPS}" pattern="off" ignoreCase="true" />
          </conditions>
          <action type="Redirect" url="https://{HTTP_HOST}/{R:1}" redirectType="Permanent" />
        </rule>
      </rules>
      <!-- Bonus points if you also implement the HSTS Header -->
      <outboundRules>
        <rule name="Add Strict-Transport-Security when HTTPS" enabled="true">
          <match serverVariable="RESPONSE_Strict_Transport_Security" pattern=".*" />
          <conditions>
            <add input="{HTTPS}" pattern="on" ignoreCase="true" />
          </conditions>
          <action type="Rewrite" value="max-age=31536000" />
        </rule>
      </outboundRules>
    </rewrite>
  </system.webServer>

The HSTS header tells your browser not to request any other resource on your site over HTTP and immediately request it over HTTPS. (In case your HTML code still links a resource to HTTP)

Improve this answer
6
  • Will this scheme with HSTS work together with the redirect rules coded in Global.asax (kinda a pipe)?
    – TecMan
    Jul 11, 2017 at 12:38
  • I do understand that we need to pass all site content used to render pages through HTTPS (css, javascript, images), but why do we need to use HTTPS for special text resources like robots.txt and sitemap.xml read by search engine bots only? And we may want to have some resources hosted on our website other users can download with special secret HTTP links not available for the public. Why do we need to provide them through HTTPS only?
    – TecMan
    Jul 11, 2017 at 12:43
  • @TecMan Don't make it difficult for yourself, put all the redirect logic in one place: the web.config file, unless you need an automated way of generating the redirects.
    – ZippyV
    Jul 11, 2017 at 13:19
  • @TecMan What reason do you have to justify the use of HTTP? It's like locking your door but keeping a window open.
    – ZippyV
    Jul 11, 2017 at 13:25
  • 1) Google sees HTTP and HTTPS urls as 2 different sites; 2) You're making it difficult for yourself by creating exceptions to the redirect rules; 3) Cookies will be visible over the unsecured connection potentially compromising security and I probably missed a bunch of other security issues.
    – ZippyV
    Jul 11, 2017 at 13:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.