9

Description

If I dig for businessbrief.xyz, I get no results, but the subdomain www returns an address. Entering businessbrief.xyz into any browser's address bar goes to a valid web address.

Data

Note that dig for @ returns no results and dig for www does:

$ dig businessbrief.xyz +short

$ dig www.businessbrief.xyz +short
target.substack-custom-domains.com.
104.18.21.132
104.18.20.132

As an aside, the IPs listed for www are Cloudflare revers proxy IPs.

But if enter this domain into Chrome, Firefox, and Safari, the www record is automatically requested.

We see this in Chrome's Devtools like so, where the first request for businessbrief.xyz is to www.businessbrief.xyz:

enter image description here

I also see the same behavior in Wireshark, where no DNS requests are sent for businessbrief.xyz, but a DNS request is sent for www.businessbrief.xyz.

As some people are not able to replicate this (and in case this website changes), I've attached a webm of this happening in Chrome.

gif of it happening

Question

Why do browsers redirect businessbrief.xyz to https://www.businessbrief.xyz?

EDIT: Related SO question scoped to just firefox: https://stackoverflow.com/questions/10186383/

0

3 Answers 3

22

Browsers generally have heuristics built in to add missing www. from domain names, but it isn't something you should rely on:

  • Browsers typically have a setting to turn off this "www fix up."
  • Even when browsers do add www. it appears they don't so when the URL starts with https:// (like https://example.com), probably for security reasons.
  • Not every browser and search engine bot will implement this feature. It isn't in a specification that I've been able to find.

You should ensure that your non-www domain resolves to a server and redirects appropriately. Relying on clients to automatically do so with their heuristics is going to lead to failures in many cases.


This site explains some of the browser setting that control auto-www:

Firefox

In about:config the setting browser.fixup.alternate.prefix is set to www. by default. You can prevent Firefox from adding the www by setting browser.fixup.alternate.enabled to false.

Chrome

Chrome implements adding www. as part of its "prediction service." You can disable that in settings under "Settings" > "Show advanced settings" and unchecking the option that starts with "Use a prediction service."

Edge and Internet Explorer

It is controlled by a setting in "Internet Options" > "General tab" > "Appearance" > "Languages" where you need to check the setting starting "Do not add 'www'."

Safari

Safari must have a setting for this because some users report an added www. and some do not. However, I have not been able to identify which setting controls the behavior.

1
  • This is a neat feature for end users, but certainly not something to rely on for webmasters
    – iBug
    Dec 9, 2021 at 12:43
2

You asked why the browsers know to do this, and Steven's answer does explain technically why they know (because the appropriate option is turned on), but not why the functionality was added. I will attempt to cover the other part.


In the grand old days of the internet, there were two different types of names: domain names (which may include subdomains), and host names. Hosts which belonged to a domain would have names under that domain.

So, the mit.edu domain would have hosts like trillium, athena, and better-mousetrap within it. It may also have subdomains, like csail.mit.edu, which could have hosts like cgh and vision03 within it. It's also possible to have a host which has the same name as a subdomain, so you could have a host called csail in the mit.edu domain.

Hosts can be named almost arbitrarily. In large networks, the hosts might be named after the user who commonly uses it (e.g. cgh), the department it's doing work for (project Athena, Comp Sci & AI Lab), or the function it performs (computer vision); or it may be given an arbitrary name just to distinguish from other machines (e.g. trillium, better-mousetrap).

Now, to access one of these hosts if you're not part of their domain, you need to use their "Fully-Qualified Domain Name", or FQDN, which would be:

  • trillium.mit.edu.
  • athena.mit.edu.
  • better-mousetrap.mit.edu.
  • cgh.csail.mit.edu.
  • vision03.csail.mit.edu.
  • csail.mit.edu

Any of these hosts may offer one or more services to the network - remote dektop, ssh, telnet, ftp, email, web, gopher, dns, network time, or anything else. On the other hand, any host may also offer no services, and simply act as a client.

Many services are discovered automatically, via dns, like dns itself, as well as email (smtp), and some things like VOIP. But many other services are accessed arbitrarily, by humans, and so humans need to know the name of the hosts which those services are on.

In order to make them easier to remember, a few well-known services, like ftp, eventually started to be used as host names for hosts which only (or primarily) provided that service. Later, when the World Wide Web became popular, www became the well-known hostname which identified the primary webserver in the domain. Other hosts might still offer web services, but commonly the www host name became instrinsically associated with websites.

(There is also the ability to create an alias hostname, so that a host which provides multiple services can be known by multiple names, to make each easier to remember, like having the host trillium be also known by the aliases www and ftp if it was the primary server for those services on the domain - but that doesn't change the core of the discussion.)

As the web became significantly more popular than most other online services, the www association became so much a part of the internet, to the point that for most websites, you could just assume it was there - if I told you to go to yahoo.com or mit.edu, you would probably assume that I meant www.yahoo.com or www.mit.edu. (From memory, historically the latter would have been a bad guess - I vaguely recall that MIT's primary web server used to be located at web.mit.edu, but eventually the www alias was added to assist those who guessed that.)

This is especially true for most websites, where the www host is the only directly-accessed host in the domain for most people.

So people started taking short-cuts, by just saying yahoo.com or mit.edu, because "everybody knew what you meant".

Eventually, we ran into the opposite situation. Now, because many people are used to saying yahoo.com when they meant www.yahoo.com, other people who aren't used to that started typing yahoo.com instead of www.yahoo.com.

Now, it used to be generally tricky to use a second-level domain as a host name. While there is nothing in the DNS rules which stops us from doing so, the difficulty is that the authoritative dns name server for a particular host is dns server for the domain that contains it. We can create a host csail.mit.edu, but it is part of the mit.edu domain, not the csail.mit.edu (sub-)domain, so it's host entry must be defined in mit.edu's name server. Meanwhile the host www.csail.mit.edu is part of the csail.mit.edu (sub-)domain, and is served by that domain's name server.

Now, this is not a big problem when both domains (mit.edu and csail.mit.edu, in this case) are controlled by the same, or cooperating, organisations - we can add the entries in both domains, or contact the owner of the parent domain and ask them to add in the relevant entry for us.

But if we want to put a host at mit.edu, then that host is part of the edu domain, and the appropriate dns name servers for that domain are not controlled by our organisation. So it's harder to get the host name created, and many domains did not do that (like businessbrief.xyz, which belongs in the xyz domain, whereas www.businessbrief.xyz belongs in the businessbrief.xyz domain).

So browser vendors assisted the unwitting users who might type in a bare domain name which did not have a host attached, by subsequently searching for a host called www within that domain.

Now, this may not be the right thing to do in all cases, which led to disagreement about whether to do it at all, which led to the configuration options that Steven posted about.

1
  • "but not why the functionality was added" => I didn't ask this. You are answering a different question. Dec 10, 2021 at 6:57
1

Entering businessbrief.xyz into any browser's address bar goes to a valid web address.

That is not true, aka the plural of anecdote is not data.

Maybe in the few tests you did, with your configuration and your browsers and your plugins. You are not mentioning any parameter of your experiment, what software? what OS? what version? Etc.

But certainly not all browsers do what you observe.

I just tried with Firefox on MacOS, and entering businessbrief.xyz just yields an error message as expected (the domain does not exist in the sense that this specific name does not resolve to an IP address), no "redirection".

Trying with the first online browser testing service I find, same behavior (error): https://www.browserling.com/browse/win/7/ie/11/http%3A%2F%2Fbusinessbrief.xyz

I suspect your problem can be related to your specific DNS configuration. While dig correctly shows there are no IP address for this name, nothing is known about how your browsers resolve names. They could use DNS over HTTPS (no idea for Safari, but Chrome and Firefox do have that option) and hence, depending on the endpoint, get an IP address, or if they ask the OS depending on how it is configured, like if you have a static mapping in /etc/hosts if on Unix or equivalent on other OS, then the name will resolve just for you.

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.