77

My question: When URLs were first designed, why was case-sensitivity made a feature? I ask this because it seems to me (i.e., a layperson) that case-insensitivity would be preferred to prevent needless errors and simplify an already complicated string of text.

Also, is there a real purpose/advantage to having a case-sensitive URL (as opposed to the vast majority of URLs that point to the same page no matter the capitalization)?

Wikipedia, for example, is a website that is sensitive to letter case (except for the first character):

https://en.wikipedia.org/wiki/StAck_Exchange is DOA.

18
  • 13
    You obviously don't run IIS on Windows
    – John Conde
    Feb 23, 2016 at 1:56
  • 58
    I imagine that itscrap.com, expertsexchange, and whorepresents.com would prefer that more people used case-sensitive names. For more, see boredpanda.com/worst-domain-names . Feb 23, 2016 at 8:52
  • 25
    URL's were designed when dinosaurs rendered on Unix systems roamed the Earth, and Unix is case sensitive. Feb 23, 2016 at 10:48
  • 13
    Wikipedia tries to use the correct capitalisation for the subject title and uses redirects for common differences. eg. html, htm and Html all redirect to HTML. But importantly, because of the enormous subject matter, it's possible to have more than one page where the URL only differs by case. For example: Latex and LaTeX
    – MrWhite
    Feb 23, 2016 at 20:28
  • 7
    @edc65 But Kobi states that parts of the URL (notably the path) are case-sensitive - so, doesn't that make the URL (as a whole) case-sensitive?
    – MrWhite
    Feb 23, 2016 at 22:00

10 Answers 10

98

URLs are not case-sensitive, only parts of them.
For example, nothing is case-sensitive in the URL https://google.com,

With reference to RFC 3986 - Uniform Resource Identifier (URI): Generic Syntax

First, from Wikipedia, a URL looks like:

 scheme:[//host[:port]][/]path[?query][#fragment]
 

(I've removed the user:password part because it is not interesting and rarely used)

schemes are case-insensitive

The host subcomponent is case-insensitive.

The path component contains data...

The query component contains non-hierarchical data...

Individual media types may define their own restrictions on or structures within the fragment identifier syntax for specifying different types of subsets, views, or external references

So, the scheme and host are case-insensitive.
The rest of the URL is case-sensitive.

Why is the path case-sensitive?

This seems to be the main question.
It is difficult to answer "why" something was done if it was not documented, but we can make a very good guess.
I've picked very specific quotes from the spec, with emphasis on data.
Let's look at the URL again:

 scheme:[//host[:port]][/]path[?query][#fragment]
 \____________________/\________________________/
        Location                 Data
  • Location - The location has a canonical form, and is case-insensitive. Why? Probably so you could buy a domain name without having to buy thousands of variants.

  • Data - the data is used by the target server, and the application can choose what it means. It wouldn't make any sense to make data case insensitive. The application should have more options, and defining case-insensitivity in the spec will limit these options.
    This is also a useful distinction for HTTPS: the data is encrypted, but the host is visible.

Is it useful?

Case-sensitivity has its pitfalls when it comes to caching and canonical URLs, but it is certainly useful. Some examples:

9
  • 1
    "URLs are not case-sensitive." / "The rest of the URL is case-sensitive." - This would seem to be a contradiction?
    – MrWhite
    Feb 23, 2016 at 15:32
  • 9
    In truth, the scheme defines what to expect in the rest of the URL. http: and related schemes mean that the URL refers to a DNS hostname. DNS was ASCII case-insensitive long before the invention of URLs. See page 55 of ietf.org/rfc/rfc883.txt
    – O. Jones
    Feb 23, 2016 at 15:33
  • 3
    Nicely detailed! I was going from a historical point of view. It was originally the file path that required to be case sensitive only if you were hitting the file system. Otherwise, it was not. But today, things have changed. For example, parameters and CGI did not exist originally. Your answer takes a current day perspective. I had to reward your efforts!! You really dug in on this one! Who knew this would blow-up the way it did?? Cheers!!
    – closetnoc
    Feb 23, 2016 at 20:04
  • 2
    @w3dk: it's a not-very-interesting quirk of terminology, but you could take "case-sensitive" to mean, "changing the case of a character can change the whole", or you could take it to mean, "changing the case of a character always changes the whole". Kobi seems to be asserting the latter, he prefers that case-sensitive should mean "any change in case is significant", which of course is not true of URLs. You prefer the former. It's just a matter of how sensitive they are to case. Feb 24, 2016 at 2:50
  • 2
    @rybo111: If a user types example.com/fOObaR, the spec requires that the server at www.example.com receive a path "/fOObaR" as given; it is silent on the question of whether the server must treat that any differently from "/foOBaR".
    – supercat
    Feb 24, 2016 at 16:33
61

Simple. The OS is case sensitive. Web servers generally do not care unless they have to hit the file system at some point. This is where Linux and other Unix-based operating systems enforce the rules of the file system in which case sensitivity is a major part. This is why IIS has never been case sensitive; because Windows was never case sensitive.

[Update]

There have been some strong arguments in the comments (since deleted) about whether URLs have any relationship with the file system as I have stated. These arguments have become heated. It is extremely short-sighted to believe that there is not a relationship. There absolutely is! Let me explain further.

Application programmers are not generally systems internals programmers. I am not being insulting. They are two separate disciplines and system internals knowledge is not required to write applications when applications can simply make calls to the OS. Since application programmers are not systems internals programmers, bypassing the OS services is not possible. I say this because these are two separate camps and they rarely cross-over. Applications are written to use OS services as a rule. There are rare some exceptions of course.

Back when web servers began to appear, application developers did not attempt to bypass OS services. There were several reasons for this. One, it was not necessary. Two, application programmers generally did not know how to bypass OS services. Three, most OSes were either extremely stable and robust, or extremely simple and light-weight and not worth the cost.

Keep in mind that the early web servers either ran on expensive computers such as the DEC VAX/VMS servers and the Unix of the day (Berkeley and Ultrix as well as others) on main-frame or mid-frame computers, then soon after on light-weight computers such as PCs and Windows 3.1. When more modern search engines began to appear, such as Google in 1997/8, Windows had moved into Windows NT and other OSes such as Novell and Linux had also began to run web servers. Apache was the dominant web server though there were others such as IIS and O'Reilly which were also very popular. None of them at the time bypassed OS services. It is likely that none of the web servers do even today.

Early web servers were quite simple. They still are today. Any request made for a resource via an HTTP request that exists on a hard-drive was/is made by the web server through the OS file system.

File systems are rather simple mechanisms. As a request is made for access to a file, if that file exists, the request is passed to the authorization sub-system and if granted, the original request is satisfied. If the resource does not exist or is not authorized, an exception is thrown by the system. When an application makes a request, a trigger is set and the application waits. When the request is answered, the trigger is thrown and the application processes the request response. It still works that way today. If the application sees that the request has been satisfied it continues, if it has failed, the application executes an error condition within it's code or dies if not handled. Simple.

In the case of a web server, assuming that a URL request for a path/file is made, the web server takes the path/file portion of the URL request (URI) and makes a request to the file system and it is either satisfied or throws an exception. The web server then processes the response. If, for example, the path and file requested is found and access granted by the authorization sub-system, then the web server processes that I/O request as normal. If the file system throws an exception, then the web server returns a 404 error if the file is Not Found or a 403 Forbidden if the reason code is unauthorized.

Since some OSes are case sensitive and file systems of this type require exact matches, the path/file that is requested of the web server must match what exists on the hard drive exactly. The reason for this is simple. Web servers do not guess what you mean. No computer does so without being programmed to. Web servers simply process requests as they receive them. If the path/file portion of the URL request being passed directly to the file system does not match what is on the hard drive, then the file system throws an exception and the web server returns a 404 Not Found error.

It is really that simple folks. It is not rocket science. There is an absolute relationship between the path/file portion of a URL and the file system.

5
  • 1
    I think you argument is flawed. While Berners-Lee didn't have any choice about the case sensitivity of ftp URLs. He got to design http URLs. He could have specified them as US-ASCII only and case insensitive. If there ever were any web servers that just passed the URL path to the file system then they were insecure and the introduction of URL encoding broke compatibility with them. Given that the path is being processed before handing to the OS smashing case would have been easy to implement. Therefore I think we have to regard this as a design decision not an implementation quirk. Feb 29, 2016 at 10:38
  • @WilliamHay This has nothing to do with Berners-Lee or the design of the web. It is about limitations and requirements of the OS. I am a retired systems internals engineer. I worked on these systems at the time. I am telling you exactly why URLs are case sensitive. It is not a guess. It is not an opinion. It is a fact. My answer was intentionally simplified. Of course there are file checks and other processes that can be done prior to issuing any open statement. And Yes(!) web servers are partially insecure still to this day as a result.
    – closetnoc
    Feb 29, 2016 at 17:57
  • Whether URLs are case sensitive has nothing to do with the design of the web? Really? Argument from Authority followed by Argument by Assertion. That web servers pass the path component of a URL more or less directly to an open call is a consequence of the design of URLs not a cause of it. Servers (or smart clients in the case of FTP) could have hidden the case sensitivity of filesystems from the user. That they don't is a design decision. Mar 1, 2016 at 9:41
  • @WilliamHay You need to slow down grass hopper and reread what I have written. I am a retired systems internals engineer writing OS components, protocol stacks and router code for the ARPA-Net, etc. I worked with Apache, O'Reilly, and IIS internals. Your FTP argument does not hold water as at least the major FTP servers remain case sensitive for the same reason. At no time did I say anything about design of URL/URI. At no time did I say web servers passed values without processing. I did say that OS services are commonly used and that the file system requires an exact match to succeed.
    – closetnoc
    Mar 1, 2016 at 15:26
  • @WilliamHay Please understand that you and I are thinking at cross-purposes. All I was saying in my answer is that for some OSes, file system calls are case sensitive by design. Applications that use system calls, and most do, are limited to the enforcement of the OS rules - in this case, case sensitivity. It is not impossible to bypass this rule. In fact this may be somewhat trivial in some cases though not practical. I used to routinely bypass the file system in my work to unscramble hard drives that went kablooie for one reason or another or to analyze database file internals, etc.
    – closetnoc
    Mar 1, 2016 at 15:47
21
  1. URLs claim to be a UNIFORM Resource locator and can point to resources that predate the web. Some of these are case sensitive (eg many ftp servers) and URLs need to be able to represent these resources in a reasonably intuitive fashion.

  2. Case insensitivity requires more work when looking for a match (either in the OS or above it).

  3. If you define URLs as case sensitive individual servers can implement them as case insensitive if they want. The reverse is not true.

  4. Case insensitivity can be non-trivial in international contexts: https://en.wikipedia.org/wiki/Dotted_and_dotless_I . Also RFC1738 allowed for the use of characters outside the ASCII range provided they were encoded but didn't specify a charset. This is fairly important for something calling itself the WORLD wide web. Defining URLs as case insensitive would open up a lot of scope for bugs.

  5. If you are trying to pack a lot of data into a URI (eg a Data URI) you can pack more in if upper and lower case are distinct.

3
  • 1
    I'm pretty sure URLs were historically limited to ASCII. So internationalization is unlikely to be an original reason. The history of Unix being case-sensitive, OTOH, probably played a huge role.
    – derobert
    Feb 23, 2016 at 17:34
  • While only a subset of ASCII can be used unencoded in a URL RFC1738 specifically states characters outside the ASCII range may be used encoded. Without specifying a charset its isn't possible to know which octets represent the same character except for case. Updated. Feb 24, 2016 at 8:33
  • 1
    Re #4: It's actually worse than that. Dotted and dotless I are a demonstration of the more general principle that, even if everything is UTF-8 (or some other UTF), you cannot capitalize or lowercase correctly without knowing the locale to which the text belongs. In the default locale, a capital Latin letter I lowercases to a lowercase Latin letter i, which is wrong in Turkish because it adds a dot (there is no "Turkish capital dotless I" code point; you're meant to use the ASCII code point). Throw in encoding differences, and this goes from "really hard" to "completely intractable."
    – Kevin
    Feb 25, 2016 at 7:55
10

Why wouldn't the URL be case sensitive?

I understand that may look like a provocative (and "devil's advocate") type of rhetorical question, but I think it's useful to consider. The design of HTTP is that a "client", which we commonly call a "web browser", asks the "web server" for data.

There are many, many different web servers that are released. Microsoft has released IIS with Windows Server operating systems (and others, including Windows XP Professional). Unix has heavyweights like nginx and Apache, not to mention smaller offerings like OpenBSD's internal httpd, or thttpd, or lighttpd. Additionally, many network-capable devices have built in web servers that can be used to configure the device, including devices with purposes specific to networks, like routers (including many Wi-Fi access points, and DSL modems) and other devices like printers or UPSs (battery-backed uninterruptable power supply units) which may have network connectivity.

So the question, "Why are URLs case-sensitive?", is asking, "Why do the web servers treat the URL as being case sensitive?" And the actual answer is: they don't all do that. At least one web server, which is fairly popular, is typically NOT case sensitive. (The web server is IIS.)

A key reason for different behavior between different web servers probably boils down to a matter of simplicity. The simple way to make a web server is to do things the same way as how the computer/device's operating system locates files. Many times, web servers locate a file in order to provide a response. Unix was designed around higher end computers, and so Unix provided the desirable functionality of allowing uppercase and lowercase letters. Unix decided to treat uppercase and lowercase as different because, well, they are different. That's the straightforward, natural thing to do. Windows has a history of being case-insensitive due to a desire to support already-created software, and this history goes back to DOS which simply did not support lowercase letters, possibly in an effort to simplify things with less powerful computers that used less memory. Since these operating systems are different, the result is that simply-designed (early versions of) web servers reflect the same differences.

Now, with all that background, here are some specific answers to the specific questions:

When URLs were first designed, why was case-sensitivity made a feature?

Why not? If all standard web servers were case-insensitive, that would indicate that the web servers were following a set of rules specified by the standard. There was simply no rule that says that case needs to be ignored. The reason that there is no rule is simply that there was no reason for there to be such a rule. Why bother to make up unnecessary rules?

I ask this because it seems to me (i.e., a layperson) that case-insensitivity would be preferred to prevent needless errors and simplify an already complicated string of text.

URLs were designed for machines to process. Although a person can type a full URL into an address bar, that wasn't a major part of the intended design. The intended design is that people would follow ("click on") hyperlinks. If average laypeople are doing that, then they really don't care whether the invisible URL is simple or complicated.

Also, is there a real purpose/advantage to having a case-sensitive URL (as opposed to the vast majority of URLs that point to the same page no matter the capitalization)?

The fifth numbered point of William Hay's answer mentions one technical advantage: URLs can be an effective way for a web browser to send a bit of information to a web server, and more information can be included if there are less restrictions, so a case sensitivity restriction would reduce how much information can be included.

However, in many cases, there isn't a super compelling benefit to case sensitivity, which is proven by the fact that IIS typically doesn't bother with it.

In summary, the most compelling reason is likely just simplicity for those who designed the web server software, particularly on a case-sensitive platform like Unix. (HTTP wasn't something that influenced the original design of Unix, since Unix is notably older than HTTP.)

4
  • "A key reason for different behavior between different web browsers probably boils down to a matter of simplicity." - I assume you mean "web servers", rather than "web browsers" here and in a couple of other places?
    – MrWhite
    Feb 23, 2016 at 18:06
  • 2
    Updated. Reviewed every case of "browsers" and made multiple replacements. Thank you for pointing this out so some quality could be improved.
    – TOOGAM
    Feb 24, 2016 at 0:47
  • 1
    I have received several excellent answers to my question, ranging from the historical to the technical. I am hesitant to go against the grain and accept a lower-rated answer, but @TOOGAM's answer was the most helpful to me. This answer is thorough and extensive yet it explains the concept in an uncomplicated, conversational fashion that I can understand. And I think this answer is a good introduction to the more in-depth explanations.
    – Kyle
    Mar 5, 2016 at 6:17
  • The reason Windows has a case-insensitive filesystem is due to it's DOS heritage. MS-DOS started life on computers like the Tandy TRS-80, which used a TV as the display, and did not originally support lower-case letters due to the lack of resolution. Since it couldn't display lower-case, mixed-case wasn't supported. MS-DOS was licensed by IBM to become the original PC-DOS. While the original PC could display lower-case, the filesystem was ported over as-is from MS-DOS.
    – RichardP
    Sep 20, 2020 at 2:29
6

I stole from the blog an Old New Thing the habit of approaching questions of the form "why is it that something is the case?" with the counter-question "what would the world be like, if it were not the case?"

Say I set up a web server to serve myself my document files from a folder so I could read them on the phone when I was out the office. Now, in my documents folder, I have three files, todo.txt, ToDo.txt and TODO.TXT (I know, but it made sense to me when I made the files).

What URL would I like to be able to use, to access these files? I would like to access them in an intuitive way, using http://www.example.com/docs/filename.

Say I have a script which lets me add a contact to my addressbook, which I can also do over the web. How should that take its parameters? Well, I'd like to use it like: http://www.example.com/addcontact.php?name=Tom McHenry von der O'Reilly. But if there were no way for me to specify the name by case, how would I do that?

How would I differentiate the wiki pages for Cat and CAT, Text and TEXT, latex and LaTeX? Disambig pages, I guess, but I prefer just getting the thing I asked for.

But all that feels like it's answering the wrong question, anyway.

The question I think you were really asking is "Why do web servers 404 you just for a case difference, when they are computers, designed to make life simpler, and they are perfectly capable of finding at least the most obvious case-variations in the URL I typed that would work?"

The answer to which is that while some sites have done this (and better, they check for other typos too), nobody's thought it worthwhile to change a webserver's default 404 error page to do that... but maybe they should?

4
  • 1
    Some sites use some kind of mechanism to convert any query to all lowercase or something consistent. In a way, this is smart.
    – closetnoc
    Feb 24, 2016 at 5:53
  • No, they shouldn't. This functionality can be, and often is, added in when it is desirable (e.g., by modules in apache.) To impose this kind of change as default behavior -- or worse, immutable behavior -- would be more disruptive than the relatively rare occasion where someone has to manually type in a URL beyond the host name. For a good example of why not to do this, recall the fiasco when Network Solutions "fixed" non-existent domain errors from public DNS queries.
    – SirNickity
    Feb 25, 2016 at 0:38
  • @SirNickity Nobody was proposing immutability at any level and webserver error pages are configurable on every webserver I've ever used; nobody was suggesting replacing 404 with 30* codes, but rather adding a list of human-clickable suggestion links to the error page; domain names are a very different topic and issue being case-insensitive, and in a different security context; and IIS already automatically "fixes" (by ignoring) case-differences in the path or filename parts of URIs. Feb 26, 2016 at 15:07
  • 1
    Since 1996, Apache has let you do this with mod_speling. It just don't seem to be a very popular thing to do. Unix/Linux people see case insensitivity as the rule, case insensitivity as the exception. Feb 27, 2016 at 23:04
4

Though the above answer is correct & good. I would like to add some more points.

To understand better, One should understand the basic difference between Unix (Linux) Vs Windows server. Unix is case sensitive & Windows is non case sensitive OS.

HTTP protocol was evolved or started getting into implementation around 1990. HTTP protocol was designed by engineers working at CERN institutes, most of those days scientist used Unix machines and not the Windows.

Most of the scientist were familiar with Unix, so they might have been influenced with Unix style file system.

Windows server was released after 2000. much before windows server became popular HTTP protocol was well matured and the spec was complete.

This could be the reason.

5
  • 2
    "Windows server was released after 2000." The Windows NT 3.1 team would have disagreed with you in 1993. NT 3.51 in 1995 was probably when NT started becoming mature and well-established enough to support business-critical server applications.
    – user
    Feb 23, 2016 at 10:08
  • NT 3.51 had the Win 3.1 interface. Windows did not take off really until Windows 95 and it took NT 4.0 to get the same interface. Feb 23, 2016 at 10:51
  • Michael Kjörling, agreed. Let me modify it.
    – Mani
    Feb 23, 2016 at 10:58
  • 1
    @ThorbjørnRavnAndersen In the server market, NT 3.51 was reasonably successful. In the consumer/prosumer market, it took until Windows 2000 (NT 5.0) before the NT line started gaining serious traction.
    – user
    Feb 23, 2016 at 15:17
  • Indeed, the WorldWideWeb was initially developed on Unix-based systems, which have case-sensitive file systems, and most URLs mapped directly to files on the file system. Feb 27, 2016 at 22:57
4

How should one read a "why was it designed this way?" question? Are you asking for a historically-accurate account of the decision-making process, or are you asking "why would anyone design it this way?"?

It's very rarely possible to get a historically-accurate account. Sometimes when decisions are made in standards committees there is a documentary trail of how the debate was conducted, but in the early days of the web decisions were made hastily by a few individuals - in this case probably by TimBL himself - and the rationale is unlikely to have been written down. But TimBL has admitted that he made mistakes in the design of URLs - see http://www.dailymail.co.uk/sciencetech/article-1220286/Sir-Tim-Berners-Lee-admits-forward-slashes-web-address-mistake.html

In the early days URLs mapped very directly to filenames, and the files were generally on Unix-like machines, and Unix-like machines have case-sensitive filenames. So my guess is that it just happened that way for implementation convenience, and usability (for end-users) was never even considered. Again, in the early days the users were all Unix programmers anyway.

1
  • End users were Unix users as well (not necessarily programmers, but high-energy physicists and the like), so they too were accustomed to case insensitivity. Feb 27, 2016 at 23:08
3

This has nothing to do with where you bought your domain, DNS is not case sensitive. But, the file system on the server you are using for hosting is.

This isn't really an issue and it's fairly common on *nix hosts. Just make sure all links you write on your pages are correct and you won't have a problem. To make it easier, I recommend always naming your pages in all lower case then you never need to double check the name when writing a link.

3

Case sensitivity does have value.

If there are 26 letters, each of them with the ability to capitalized, that's 52 characters.

4 characters has the possiblity of 52*52*52*52 combinations, equaling 7311616 combinations.

If you cannot capitalize the characters, the amount of combinations is 26*26*26*26 = 456976

The are over 14 times more combinations for 52 characters than there are for 26. So for storing data, Urls can be shorter and more information can be passed over networks with less data transferred.

This is why you see youtube using URLs like https://www.youtube.com/watch?v=xXxxXxxX

1
  • All of the answers are good, but I wanted to contribute a comment related to Michael's answer here, since it doesn't really rise to level of formal answer. Digital computers recognize only strings of 1/0 (because electronics recognizes on/off states of switches). They don't recognize alphabet case or anything else unless a programmer arranges it. To ignore the codes for different case requires more programming in other words. You lose additional combinatorial richness too, as Michael pointed out. Dec 14, 2022 at 15:03
2

Closetnoc is right about the OS. Some file systems treat the same name with different casing as different files.

Also, is there a real purpose/advantage to having a case-sensitive URL (as opposed to the vast majority of URLs that point to the same page no matter the capitalization)?

Yes. to avoid duplicate content issues.

If you had for example the following URLs:

http://example.com/page-1
http://example.com/Page-1
http://example.com/paGe-1
http://example.com/PAGE-1
http://example.com/pAGE-1

and they all pointed to the exact same page with the exact same content, then you would have duplicate content, and I'm sure if you have a Google search console (webmaster tools) account, Google will indicate this to you.

What I would suggest doing if you are in that situation is to use all lower-case URLs, then redirect the URLs with at least one capital letter in it to the lower case version. So in the list of URLs above, redirect all the URLs to the first URL.

2
  • "Yes. to avoid duplicate content issues." - But the opposite would seem to be true? The fact that URLs can be case-sensitive (and this is how search engines treat them) causes the duplicate content issues you mention. If URLs were universally case-insensitive then there would be no duplicate content issues with differing case. page-1 would be the same as PAGE-1.
    – MrWhite
    Feb 23, 2016 at 15:18
  • I think a poor server configuration is what can cause duplicate content when it comes to casing. For example, the statement RewriteRule ^request-uri$ /targetscript.php [NC] stored in .htaccess would match http://example.com/request-uri and http://example.com/ReQuEsT-Uri because the [NC] indicates that casing doesn't matter when evaluating that one regular expression. Feb 24, 2016 at 0:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.