Sign up ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

An SEO expert was testing my site, and noticed that my URLs contained the special character :. He said that would create duplicate content, because google would interpret any url containing : as two separate URLs: one with : and one with %3A. Is he right?

share|improve this question
I'm tempted to edit this and put "so called" before "SEO export" (and keep the quotes around "so called"). – paulmorriss Jun 28 '12 at 13:28
Also, next time you ask a question, can you not begin the title with SEO? Thanks – Hamlet Jun 28 '12 at 17:20
I think that he ran a software or website on your site and it told him that. Different search engines behave differently. Regardless, it might be a best practice to avoid : in URL's. – B Seven Jul 4 '12 at 19:03

2 Answers 2

up vote 2 down vote accepted

I suspect this is lies/misinterpretation from your "SEO expert" (such roles do not exist IMO). Essentially %3A and : are exactly the same thing, one is just encoded and means exactly the same thing, anything that reads an URL will know that.

Otherwise you could argue that any non-alphanumeric character could cause duplicate content as they all have an URL encoded entity for someing (eg %2d is -).



Both resolve to the same place, except that - is url encoded in the latter and is what will be honoured by browsers/search engines.

share|improve this answer
well I know that and I told him that this is just the encoding. But he told me that search engines see those as different things – gong Jun 28 '12 at 13:19
Well he's an idiot. SEO experts is such a non-existent profession in my opinion, if were so good at SEO and good become #1 in SERPs for any website you would make much more money doing that then teaching others! – Dunhamzzz Jun 28 '12 at 13:37
"any non-alphanumeric character could cause duplicate content" <- or any character at all, since alphanumeric characters can also be encoded. – DisgruntledGoat Jun 29 '12 at 13:27
@DisgruntledGoat src? How would that even work? Isn't the idea of url encoding is to convert the potentially out-of-chracterset symbols to alhapnumeric? – Dunhamzzz Jun 29 '12 at 14:33
I had to downvote this because it's wrong (except for the "SEO expert" being a liar, that's probably right). Only unreserved characters can be decoded and be equivalent to their percent encoding, and the colon is not one of them. The colon is a special character and, unfortunately, due to a bizarre quirk of the specification, isn't always safe to use unencoded anyway. – Michael Hampton Apr 14 at 6:57

Your "SEO expert" might be a lying bastard, but this probably isn't the reason. He's absolutely right about this. This is a little known edge case in URL construction.

RFC 3986 is the official definition of the URL format and rules on how to encode and decode URL. Any URL parser should be following this as closely as possible to avoid errors and be interoperable with the rest of the Internet. That includes search engines which, if they apply the rules incorrectly, won't actually be able to crawl or index certain resources at all (e.g. they will get a 404 because they screwed up your URL, or your application will misinterpret the URL or query string).

The RFC gives rules on how to do the percent encoding and decoding, which we're all familiar with, but it also explains when to do the encoding and decoding, and to which characters.

Note that search engines normalize URLs (so that they can be compared) but they do not dereference them. Web servers dereference URLs to locate documents and to pass decoded data to your web application. When a URL is normalized, only a subset of percent encoded characters are decoded; when it is dereferenced, all of them are decoded.

In particular it specifies how to compare two URLs for equivalence (all of section 6) and which characters must be percent decoded before doing so (section 6.2.1 and 6.2.2). Here we find out that the only characters to be decoded before comparing URLs for equivalence are the so-called unreserved characters. These are defined (in section 2.3) as "uppercase and lowercase letters, decimal digits, hyphen, period, underscore, and tilde." Percent encoding is meant to prevent browsers, search engines, and the like from misinterpreting special characters in URLs, but since none of the unreserved characters have special meanings in a URL, these can be decoded by anyone at any time.

So, the %3A is not decoded to a colon : before two URLs are compared for equivalence. The colon actually has some unusual rules that apply to its use in the path component of a URL (explained in section 4.2); it cannot appear in the first path component of a relative URL (but is allowed in subsequent components) because it could be confused for a URL scheme.

To construct valid relative URLs with a colon as the first path component, we would either have to encode it some of the time and not others, prefix all such relative URLs with ./, or forgo relative URLs entirely (which is usually what happens, but relative URLs are much more common than you might think).

This part of the URL spec could use some clarification, but given the circumstances I would strongly recommend that if you are going to use colons in your URLs that you always encode them. This removes any possible ambiguity with respect to URL equivalence and ensures that you won't hit this edge case even if you use relative URLs.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.