Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

I have created many very simple pure HTML websites over the years. Most of them appear to work fine most of the time. But there is one recurring problem which I have never quite sorted out involving strange characters.

The scenario goes like this: I create the site. I look at it in my browser, everything appears fine. I may look at it a great many times over the coming weeks or months as I make additions here and there. Perhaps on a variety of browsers on a variety of PC's. Then one day I look at the page and see a random sprinkling of white question marks against dark diamond shapes. These might appear where I had expected to see hyphens or quotes or apostrophes. My immediate thought is that my browser got into some strange state because I was looking at some foreign website with strange characters, but I'm never quite sure. I'm left with that nagging feeling that perhaps half the planet is seeing my website with funny question marks all over it.

So my question is what's going on? What should I do to ensure that as many people as possible around the world can view my text as I originally intended? Should I be using those special html sequences like

£

for all non alphanumeric characters? Should I worry at all?

Edit: Right now I have the problem occurring on this page: http://www.fullreservebanking.com/papers.htm ... part of it looks like this:

enter image description here

I am using FireFox 5 and the character encoding currently appears to be "UNICODE (UTF-8)". I do not remember manually setting the character encoding to anything since installation. I do occasionally look at Japanese websites for work related reasons - though when I do so, I do not manually make any changes to firefox settings.

Edit: Now fixed. Web page altered accordingly.

share|improve this question

2 Answers 2

up vote 3 down vote accepted

These are called gremlins and they are usually caused because whichever program is putting the quotes in is using the actual pretty / curly / smart quotes instead of the proper HTML entities. The fonts don't display right or don't have those characters in them and instead produced the funny symbol.

See this great article from A List Apart on non-quote related typographical oddities in HTML.

The best thing to do would be to create a text processor that runs over the content of your webpage before it is sent to the user (actually, it's best to run this after the content is generated and before it's saved on the server). This processor will do a simple text replacement for those special characters and provide the appropriate HTML entity in its place.

This page from the PHP manual gives excellent code examples in PHP.

share|improve this answer
    
When you say "HTML entities" to you mean those sequences starting with ampersand or do you mean plain old vanilla ASCII characters? Which should I be using? –  Mick Jun 22 '11 at 18:59
1  
HTML entities are the ampersand-starting expressions. E.g. “ is the HTML entity for an opening double-quote (Unicode character 8220 in decimal). See the w3schools page about HTML entities - w3schools.com/html/html_entities.asp. –  dmsnell Jun 22 '11 at 19:04
    
Ok, I see - I should use &# followed by number rather than & followed by a word. Thanks for your help. –  Mick Jun 22 '11 at 19:17
    
You can use either method equally well. For example, “ is identical to &#8220. –  dmsnell Jun 22 '11 at 19:58
1  
@dmsnell: According to the article in the answer - the "number" method is more reliable. –  Mick Jun 22 '11 at 21:39

dmsnell's answer about using HTML entities is fine, but this issue can usually be fixed by making sure you are using the proper UTF-8 throughout the entire page generation and serving to users.

For example, if your data is stored in a database, make sure all the text fields use UTF-8 encoding. In MySQL you will also need to run a query SET NAMES utf8 once before you start fetching data.

PHP handles UTF-8 fine if you are not modifying strings. If you are, you will need to look into using its multi-byte mb_* functions.

On the page itself, add the content type meta tag. It should go right after the <head> tag.

<meta http-equiv="Content-Type" content="text/html; charset=utf-8">

You could also set this as a HTTP header instead.

share|improve this answer
    
That all sounds very well if its a database generated website - but mine is pure HTML created in a text editor. –  Mick Jun 23 '11 at 14:10
1  
In that case, you need to make sure you are creating the content with the correct encoding in the first instance. Make sure you have the right encoding in your text editor - Notepad++ is very good at this. –  DisgruntledGoat Jun 24 '11 at 0:23
    
Interesting... I often use notepad++... what exactly do I need to set in it? –  Mick Jun 24 '11 at 7:38
    
@Mick: OTOH there are options under the Edit menu like "Encode in UTF-8" and "Convert to UTF-8". If your symbols come up as "gremlins" in NP++ then you'll probably want to choose a different encoding option until they display. Once everything looks fine in NP++ then you will likely want to convert to UTF-8. –  DisgruntledGoat Jun 24 '11 at 11:26

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.