4

I have created many very simple pure HTML websites over the years. Most of them appear to work fine most of the time. But there is one recurring problem which I have never quite sorted out involving strange characters.

The scenario goes like this: I create the site. I look at it in my browser, everything appears fine. I may look at it a great many times over the coming weeks or months as I make additions here and there. Perhaps on a variety of browsers on a variety of PC's. Then one day I look at the page and see a random sprinkling of white question marks against dark diamond shapes. These might appear where I had expected to see hyphens or quotes or apostrophes. My immediate thought is that my browser got into some strange state because I was looking at some foreign website with strange characters, but I'm never quite sure. I'm left with that nagging feeling that perhaps half the planet is seeing my website with funny question marks all over it.

So my question is what's going on? What should I do to ensure that as many people as possible around the world can view my text as I originally intended? Should I be using those special html sequences like 

£

for all non alphanumeric characters? Should I worry at all?

Edit: Right now I have the problem occurring on this page: http://www.fullreservebanking.com/papers.htm ... part of it looks like this:

enter image description here

I am using FireFox 5 and the character encoding currently appears to be "UNICODE (UTF-8)". I do not remember manually setting the character encoding to anything since installation. I do occasionally look at Japanese websites for work related reasons - though when I do so, I do not manually make any changes to firefox settings.

Edit: Now fixed. Web page altered accordingly.

Improve this question

3 Answers 3

Reset to default
7

These are called gremlins and they are usually caused because whichever program is putting the quotes in is using the actual pretty / curly / smart quotes instead of the proper HTML entities. The fonts don't display right or don't have those characters in them and instead produced the funny symbol.

See this great article from A List Apart on non-quote related typographical oddities in HTML.

The best thing to do would be to create a text processor that runs over the content of your webpage before it is sent to the user (actually, it's best to run this after the content is generated and before it's saved on the server). This processor will do a simple text replacement for those special characters and provide the appropriate HTML entity in its place.

This page from the PHP manual gives excellent code examples in PHP.

Improve this answer
7
  • When you say "HTML entities" to you mean those sequences starting with ampersand or do you mean plain old vanilla ASCII characters? Which should I be using?
    – Mick
    Jun 22, 2011 at 18:59
  • 1
    HTML entities are the ampersand-starting expressions. E.g. “ is the HTML entity for an opening double-quote (Unicode character 8220 in decimal). See the w3schools page about HTML entities - w3schools.com/html/html_entities.asp.
    – dmsnell
    Jun 22, 2011 at 19:04
  • Ok, I see - I should use &# followed by number rather than & followed by a word. Thanks for your help.
    – Mick
    Jun 22, 2011 at 19:17
  • You can use either method equally well. For example, “ is identical to &#8220.
    – dmsnell
    Jun 22, 2011 at 19:58
  • 1
    @dmsnell: According to the article in the answer - the "number" method is more reliable.
    – Mick
    Jun 22, 2011 at 21:39
7

dmsnell's answer about using HTML entities is fine, but this issue can usually be fixed by making sure you are using the proper UTF-8 throughout the entire page generation and serving to users.

For example, if your data is stored in a database, make sure all the text fields use UTF-8 encoding. You should also set the charset when connecting to the database (if using PDO in PHP) or run a query SET NAMES utf8 after connecting, before you start fetching data.

PHP handles UTF-8 fine if you are not modifying strings. If you are, you will need to look into using its multi-byte mb_* functions.

On the page itself, add the content type meta tag. It should go right after the opening <head> tag.

<meta charset="utf-8">

You could also set this as a HTTP header instead.

Improve this answer
4
  • That all sounds very well if its a database generated website - but mine is pure HTML created in a text editor.
    – Mick
    Jun 23, 2011 at 14:10
  • 1
    In that case, you need to make sure you are creating the content with the correct encoding in the first instance. Make sure you have the right encoding in your text editor - Notepad++ is very good at this.
    – DisgruntledGoat
    Jun 24, 2011 at 0:23
  • Interesting... I often use notepad++... what exactly do I need to set in it?
    – Mick
    Jun 24, 2011 at 7:38
  • @Mick: OTOH there are options under the Edit menu like "Encode in UTF-8" and "Convert to UTF-8". If your symbols come up as "gremlins" in NP++ then you'll probably want to choose a different encoding option until they display. Once everything looks fine in NP++ then you will likely want to convert to UTF-8.
    – DisgruntledGoat
    Jun 24, 2011 at 11:26
1

In other words if you are in WordPress and HTML5, only copy proper utf-8 encoded characters over and problem is solved. Just Google "utf-8 list of characters" and copy straight from your browser to your editor(in visual mode). Then the question mark ? wont appear like �

Improve this answer
1
  • BTW they operate like SVG so can be scaled up infinitely without increase in bandwidth. Great for logos.
    – Tim Helmer
    Oct 30, 2017 at 4:31

Not the answer you're looking for? Browse other questions tagged or ask your own question.