I'm managing a CMS-based website. The primary language of the CMS is English. Our content is mostly in English, but user postings are sometimes in other languages. The supporting mySQL database of my site is set to UTF-8 encoding.
Until recently, "other language" (e.g. Russian) postings displayed correctly. This in spite of the fact that the CMS declares in the header of each generated page:
<meta http-equiv='Content-Type' content='text/html; charset=iso-8859-1' />
My best guess is that in the past, browsers commonly took this header declaration as advice, and also looked at content to determine the actual encoding. If, for example, UTF-8 encoding was detected, the heading declaration was overridden. That's how "other language" text was correctly displayed. It helps that iso-8859-encoded text will be correctly interpreted by a UTF-8 decoder, that is, iso-8859-1 is effectively a subset of UTF-8.
Question 1: iso-8859-1 is effectively a subset of UTF-8, right?
As far as I know, nothing about the site, the CMS installation, or the server has changed recently. But, of course, browser tech is constantly evolving and new browser releases arrive all the time.
In the past few weeks, "other language" content is showing up as garbage (mojibake) in the latest versions of Safari (5.1) Chrome (15.0.874.81), and Firefox (7.0.1), at least. All indications are that the "auto" text encoding mode of each is now determining the site pages are iso-8859-1 encoded, end-of-story. When I manually set the browser to use UTF-8, the garbage disappears, and all text is correctly rendered, as before.
Question 2: Has the standard/practice changed, so that recent browser releases obey the header declaration of text encoding and won't override it no matter what encoding is actually present?
Question 3: If not, what other factors could account for the sudden appearance of garbage where there was none before? (For example, could a subtle change to Apache made by the hosting provider have this effect?)
I have no idea why the CMS primary distribution is iso-8859-1 encoded. I've looked at the CMS docs and submitted several queries to support channels, but … no answer. The CMS is definitely capable of supporting other languages; many "official" alternates are available, and so far as I've checked, they all declare "utf-8" in the headers of generated pages
There may be some 8859-1 encoding-dependent code deep in the CMS, I suppose. (No, I'm not going to try to find it!) But the existence of a large number of alternative language packs would seem to argue against that.
Question 4: (bonus!) If there is no encoding-dependent code in the CMS, what kind of technical reasons might make the CMS developers reluctant to move their primary distribution to UTF-8?
Question 5: Am I totally missing some point? Am I totally or partially confused about how text encoding works?