Character Encoding for Web Pages
HTTP file headers
Hypertext Transfer Protocol (HTTP) is the language used by a Web server and a Web browser to communicate with each other.
Every time a Web server serves
a page to the browser, it adds an invisible "HTTP response header" to
the file. This file header (not to be confused with the <head> section
of a document) is metadata, which means it contains information about the file
that it is serving to the browser. Many servers include character encoding
(the character set used in the document) in this header. In other words, the
server tells the browser how the document
is encoded. (But, what the server says about the encoding
may not be how the documents it serves are actually encoded;
this is determined by your
text
editor when it saves a file.)
Most servers will say the files it serves are either ISO-8859-1 (Western European Latin characters) or the newer Unicode standard, UTF-8. In documents written in English there is little difference between these two standards; most noticeable differences will be in special punctuation. But if you wish to have your Web page display more than Western European languages (perhaps a page that mixes English with Chinese, for example), you cannot use ISO-8859-1; UTF-8 would be the way to go. At any rate, UTF-8 is the current standard for Web pages.
Browsers (also called "user agents") use an "HTTP request header" when requesting a file from the server. In this request header the browser identifies itself with a "user agent string." By keeping track of these user agent strings, Web traffic analysis software can create logs showing information on what percentage of users come to a site with Firefox, Safari, or Internet Explorer, for example.
The character encoding meta tag
Many
servers, but not all, indicate the character encoding in the HTTP response
header. But whether or not your pages are on a server that indicates
the encoding, you
should do so in the document itself, by adding the appropriate character encoding
meta tag to the head section. Assuming the document was encoded in UTF-8, and is using HTML 5, you
simply need to add this to the <head> section:
<meta charset="utf-8">
Because
this tag tells the browser how to interpret the characters used in the document,
it should be the first tag in the <head> section.
The HTML validator needs to know which character set the document is encoded in so it can properly evaluate it. So if your pages are on a server that does not indicate the encoding, it is necessary to add the character encoding meta tag to your HTML or the validator may not be able to properly evaluate the page. Additionally, if your pages are not being served by a Web server (they are being viewed locally, or perhaps they are to be burned onto a CD) the browser will need the character encoding meta tag in your HTML to properly display the page.
In the case where the server indicates the encoding, and you also indicate the encoding with the character encoding meta tag, there is no problem if your tag and the file header from the server indicate the same encoding. But if the server says the encoding is, for example, ISO-8859-1, and your meta tag says UTF-8, the validator will give you a "character encoding mismatch" error. In such a situation the validator assumes the server's encoding is the one to use. In other words, in the event of a conflict between your tag and the server, the server overrides your character encoding meta tag.
How to see a server’s character encoding setting
To see what your server says about the pages it is serving, you need a way to see its invisible file response header. A tool such as Rex Swain's HTTP Viewer will allow you to see this, including the character encoding, if any is indicated. Go there and enter a URL for a server that you wish to check (such as "http://webhawks.org"), then look at the "Receiving header."
If you
see something
like:
Content-Type:·text/html;·charset=UTF-8
then you know the server says the documents it is serving are encoded as
UTF-8.
If you see something like:
Content-Type:·text/html;·charset=iso-8859-1
then you know the server says the documents it is serving are encoded as ISO-8859-1.
If you see something like:
Content-Type:·text/html
then your server is not indicating the character encoding.
Remember that if your server does not indicate the character encoding, it is even more important that your pages contain the character encoding meta tag. This will give the browser the information it needs to properly render your page (otherwise it will have to guess at the encoding, and it may guess wrong), and it will give the HTML validator the information it needs to properly evaluate your page.
Keep everything in synch
To make sure the browser renders your page as you intend (so that special characters such as curly quotes, apostrophes, accented characters, dashes, etc. display properly), and to enable the HTML validator to work, it is important that the server’s character encoding setting, your page’s character encoding meta tag, and the actual character encoding of the document all match.
When you see a Web page that displays strange characters (such as the two odd charaters in the example below), this is an indication of a character encoding mismatch:
It is recommneded that Web pages use the UTF-8 encoding. To ensure that your pages are actually encoded as UTF-8, your text editor must be configured to save your pages as UTF-8, as explained in Text Editors for HTML and Script Editing.
You can decide which character encoding meta tag you use, and you can decide which encoding to use when you save your document, but you may have no control over the character encoding indicated in the server's response header. For example, your pages may be on an old server that still serves as ISO-8859-1, but you may wish to use UTF-8. In such a case it may be possible to override the setting in the server’s HTTP header, if your Web host allows you to use a custom ".htaccess" file. See Setting charset information in .htaccess for information on how to do this.