Unicode and LaTeX

pdfTeX Engine (pdfLaTeX)

When using “basic” LaTeX with the pdfTeX engine, enabling UTF-8 input requires simply placing the following commands at the top of your document preamble:

\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}

The exact effects of these commands are as follows:

• \usepackage[T1]{fontenc} sets the output font encoding to T1. A font encoding is a mapping between character codes and glyphs in a font; the font encodings defined by LaTeX are described in [4].

  By default, LaTeX uses the OT1 font encoding, a 7-bit encoding in which accented characters like “ö” are formed by adding an accent glyph to the base letter. T1, by contrast, is an 8-bit encoding supporting widespread European languages in which many letter+accent combinations exist as single glyphs. [3] [14]

  Without this command, characters will be represented using what’s available in OT1, and, as a result, words containing accented characters won’t be correctly hyphenated, copying-and-pasting accented characters from a built PDF won’t work correctly, and inputting a |, <, or > will produce a completely different character in the resulting PDF. [14]

  If one passes a comma-separated list of font encodings to the fontenc package, the last encoding in the list becomes the document’s default encoding, and switching to the other encodings (e.g., in order to enter characters only defined by those encodings) becomes possible using the command sequence \fontencoding{INSERT ENCODING NAME HERE} \selectfont. [6] [13] [9]

• \usepackage[utf8]{inputenc} sets the input encoding for the document source to UTF-8, allowing UTF-8 characters to appear in the input document. Additionally, for each font encoding used in the document, inputenc loads a mapping of UTF-8 characters to commands usable in that font encoding. [1] When paired with the fontenc command above, on recent LaTeX versions, inputenc loads mappings from the files omsenc.dfu, ot1enc.dfu, t1enc.dfu, and ts1enc.dfu [13], located in $TEXDIR/texmf-dist/tex/latex/base in an installed TeX Live distribution; this in turn allows the user to input any of the characters mapped by those files into a document and have the characters be typeset appropriately.

  Without this command, older versions of LaTeX will use a “raw” input encoding in which each input byte is typeset as the glyph in the same position in the current font. [8]

  Beginning with the 2018-04-01 release, LaTeX uses UTF-8 as the default encoding for source files, making \usepackage[utf8]{inputenc} redundant. [8]

Once the above commands are added to your document preamble, you will be able to enter a number of UTF-8 characters directly into your document and have them show up in the built PDF without having to type out their commands. You’ll even be able to write smart quotes (“ and ”) directly instead of typing quotes out as `` and ''.

So instead of writing this:

``My na\"\i{}ve r\'esum\'e is attached.''  --- Se\~nor \TH{}or

you can write this:

“My naïve résumé is attached.”  — Señor Þor

and LaTeX will handle the input correctly.

Note that LaTeX does not support combining characters; input must be in a composed form.

If LaTeX encounters a Unicode character that it doesn’t have a definition for, typesetting will stop with an error message of the form:

! Package inputenc Error: Unicode char ☃ (U+2603)
(inputenc)                not set up for use with LaTeX.

If you want to use a certain character in your document that LaTeX doesn’t recognize, you can use the \DeclareUnicodeCharacter{hexcode}{cmd} command provided by inputenc. Its first argument is the hexadecimal code point of the Unicode character to define, and the second argument is the LaTeX command to execute when the character is encountered. [1] For example:

\usepackage{tikzsymbols}  % provides \Snowman
\DeclareUnicodeCharacter{2603}{\Snowman}
% Now you can put ☃ in your document!

If you don’t want to have to enter characters as codepoints, the \newunicodechar command provided by the newunicodechar package lets you use the character itself instead, [2] allowing us to rewrite the example above as:

\usepackage{newunicodechar}
\usepackage{tikzsymbols}  % provides \Snowman
\newunicodechar{☃}{\Snowman}
% Now you can put ☃ in your document!

As a special case, loading the textcomp package lets you input all of the Unicode characters that can be output with textcomp’s commands; for example, textcomp defines a \textmusicalnote command that produces ♪ (U+266A, EIGHTH NOTE), and so including textcomp in your preamble allows you to write “♪” in your document and have it be treated as the \textmusicalnote command, producing a “♪” in the output.

\documentclass{article}
\usepackage[X2]{fontenc}
\usepackage[utf8]{inputenc}
\begin{document}
Пролетарии всех стран, соединяйтесь!
\end{document}

\documentclass{article}
\usepackage[X2,T1]{fontenc}
\usepackage[utf8]{inputenc}
\begin{document}
“{\fontencoding{X2}\selectfont Пролетарии всех стран, соединяйтесь!}” said
Señor Þor.
\end{document}

\documentclass{article}
% If we don't explicitly load a Cyrillic font encoding, babel emits a
% warning and defaults to loading T2A.
\usepackage[T2A,T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage[russian,english]{babel}
\begin{document}
“\foreignlanguage{russian}{Пролетарии всех стран, соединяйтесь!}” said
Señor Þor.
\end{document}

\documentclass{article}
\usepackage[LGR,T1]{fontenc}
\usepackage[utf8]{inputenc}
\begin{document}
“{\fontencoding{LGR}\selectfont Ἄνδρα μοι ἔννεπε, Μοῦσα, πολύτροπον, ὃς
μάλα πολλὰ}” said Homer.

“Is he talking about me?” wondered Señor Þor.
\end{document}

\documentclass{article}
% No need to explicitly load LGR!
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage[greek,english]{babel}
\begin{document}
“\foreignlanguage{greek}{Ἄνδρα μοι ἔννεπε, Μοῦσα, πολύτροπον, ὃς μάλα
πολλὰ}” said Homer.

“Is he talking about me?” wondered Señor Þor.
\end{document}

\documentclass{article}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{textalpha}
\begin{document}
“Ἄνδρα μοι ἔννεπε, Μοῦσα, πολύτροπον, ὃς μάλα πολλὰ” said Homer.

“Is he talking about me?” wondered Señor Þor.
\end{document}

XeTeX Engine (XeLaTeX) and LuaTeX Engine (LuaLaTeX)

Besides pdfTeX, LaTeX can also run on two major alternative engines:

• The XeTeX engine, on which LaTeX runs as XeLaTeX

• The LuaTeX engine, on which LaTeX runs as LuaLaTeX. This is a TeX engine with an embedded interpreter for the Lua programming language that allows developers to extend the engine by coding in Lua. [12] [15]

Both engines fully support Unicode input and support modern font technologies, including being able to use fonts from the operating system. [16] [12] When it comes to Unicode support, the major differences between pdfLaTeX and XeLaTeX/LuaLaTeX are:

• XeLaTeX and LuaLaTeX documents must always be written in UTF-8, while pdfLaTeX accepts document in various input encodings. [10] [13]

• The fontenc and inputenc commands used in pdfLaTeX should be omitted when working with XeLaTeX/LuaLaTeX; the Unicode engines ignore (and give a warning about) inputenc, while setting fontenc can actually cause some characters (like smart quotes) to not be recognized. Instead, you can just start entering Unicode characters directly into your document without having to include any packages.

• The set of available Unicode characters in XeLaTeX/LuaLaTeX is determined by what characters are defined in the current font. [13] The default font in both XeLaTeX and LuaLaTeX is Latin Modern, a derivative of TeX’s Computer Modern default font that adds many more characters.

• If XeLaTeX encounters a Unicode character that does not exist in the current font, the resulting PDF will show the font’s placeholder character if it has one; if the font has no placeholder character, nothing will be shown. Either way, the .log file will contain a line of the form:

  Missing character: There is no ☃ in font [lmroman10-regular]:mapping=tex-text;!
  

• If LuaLaTeX encounters a Unicode character that does not exist in the current font, the character will be omitted in the resulting PDF. No warning will be emitted or logged.

• \DeclareUnicodeCharacter is not a valid command in XeLaTeX or LuaLaTeX; one must instead write something like:

  \usepackage{tikzsymbols}  % provides \Snowman
  \catcode`☃=\active
  \protected\def ☃{\Snowman}
  

  \newunicodechar can still be used in place of this method, though. [2]

• Being able to write in another alphabet is largely a matter of switching to a font that supports that alphabet. See the fontspec package for how to change fonts in XeLaTeX and LuaLaTeX.

• While neither XeLaTeX nor LuaLaTeX natively supports combining characters, the Lua scripting capabilities in the latter can be used to give combining characters in your source code the desired effect; see <https://tex.stackexchange.com/a/149197> for an example.

References