Character Encoding Problems in Subtitle Files

The problem in one example

A translator delivers a Spanish subtitle file that was correct on their screen. On the client’s player, “canción” shows up as something like “canción”, an apostrophe turns into a box, and a dash becomes a question mark. Nothing was mistyped. The file and the player disagree about how the stored numbers should be turned back into letters.

This is a character encoding problem. It is one of the most common reasons a subtitle file looks fine in one place and broken in another, and it nearly always affects non-English text first, because plain English letters survive almost any encoding.

What encoding means

A computer stores text as numbers. A character encoding is the table that says which number stands for which character. If the file was saved using one table and read using another, the numbers are looked up in the wrong place and you get the wrong characters.

Older encodings were designed for particular languages or regions. Each could represent only a few hundred characters, so a file made for Western European languages would have a different table from one for Cyrillic, Greek or Central European text. The same number could mean a different letter depending on which table was used. Newer, universal encodings cover characters from essentially all writing systems in one table, which is why they are generally preferred today.

The most widely used universal encoding is UTF-8. It stores common English letters in a single byte, as older encodings did, and uses more bytes for other characters. Because of this, a file containing only basic English letters looks identical in many encodings, which hides the problem until accents or symbols appear.

How garbled text happens

Reading with the wrong table. The most common case is a file saved as UTF-8 that a player reads as an older single-byte encoding. Each accented letter, which takes two bytes in UTF-8, is shown as two odd characters. That is where sequences such as “é” in place of “é” come from. This pattern is sometimes called mojibake, a term borrowed from Japanese for text that has come out scrambled.

Saving with the wrong table. A tool may convert text to an older encoding that lacks some characters. Anything missing, such as curly quotation marks, long dashes or letters from another language, is replaced with a question mark or a box. Unlike the first case, this loss usually cannot be undone, because the original character is gone from the file.

Double conversion. A file is converted once incorrectly, then saved, then converted again. The damage compounds and becomes hard to reverse. This often happens when files pass through several people and tools.

Hidden marker confusion. Some files start with a small invisible marker, the byte order mark, that tells software the file is in a universal encoding. Some players need it, and others show it as stray characters at the start of the first cue or fail to read the first line. If the first subtitle looks strange and the rest are fine, this marker is a candidate cause.

Missing characters in the font. Sometimes the encoding is correct but the font used to draw the text lacks the character, and a box or blank appears instead. The file is fine; the display is not. Playing the same file in another player is a quick way to tell the two cases apart.

Clues that help you diagnose

• Pairs of odd characters in place of one accented letter usually mean UTF-8 read as an older encoding.
• Question marks in place of accents or symbols often mean characters were lost on saving.
• Boxes or empty rectangles often mean the font lacks the character, though they can also mean an encoding error.
• Only the first cue is wrong points toward the byte order marker.
• Text that is correct in one player and wrong in another points toward a difference in how the players guess the encoding. Most guess, because plain text files do not always declare theirs.

Reducing the risk

1. Agree on one encoding at the start of the project. UTF-8 is the usual choice for text-based subtitle files, though some broadcast or legacy workflows require something else. Check the receiving system’s requirements.
2. Say which encoding you used when you hand a file over. A sentence in the delivery notes can save a lot of time.
3. Use a text editor that shows and sets the encoding explicitly. Avoid editors that silently convert on save, and when saving, choose the encoding yourself rather than accepting a default.
4. Test early with the hardest text. Include a few lines with accents, quotation marks, dashes, currency signs, and any non-Latin script the project uses. If those survive the whole chain, the rest will too.
5. Avoid copying text through tools that alter it. Word processors and email programs sometimes swap straight quotes for curly ones or replace three dots with a single ellipsis character. That is normally fine in a universal encoding but may break in a limited one.
6. Keep the original file. Before converting, copy the file. If a conversion goes wrong, the original lets you start again.

Languages that need extra care

Languages written in non-Latin alphabets, or with large character sets, depend entirely on a universal encoding or a matching regional one. Scripts that run right to left bring extra issues, such as how punctuation is placed, covered in the article on right-to-left scripts. Languages written without spaces between words raise separate layout questions, covered under word segmentation. These are layout matters, not encoding matters, but they can be confused with one another when something looks wrong.

Also remember that some tools limit characters per line or restrict allowed characters. A broadcast workflow may accept only a defined character set, so check the receiving specification rather than assuming everything typeable will pass.

A short repair routine

If a file arrives garbled, do not edit the visible text by hand yet. First, open it in a tool that lets you choose how to interpret the file, and try UTF-8. If the text looks right, resave it explicitly as UTF-8. If it does not, try other likely encodings for that language. Only when no interpretation works is it likely that characters were lost, and the fix is to ask for the file again from its source. Finally, add a check for odd characters to your quality checklist so the problem is caught before delivery rather than by the viewer.