Unicode normalization.

There are multiple ways to normalize a string, referred to as Normalization Forms. Which one you want depends on what you want to do with the string, but the most common is NFC, which preserves the exact visual appearance of the string. If you don't care about the details, just use normalizeNFC.

import Unicode.Normalize exposing (Form(..), normalize)


normalizeNFC "é"
--> "é"


normalizeNFC (String.fromList [ 'ᄉ', 'ᅥ', 'ᆼ' ])
--> "성"

The different ways to normalize a string are:

• NFC: Canonical decomposition followed by canonical composition.
• NFD: Canonical decomposition without recomposition.
• NFKC: Compatibility decomposition followed by canonical composition.
• NFKD: Compatibility decomposition without recomposition.

Canonical decomposition splits single characters into sequences of equivalent combining characters. Compatibility decomposition extends canonical decomposition by also replacing variants of the same logical character (such as replacing ¼ with 1/4).

Canonical composition undoes the process of canonical decomposition, combining sequences into single characters wherever possible.

For example, the character "ñ" can be represented as a single code point (U+00F1) or as two code points ("n" followed by a combining tilde, U+006E U+0303). The normalization forms handle this differently:

-- NFC composes into a single character
normalize NFC "ñ"  -- (n + combining tilde)
--> "ñ"            -- (single ñ)


-- NFD decomposes into base + combining character
normalize NFD "ñ"  -- (single ñ)
--> "ñ"            -- (n + combining tilde)

The compatibility forms also replace visual variants:

-- NFKC replaces fi ligature with "fi" and composes
normalize NFKC "fiñ"
--> "fiñ"


-- NFKD replaces fi ligature with "fi" and decomposes
normalize NFKD "fiñ"
--> "fiñ"

Normalization Forms

JavaScript-style API

This function mirrors the String.prototype.normalize function from JavaScript, which takes an argument specifying which normalization form to produce.