Combining-character (NFD) text never matches its precomposed (NFC) equivalent

[matt-edmondson]

What happens

Fuzzy.Contains/CalculateScore (FuzzySearch/Fuzzy.cs, lines 46-69 and 99-200) compare subject and pattern purely per UTF-16 char, with no Unicode normalization. A precomposed character (e.g. é, U+00E9) and its canonically-equivalent decomposed form (e + combining acute accent, U+0301) are visually and semantically identical text, but are different code-unit sequences, so they're treated as a non-match.

Reproduction

Fuzzy.Contains("café", "café");     // → true  (precomposed vs precomposed)
Fuzzy.Contains("café", "café");    // → false (decomposed vs precomposed — same visible text!)

Why it matters

Decomposed (NFD) text is common in real input — e.g. filenames read from HFS+/APFS-backed filesystems, or text pasted from certain sources, are frequently NFD-normalized. Any consumer using FuzzySearch to match against such filenames or text will silently miss matches for accented/composed characters despite the visible text being identical to the search term.

Suggested fix

Normalize both spans to a common form (e.g. string.Normalize(NormalizationForm.FormC)) before comparing, or explicitly document this as a known limitation if avoiding the extra allocation on the hot path is a priority.

Acceptance criteria

• Fuzzy.Contains (and the scoring overload) treat canonically-equivalent NFC/NFD forms of the same text as matching.
• Add a test covering an NFD-decomposed subject or pattern against its NFC-precomposed counterpart.

[matt-edmondson]

Triage

Category: Bug
Priority: Medium
Suggested assignment: none — no specific team/area indicated; owner is the FuzzySearch matching core (FuzzySearch/Fuzzy.cs)
Possible duplicate: none found — closely related to #71 (lone surrogate matching inside an emoji). Both are Unicode-correctness gaps in the same char-by-char matching loop in Fuzzy.cs (lines 46-69, 99-200), and both are fixed by moving that loop off raw UTF-16 char comparison. Not duplicates — different defects, different acceptance criteria — but they should be scheduled and reviewed together, since fixing either one in isolation means touching the same hot path twice.
Already in progress: no — no open PR in the org touches this

Why Medium: silent missed matches on input a consumer has no reason to suspect is different — NFD is the normal form for filenames on HFS+/APFS, which is a plausible primary use case for a fuzzy matcher. It stays below High because it is confined to accented/composed text and because "document as a known limitation" is a legitimate resolution here, given the allocation cost of normalizing on the hot path.

Suggested next step: this is the higher-value of the two Unicode issues and should be decided first, because the decision (normalize on the hot path vs. document the limitation) determines whether #71 is fixed as a side effect of codepoint-aware iteration or needs its own handling. If normalization is rejected on allocation grounds, measure before settling — a fast path that skips normalization when both inputs are already ASCII would avoid the cost for the common case.

---

Generated by Claude Code