Fix: resolve AGL underscore ligature glyph names (f_i/f_l/f_f/...)

docs/reqstream/canvas-net-pdf/pdf-document.yaml

@@ -1045,6 +1045,7 @@ sections:
               - PdfDocument_Fonts_Type1_UnrecognizedFontFile3Bytes_ThrowsUnsupportedImageFeatureException
               - PdfDocument_Fonts_Differences_ThinspaceName_ResolvesAndPaintsEmbeddedGlyph
               - PdfDocument_Fonts_Differences_Uni03BCName_ResolvesViaEnrichedEmbeddedFontGlyphMap
+              - PdfDocument_Fonts_Differences_LigatureUnderscoreName_ResolvesViaEnrichedEmbeddedFontGlyphMap
               - PdfDocument_Load_TextType1CDifferencesAglLigaturesFixture_PaintsVisibleGlyphInk
 
           - id: CanvasNetPdf-PdfDocument-Type3FontResolution
@@ -1281,8 +1282,13 @@ sections:
               encodings (throwing UnsupportedImageFeatureException for any other base encoding
               name), and applying an /Encoding/Differences array's code-then-glyph-name
               run-length overrides via a standard Adobe-glyph-name-to-Unicode-codepoint mapping
-              (itself falling back, for a name this mapping does not directly cover, to the Adobe
-              Glyph List's own generic uniXXXX/uXXXX hex-codepoint naming convention),
+              (itself falling back, for a name this mapping does not directly cover, first to the
+              Adobe Glyph List's own generic uniXXXX/uXXXX hex-codepoint naming convention (itself
+              rejecting a value above 0x10FFFF or within the UTF-16 surrogate range
+              0xD800-0xDFFF as unresolved, since neither is a valid Unicode scalar value), then
+              to the AGL's own underscore ligature-naming convention - two or more
+              underscore-separated component names, each itself resolved the same way, whose
+              concatenated characters match one of the mapping's own dedicated ligature names),
               tolerating an unrecognized glyph name (leaving that code at its base-encoding
               mapping) but throwing InvalidDataException for /Differences beginning with a glyph
               name before any starting code.
@@ -1297,7 +1303,13 @@ sections:
               /Differences glyph name is not evidence of document corruption - real-world
               producers routinely emit producer-specific names this library does not itself
               recognize - so that one code is left at its base-encoding mapping instead of
-              rejecting the whole document.
+              rejecting the whole document. The underscore ligature-naming convention is likewise
+              required because real-world PDF producers (for example XeTeX/LuaTeX-derived
+              toolchains) routinely spell a subsetted font's own ligature glyphs as
+              underscore-joined component names (for example f_i for the "fi" ligature) instead
+              of the Adobe Glyph List's own dedicated ligature names, and without recognizing
+              that convention those ligature glyphs render as missing/tofu glyphs even though the
+              embedded font and a dedicated AGL codepoint for the same ligature both exist.
             tests:
               - PdfDocument_Fonts_UnrecognizedEncoding_ThrowsUnsupportedImageFeatureException
               - PdfDocument_Fonts_DefaultEncoding_IsWinAnsiEncoding
@@ -1309,6 +1321,10 @@ sections:
               - PdfDocument_Fonts_Differences_NacuteName_DoesNotThrow
               - PdfDocument_Fonts_Differences_ThinspaceName_ResolvesAndPaintsEmbeddedGlyph
               - PdfDocument_Fonts_Differences_Uni03BCName_ResolvesViaEnrichedEmbeddedFontGlyphMap
+              - PdfDocument_Fonts_Differences_LigatureUnderscoreName_ResolvesViaEnrichedEmbeddedFontGlyphMap
+              - PdfDocument_Fonts_Differences_UnrecognizedUnderscoreName_FallsBackToBaseEncoding
+              - PdfDocument_Fonts_Differences_SurrogateRangeUniHexName_FallsBackToBaseEncoding
+              - PdfDocument_Fonts_Differences_LigatureComponentSurrogateRange_FallsBackToBaseEncoding
               - PdfDocument_Fonts_Differences_NameBeforeStartingCode_ThrowsInvalidDataException
               - PdfDocument_Fonts_StandardEncoding_DiffersFromWinAnsiEncoding
               - PdfDocument_Fonts_StandardEncoding_Absent_DefaultWinAnsiDoesNotPaintQuoteright

docs/verification/canvas-net-pdf/pdf-document.md

@@ -1236,11 +1236,15 @@ trimmed, genuinely real-world `text-type1c-differences-agl-ligatures.pdf` fixtur
 `PdfFixtures\README.md`) - two actual embedded, subsetted Type1C fonts (`Gotham-Bold`/
 `Gotham-Book`) whose own `/Differences` arrays name glyphs (`/uni03BC`, `/thinspace`) this exact
 regression previously left unresolved - and asserts visible glyph ink paints; this fixture's
-content stream also exercises an `/f_f` ligature-glyph name that remains unresolved by design (an
-AGL underscore-ligature decomposition rule, not a `uniXXXX`/`uXXXX` hex name, and explicitly out
-of this fix's scope), so this broad fixture test deliberately only asserts "some ink paints
-somewhere" rather than per-glyph pixel positions - the two precise synthetic tests above already
-isolate the fixed mechanisms exactly.
+content stream also genuinely contains an `/f_f` ligature-glyph name (an AGL underscore-ligature
+decomposition naming the "ff" ligature glyph) against real body text, which this fix also allows
+to resolve via `TryResolveLigatureUnderscoreName` - but this broad fixture test only asserts "some
+ink paints somewhere" on the whole page (a page that already painted ink before this fix, from the
+unrelated text surrounding the ligature), so it cannot by itself distinguish a resolved `/f_f`
+glyph from an unresolved tofu glyph; the dedicated, isolated proof that the mechanism itself works
+is `PdfDocument_Fonts_Differences_LigatureUnderscoreName_ResolvesViaEnrichedEmbeddedFontGlyphMap`
+above, which pixel-asserts the specific glyph position against a synthetic embedded font built
+for exactly that purpose.
 
 #### CanvasNetPdf-PdfDocument-Type3FontResolution: Type3 Fonts Resolve Required Fields, Fail Closed on Malformed Ones
 
@@ -1475,6 +1479,10 @@ dogfooding shape for the 12 pre-existing Liberation files.
 `PdfDocument_Fonts_Differences_UnrecognizedGlyphName_FallsBackToBaseEncoding`,
 `PdfDocument_Fonts_Differences_FfFfiFflLigatureNames_DoNotThrow`,
 `PdfDocument_Fonts_Differences_NacuteName_DoesNotThrow`,
+`PdfDocument_Fonts_Differences_LigatureUnderscoreName_ResolvesViaEnrichedEmbeddedFontGlyphMap`,
+`PdfDocument_Fonts_Differences_UnrecognizedUnderscoreName_FallsBackToBaseEncoding`,
+`PdfDocument_Fonts_Differences_SurrogateRangeUniHexName_FallsBackToBaseEncoding`,
+`PdfDocument_Fonts_Differences_LigatureComponentSurrogateRange_FallsBackToBaseEncoding`,
 `PdfDocument_Fonts_Differences_NameBeforeStartingCode_ThrowsInvalidDataException`,
 `PdfDocument_Fonts_StandardEncoding_DiffersFromWinAnsiEncoding`,
 `PdfDocument_Fonts_StandardEncoding_Absent_DefaultWinAnsiDoesNotPaintQuoteright`
@@ -1510,7 +1518,37 @@ helper `ApplyDifferences` now calls instead of consulting `StandardGlyphNames` d
 this resolves end to end against an actual embedded font (both `ApplyDifferences`'s own codepoint
 resolution and the embedded-font loader's own synthetic-cmap-equivalent glyph-name lookup must
 agree on the same codepoint for a glyph to paint, so those tests exercise the full pipeline, not
-only this requirement's own codepoint-resolution half). As of
+only this requirement's own codepoint-resolution half). A further theory,
+`PdfDocument_Fonts_Differences_LigatureUnderscoreName_ResolvesViaEnrichedEmbeddedFontGlyphMap`,
+proves the AGL's own underscore ligature-naming convention (two or more underscore-separated
+component glyph names, for example `f_i`, `f_l`, `f_f`, `f_f_i`, `f_f_l` - a real-world spelling
+several PDF producers, for example XeTeX/LuaTeX-derived toolchains, use for a subsetted font's own
+ligature glyphs, as distinct from `StandardGlyphNames`'s own dedicated `fi`/`fl`/`ff`/`ffi`/`ffl`
+names) resolves end to end against an actual embedded Type1C font whose own CFF charset literally
+spells its only non-`.notdef` glyph with that exact underscore name: the new
+`TryResolveLigatureUnderscoreName` helper resolves each component recursively (so a component may
+itself use the `uniXXXX`/`uXXXX` hex convention), concatenates the resolved characters, and looks
+the concatenated string up as an AGL glyph name in its own right - `f` + `i` concatenates to `fi`,
+which `StandardGlyphNames` already maps to U+FB01. This is the real-world regression a PDF
+producer's `/Differences` array naming ligature glyphs this way previously rendered as
+missing/tofu glyphs. A companion test,
+`PdfDocument_Fonts_Differences_UnrecognizedUnderscoreName_FallsBackToBaseEncoding`, declares an
+underscore-joined name whose concatenation (`"foobar"`) is not any recognized AGL ligature name,
+asserting it is tolerated identically to any other unrecognized `/Differences` name (falling back
+to the base encoding) rather than throwing or fabricating a codepoint. Two further tests prove
+`TryParseUppercaseHexDigits` also rejects a value in the UTF-16 surrogate range
+(`0xD800`-`0xDFFF`), not only a value above `0x10FFFF`: a lone surrogate is not a valid Unicode
+scalar value on its own, and - unlike every other unresolved codepoint, which is merely tolerated
+
+- would otherwise reach `TryResolveLigatureUnderscoreName`'s `char.ConvertFromUtf32` call and
+throw `ArgumentOutOfRangeException` instead of failing closed.
+`PdfDocument_Fonts_Differences_SurrogateRangeUniHexName_FallsBackToBaseEncoding` declares
+`/uniD800` directly (the generic hex-codepoint path, with no ligature involved), asserting no
+exception and a base-encoding fallback exactly like the above-`0x10FFFF` case;
+`PdfDocument_Fonts_Differences_LigatureComponentSurrogateRange_FallsBackToBaseEncoding` declares
+`/uniD800_i` (a ligature-underscore name whose first component is the surrogate-range value),
+asserting the same no-exception fallback - proving the surrogate rejection is load-bearing for
+the ligature path specifically, not merely a defensive check that happens to also work there. As of
 Phase B, a font explicitly
 declaring `/StandardEncoding` and showing byte code `0x27` (which diverges between the two base
 encodings - `StandardEncoding` maps it to U+2019 "quoteright", `WinAnsiEncoding` maps it to

src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Fonts.cs

@@ -17,6 +17,7 @@
 // cspell:ignore ZapfDingbats Dingbats registerserif registersans copyrightserif copyrightsans
 // cspell:ignore trademarkserif trademarksans radicalex Noto thinspace
 using System.Globalization;
+using System.Text;
 using DemaConsulting.CanvasNet.Codecs;
 using DemaConsulting.CanvasNet.Fonts;
 
@@ -551,18 +552,73 @@ private void ApplyDifferences(PdfObject differences, int[] table) =>
     ///     first consults <see cref="StandardGlyphNames"/> (the common-name subset), then - for a
     ///     name that subset does not itself cover - falls back to the AGL's own generic
     ///     <c>uniXXXX</c>/<c>uXXXX</c> hex-codepoint naming convention (four uppercase hex digits
-    ///     after <c>uni</c>, or four-to-six uppercase hex digits after <c>u</c>), per the
-    ///     published Adobe Glyph List specification. Introduced so a <c>/Differences</c> array
-    ///     naming a glyph this way (for example <c>/uni03BC</c>, seen in real-world subsetted
-    ///     fonts whose own CFF charset uses that exact literal spelling) resolves to the correct
-    ///     codepoint instead of being silently tolerated as unrecognized (see
-    ///     <see cref="ApplyDifferences"/>'s own remarks for that tolerant fallback).
+    ///     after <c>uni</c>, or four-to-six uppercase hex digits after <c>u</c>), then finally to
+    ///     the AGL's own underscore ligature-naming convention (see
+    ///     <see cref="TryResolveLigatureUnderscoreName"/>), per the published Adobe Glyph List
+    ///     specification. Introduced so a <c>/Differences</c> array naming a glyph this way (for
+    ///     example <c>/uni03BC</c>, seen in real-world subsetted fonts whose own CFF charset uses
+    ///     that exact literal spelling) resolves to the correct codepoint instead of being
+    ///     silently tolerated as unrecognized (see <see cref="ApplyDifferences"/>'s own remarks
+    ///     for that tolerant fallback).
     /// </summary>
     /// <param name="name">The <c>/Differences</c> array's glyph name to resolve.</param>
     /// <param name="codepoint">The resolved Unicode codepoint, or <c>0</c> when unresolved.</param>
     /// <returns><see langword="true"/> if <paramref name="name"/> resolved to a codepoint.</returns>
     private static bool TryResolveGlyphNameToCodepoint(string name, out int codepoint) =>
-        StandardGlyphNames.TryGetValue(name, out codepoint) || TryParseAdobeGlyphListHexName(name, out codepoint);
+        StandardGlyphNames.TryGetValue(name, out codepoint) ||
+        TryParseAdobeGlyphListHexName(name, out codepoint) ||
+        TryResolveLigatureUnderscoreName(name, out codepoint);
+
+    /// <summary>
+    ///     Resolves an Adobe-Glyph-List "ligature" glyph name - two or more component glyph names
+    ///     joined by underscores (for example <c>f_i</c>, a common real-world subsetted-font
+    ///     spelling of the "fi" ligature glyph, as distinct from the AGL's own dedicated <c>fi</c>
+    ///     name already covered by <see cref="StandardGlyphNames"/>) - per the published AGL
+    ///     specification's own ligature-naming convention: each underscore-separated component is
+    ///     itself resolved to a Unicode codepoint (recursively, via
+    ///     <see cref="TryResolveGlyphNameToCodepoint"/>, so a component may itself use the
+    ///     <c>uniXXXX</c>/<c>uXXXX</c> hex convention, for example <c>uni0066_uni0069</c>), the
+    ///     resolved characters are concatenated in order, and the resulting string is looked up as
+    ///     an AGL glyph name in its own right - for example <c>f_i</c> concatenates to <c>fi</c>,
+    ///     which <see cref="StandardGlyphNames"/> already maps to the dedicated "LATIN SMALL
+    ///     LIGATURE FI" codepoint <c>U+FB01</c>. Introduced because real-world PDF producers (for
+    ///     example XeTeX/LuaTeX and several PostScript-derived toolchains) routinely name a
+    ///     subsetted font's ligature glyphs this way rather than with the AGL's own dedicated
+    ///     ligature names, leaving a <c>/Differences</c> array entry naming such a glyph otherwise
+    ///     unresolved (and its text rendered as missing/tofu glyphs) even though
+    ///     <see cref="StandardGlyphNames"/> already has a perfectly good codepoint for the exact
+    ///     same ligature under its dedicated name.
+    /// </summary>
+    /// <param name="name">The glyph name to resolve.</param>
+    /// <param name="codepoint">The resolved Unicode codepoint, or <c>0</c> when unresolved.</param>
+    /// <returns>
+    ///     <see langword="true"/> if <paramref name="name"/> has at least two non-empty
+    ///     underscore-separated components, every component itself resolves to a codepoint, and
+    ///     the components' concatenated characters match a name in <see cref="StandardGlyphNames"/>.
+    /// </returns>
+    private static bool TryResolveLigatureUnderscoreName(string name, out int codepoint)
+    {
+        codepoint = 0;
+
+        var components = name.Split('_');
+        if (components.Length < 2)
+        {
+            return false;
+        }
+
+        var concatenated = new StringBuilder();
+        foreach (var component in components)
+        {
+            if (component.Length == 0 || !TryResolveGlyphNameToCodepoint(component, out var componentCodepoint))
+            {
+                return false;
+            }
+
+            concatenated.Append(char.ConvertFromUtf32(componentCodepoint));
+        }
+
+        return StandardGlyphNames.TryGetValue(concatenated.ToString(), out codepoint);
+    }
 
     /// <summary>
     ///     Parses the Adobe Glyph List's generic hex-codepoint glyph-name conventions: <c>uniXXXX</c>
@@ -604,17 +660,25 @@ private static bool TryParseAdobeGlyphListHexName(string name, out int codepoint
     ///     digit <c>uXXXX</c>/.../<c>uXXXXXX</c> convention can spell a six-digit value as large as
     ///     <c>0xFFFFFF</c>, which is not a valid AGL codepoint name at all, so such a name must
     ///     stay unresolved (per <see cref="ApplyDifferences"/>'s own tolerant handling for
-    ///     unrecognized names) rather than being treated as resolved.
+    ///     unrecognized names) rather than being treated as resolved. Likewise rejects a value in
+    ///     the UTF-16 surrogate range (<c>0xD800</c>-<c>0xDFFF</c>): a lone surrogate value is not
+    ///     a valid Unicode scalar value on its own (surrogates only exist in pairs, as a UTF-16
+    ///     encoding detail, never as a standalone codepoint), so resolving one here would hand
+    ///     <see cref="TryResolveLigatureUnderscoreName"/>'s <c>char.ConvertFromUtf32</c> call (and
+    ///     any other codepoint consumer) a value it cannot legally convert, throwing
+    ///     <see cref="ArgumentOutOfRangeException"/> instead of this method's own documented
+    ///     fail-closed "stay unresolved" contract.
     /// </summary>
     /// <param name="digits">The candidate hex digit span to parse.</param>
     /// <param name="value">
     ///     The parsed value, or <c>0</c> when <paramref name="digits"/> contains any
-    ///     non-uppercase-hex character, or when the parsed value exceeds <c>0x10FFFF</c>.
+    ///     non-uppercase-hex character, when the parsed value exceeds <c>0x10FFFF</c>, or when the
+    ///     parsed value falls within the UTF-16 surrogate range (<c>0xD800</c>-<c>0xDFFF</c>).
     /// </param>
     /// <returns>
     ///     <see langword="true"/> if every character in <paramref name="digits"/> was an
-    ///     uppercase hex digit and the parsed value is a valid Unicode codepoint (at most
-    ///     <c>0x10FFFF</c>).
+    ///     uppercase hex digit and the parsed value is a valid Unicode scalar value (at most
+    ///     <c>0x10FFFF</c> and outside the surrogate range).
     /// </returns>
     private static bool TryParseUppercaseHexDigits(ReadOnlySpan<char> digits, out int value)
     {
@@ -628,7 +692,7 @@ private static bool TryParseUppercaseHexDigits(ReadOnlySpan<char> digits, out in
         }
 
         if (!int.TryParse(digits, NumberStyles.HexNumber, CultureInfo.InvariantCulture, out value) ||
-            value > 0x10FFFF)
+            value > 0x10FFFF || value is >= 0xD800 and <= 0xDFFF)
         {
             value = 0;
             return false;