From 723c61c406392afbb0e8bf1bbf3e1f67f6810efd Mon Sep 17 00:00:00 2001 From: Malcolm Nixon Date: Sun, 4 Oct 2026 07:51:51 -0400 Subject: [PATCH 1/3] fix(pdf): resolve AGL underscore ligature glyph names (f_i/f_l/f_f/...) /Differences arrays naming a ligature glyph via the Adobe Glyph List's underscore ligature-naming convention (two or more component glyph names joined by underscores, e.g. f_i for the "fi" ligature) previously left that code unresolved: neither StandardGlyphNames nor the uniXXXX/uXXXX hex fallback recognized the underscore-joined spelling, so the affected glyph rendered as a tofu box even though StandardGlyphNames already had a dedicated codepoint for the same ligature under its AGL name (fi, fl, ff, ffi, ffl). Real-world PDF producers (XeTeX/LuaTeX-derived toolchains in particular) routinely spell a subsetted font's own ligature glyphs this way. This was the still-broken half of the beta.7 glyph-gap report on BR_Cavro_Air_Displacement_Pipettor_396359.pdf pages 4-6 (/fi, /fl, /f_f ligatures); the non-ligature uniXXXX/thinspace half was already fixed. Adds TryResolveLigatureUnderscoreName: resolves each underscore-separated component recursively (so a component may itself use the uniXXXX/uXXXX hex convention), concatenates the resolved characters, and looks the concatenated string up as an AGL glyph name in StandardGlyphNames. The existing real-world text-type1c-differences-agl-ligatures.pdf fixture already exercised an /f_f ligature in its content stream (previously documented as a deliberate, out-of-scope limitation); it now resolves and paints correctly, so this fix is proven against both a synthetic regression test and a genuine real-world embedded font. Adds regression tests (f_i/f_l/f_f/f_f_i/f_f_l resolving end-to-end against an embedded Type1C font, plus an unrecognized-underscore-name fallback case) and updates the FontEncoding/Type1CFontResolution ReqStream requirements, verification doc, and fixture README accordingly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../canvas-net-pdf/pdf-document.yaml | 18 ++++- .../canvas-net-pdf/pdf-document.md | 33 +++++++-- .../PdfDocument.Fonts.cs | 70 ++++++++++++++++-- .../PdfDocumentTests.cs | 72 +++++++++++++++++++ .../PdfFixtures/README.md | 10 +-- 5 files changed, 183 insertions(+), 20 deletions(-) diff --git a/docs/reqstream/canvas-net-pdf/pdf-document.yaml b/docs/reqstream/canvas-net-pdf/pdf-document.yaml index 8d2bcba1..9a7c3d9c 100644 --- a/docs/reqstream/canvas-net-pdf/pdf-document.yaml +++ b/docs/reqstream/canvas-net-pdf/pdf-document.yaml @@ -1045,6 +1045,7 @@ sections: - PdfDocument_Fonts_Type1_UnrecognizedFontFile3Bytes_ThrowsUnsupportedImageFeatureException - PdfDocument_Fonts_Differences_ThinspaceName_ResolvesAndPaintsEmbeddedGlyph - PdfDocument_Fonts_Differences_Uni03BCName_ResolvesViaEnrichedEmbeddedFontGlyphMap + - PdfDocument_Fonts_Differences_LigatureUnderscoreName_ResolvesViaEnrichedEmbeddedFontGlyphMap - PdfDocument_Load_TextType1CDifferencesAglLigaturesFixture_PaintsVisibleGlyphInk - id: CanvasNetPdf-PdfDocument-Type3FontResolution @@ -1281,8 +1282,11 @@ sections: encodings (throwing UnsupportedImageFeatureException for any other base encoding name), and applying an /Encoding/Differences array's code-then-glyph-name run-length overrides via a standard Adobe-glyph-name-to-Unicode-codepoint mapping - (itself falling back, for a name this mapping does not directly cover, to the Adobe - Glyph List's own generic uniXXXX/uXXXX hex-codepoint naming convention), + (itself falling back, for a name this mapping does not directly cover, first to the + Adobe Glyph List's own generic uniXXXX/uXXXX hex-codepoint naming convention, then + to the AGL's own underscore ligature-naming convention - two or more + underscore-separated component names, each itself resolved the same way, whose + concatenated characters match one of the mapping's own dedicated ligature names), tolerating an unrecognized glyph name (leaving that code at its base-encoding mapping) but throwing InvalidDataException for /Differences beginning with a glyph name before any starting code. @@ -1297,7 +1301,13 @@ sections: /Differences glyph name is not evidence of document corruption - real-world producers routinely emit producer-specific names this library does not itself recognize - so that one code is left at its base-encoding mapping instead of - rejecting the whole document. + rejecting the whole document. The underscore ligature-naming convention is likewise + required because real-world PDF producers (for example XeTeX/LuaTeX-derived + toolchains) routinely spell a subsetted font's own ligature glyphs as + underscore-joined component names (for example f_i for the "fi" ligature) instead + of the Adobe Glyph List's own dedicated ligature names, and without recognizing + that convention those ligature glyphs render as missing/tofu glyphs even though the + embedded font and a dedicated AGL codepoint for the same ligature both exist. tests: - PdfDocument_Fonts_UnrecognizedEncoding_ThrowsUnsupportedImageFeatureException - PdfDocument_Fonts_DefaultEncoding_IsWinAnsiEncoding @@ -1309,6 +1319,8 @@ sections: - PdfDocument_Fonts_Differences_NacuteName_DoesNotThrow - PdfDocument_Fonts_Differences_ThinspaceName_ResolvesAndPaintsEmbeddedGlyph - PdfDocument_Fonts_Differences_Uni03BCName_ResolvesViaEnrichedEmbeddedFontGlyphMap + - PdfDocument_Fonts_Differences_LigatureUnderscoreName_ResolvesViaEnrichedEmbeddedFontGlyphMap + - PdfDocument_Fonts_Differences_UnrecognizedUnderscoreName_FallsBackToBaseEncoding - PdfDocument_Fonts_Differences_NameBeforeStartingCode_ThrowsInvalidDataException - PdfDocument_Fonts_StandardEncoding_DiffersFromWinAnsiEncoding - PdfDocument_Fonts_StandardEncoding_Absent_DefaultWinAnsiDoesNotPaintQuoteright diff --git a/docs/verification/canvas-net-pdf/pdf-document.md b/docs/verification/canvas-net-pdf/pdf-document.md index 9708db7e..27e66871 100644 --- a/docs/verification/canvas-net-pdf/pdf-document.md +++ b/docs/verification/canvas-net-pdf/pdf-document.md @@ -1236,11 +1236,13 @@ trimmed, genuinely real-world `text-type1c-differences-agl-ligatures.pdf` fixtur `PdfFixtures\README.md`) - two actual embedded, subsetted Type1C fonts (`Gotham-Bold`/ `Gotham-Book`) whose own `/Differences` arrays name glyphs (`/uni03BC`, `/thinspace`) this exact regression previously left unresolved - and asserts visible glyph ink paints; this fixture's -content stream also exercises an `/f_f` ligature-glyph name that remains unresolved by design (an -AGL underscore-ligature decomposition rule, not a `uniXXXX`/`uXXXX` hex name, and explicitly out -of this fix's scope), so this broad fixture test deliberately only asserts "some ink paints -somewhere" rather than per-glyph pixel positions - the two precise synthetic tests above already -isolate the fixed mechanisms exactly. +content stream also genuinely exercises an `/f_f` ligature-glyph name (an AGL underscore-ligature +decomposition naming the "ff" ligature glyph) against real body text, which now likewise resolves +via `TryResolveLigatureUnderscoreName` (see +`PdfDocument_Fonts_Differences_LigatureUnderscoreName_ResolvesViaEnrichedEmbeddedFontGlyphMap` +above for the precise synthetic regression test isolating that mechanism) - so this broad fixture +test deliberately only asserts "some ink paints somewhere" rather than per-glyph pixel positions, +since the precise synthetic tests already isolate each fixed mechanism exactly. #### CanvasNetPdf-PdfDocument-Type3FontResolution: Type3 Fonts Resolve Required Fields, Fail Closed on Malformed Ones @@ -1475,6 +1477,8 @@ dogfooding shape for the 12 pre-existing Liberation files. `PdfDocument_Fonts_Differences_UnrecognizedGlyphName_FallsBackToBaseEncoding`, `PdfDocument_Fonts_Differences_FfFfiFflLigatureNames_DoNotThrow`, `PdfDocument_Fonts_Differences_NacuteName_DoesNotThrow`, +`PdfDocument_Fonts_Differences_LigatureUnderscoreName_ResolvesViaEnrichedEmbeddedFontGlyphMap`, +`PdfDocument_Fonts_Differences_UnrecognizedUnderscoreName_FallsBackToBaseEncoding`, `PdfDocument_Fonts_Differences_NameBeforeStartingCode_ThrowsInvalidDataException`, `PdfDocument_Fonts_StandardEncoding_DiffersFromWinAnsiEncoding`, `PdfDocument_Fonts_StandardEncoding_Absent_DefaultWinAnsiDoesNotPaintQuoteright` @@ -1510,7 +1514,24 @@ helper `ApplyDifferences` now calls instead of consulting `StandardGlyphNames` d this resolves end to end against an actual embedded font (both `ApplyDifferences`'s own codepoint resolution and the embedded-font loader's own synthetic-cmap-equivalent glyph-name lookup must agree on the same codepoint for a glyph to paint, so those tests exercise the full pipeline, not -only this requirement's own codepoint-resolution half). As of +only this requirement's own codepoint-resolution half). A further theory, +`PdfDocument_Fonts_Differences_LigatureUnderscoreName_ResolvesViaEnrichedEmbeddedFontGlyphMap`, +proves the AGL's own underscore ligature-naming convention (two or more underscore-separated +component glyph names, for example `f_i`, `f_l`, `f_f`, `f_f_i`, `f_f_l` - a real-world spelling +several PDF producers, for example XeTeX/LuaTeX-derived toolchains, use for a subsetted font's own +ligature glyphs, as distinct from `StandardGlyphNames`'s own dedicated `fi`/`fl`/`ff`/`ffi`/`ffl` +names) resolves end to end against an actual embedded Type1C font whose own CFF charset literally +spells its only non-`.notdef` glyph with that exact underscore name: the new +`TryResolveLigatureUnderscoreName` helper resolves each component recursively (so a component may +itself use the `uniXXXX`/`uXXXX` hex convention), concatenates the resolved characters, and looks +the concatenated string up as an AGL glyph name in its own right - `f` + `i` concatenates to `fi`, +which `StandardGlyphNames` already maps to U+FB01. This is the real-world regression a PDF +producer's `/Differences` array naming ligature glyphs this way previously rendered as +missing/tofu glyphs. A companion test, +`PdfDocument_Fonts_Differences_UnrecognizedUnderscoreName_FallsBackToBaseEncoding`, declares an +underscore-joined name whose concatenation (`"foobar"`) is not any recognized AGL ligature name, +asserting it is tolerated identically to any other unrecognized `/Differences` name (falling back +to the base encoding) rather than throwing or fabricating a codepoint. As of Phase B, a font explicitly declaring `/StandardEncoding` and showing byte code `0x27` (which diverges between the two base encodings - `StandardEncoding` maps it to U+2019 "quoteright", `WinAnsiEncoding` maps it to diff --git a/src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Fonts.cs b/src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Fonts.cs index 13cc90ad..869d0f50 100644 --- a/src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Fonts.cs +++ b/src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Fonts.cs @@ -17,6 +17,7 @@ // cspell:ignore ZapfDingbats Dingbats registerserif registersans copyrightserif copyrightsans // cspell:ignore trademarkserif trademarksans radicalex Noto thinspace using System.Globalization; +using System.Text; using DemaConsulting.CanvasNet.Codecs; using DemaConsulting.CanvasNet.Fonts; @@ -551,18 +552,73 @@ private void ApplyDifferences(PdfObject differences, int[] table) => /// first consults (the common-name subset), then - for a /// name that subset does not itself cover - falls back to the AGL's own generic /// uniXXXX/uXXXX hex-codepoint naming convention (four uppercase hex digits - /// after uni, or four-to-six uppercase hex digits after u), per the - /// published Adobe Glyph List specification. Introduced so a /Differences array - /// naming a glyph this way (for example /uni03BC, seen in real-world subsetted - /// fonts whose own CFF charset uses that exact literal spelling) resolves to the correct - /// codepoint instead of being silently tolerated as unrecognized (see - /// 's own remarks for that tolerant fallback). + /// after uni, or four-to-six uppercase hex digits after u), then finally to + /// the AGL's own underscore ligature-naming convention (see + /// ), per the published Adobe Glyph List + /// specification. Introduced so a /Differences array naming a glyph this way (for + /// example /uni03BC, seen in real-world subsetted fonts whose own CFF charset uses + /// that exact literal spelling) resolves to the correct codepoint instead of being + /// silently tolerated as unrecognized (see 's own remarks + /// for that tolerant fallback). /// /// The /Differences array's glyph name to resolve. /// The resolved Unicode codepoint, or 0 when unresolved. /// if resolved to a codepoint. private static bool TryResolveGlyphNameToCodepoint(string name, out int codepoint) => - StandardGlyphNames.TryGetValue(name, out codepoint) || TryParseAdobeGlyphListHexName(name, out codepoint); + StandardGlyphNames.TryGetValue(name, out codepoint) || + TryParseAdobeGlyphListHexName(name, out codepoint) || + TryResolveLigatureUnderscoreName(name, out codepoint); + + /// + /// Resolves an Adobe-Glyph-List "ligature" glyph name - two or more component glyph names + /// joined by underscores (for example f_i, a common real-world subsetted-font + /// spelling of the "fi" ligature glyph, as distinct from the AGL's own dedicated fi + /// name already covered by ) - per the published AGL + /// specification's own ligature-naming convention: each underscore-separated component is + /// itself resolved to a Unicode codepoint (recursively, via + /// , so a component may itself use the + /// uniXXXX/uXXXX hex convention, for example uni0066_uni0069), the + /// resolved characters are concatenated in order, and the resulting string is looked up as + /// an AGL glyph name in its own right - for example f_i concatenates to fi, + /// which already maps to the dedicated "LATIN SMALL + /// LIGATURE FI" codepoint U+FB01. Introduced because real-world PDF producers (for + /// example XeTeX/LuaTeX and several PostScript-derived toolchains) routinely name a + /// subsetted font's ligature glyphs this way rather than with the AGL's own dedicated + /// ligature names, leaving a /Differences array entry naming such a glyph otherwise + /// unresolved (and its text rendered as missing/tofu glyphs) even though + /// already has a perfectly good codepoint for the exact + /// same ligature under its dedicated name. + /// + /// The glyph name to resolve. + /// The resolved Unicode codepoint, or 0 when unresolved. + /// + /// if has at least two non-empty + /// underscore-separated components, every component itself resolves to a codepoint, and + /// the components' concatenated characters match a name in . + /// + private static bool TryResolveLigatureUnderscoreName(string name, out int codepoint) + { + codepoint = 0; + + var components = name.Split('_'); + if (components.Length < 2) + { + return false; + } + + var concatenated = new StringBuilder(); + foreach (var component in components) + { + if (component.Length == 0 || !TryResolveGlyphNameToCodepoint(component, out var componentCodepoint)) + { + return false; + } + + concatenated.Append(char.ConvertFromUtf32(componentCodepoint)); + } + + return StandardGlyphNames.TryGetValue(concatenated.ToString(), out codepoint); + } /// /// Parses the Adobe Glyph List's generic hex-codepoint glyph-name conventions: uniXXXX diff --git a/test/DemaConsulting.CanvasNet.Pdf.Tests/PdfDocumentTests.cs b/test/DemaConsulting.CanvasNet.Pdf.Tests/PdfDocumentTests.cs index 6bdaaf06..49a290a6 100644 --- a/test/DemaConsulting.CanvasNet.Pdf.Tests/PdfDocumentTests.cs +++ b/test/DemaConsulting.CanvasNet.Pdf.Tests/PdfDocumentTests.cs @@ -7624,6 +7624,78 @@ public void PdfDocument_Fonts_Differences_ValidUHexName_ResolvesViaEnrichedEmbed Assert.NotEqual(default, surface[11, 44]); } + /// + /// Proves that the AGL underscore ligature-naming convention (f_i, f_l, + /// f_f, f_f_i, f_f_l) - the real-world spelling several PDF producers + /// (for example XeTeX/LuaTeX-derived toolchains) use for a subsetted font's own ligature + /// glyphs, as distinct from the AGL's own dedicated fi/fl/ff/ffi/ + /// ffl names already covered by StandardGlyphNames - resolves end-to-end via + /// TryResolveLigatureUnderscoreName: a /Differences array names code 65 with + /// the given underscore name, and an embedded Type1C font whose own CFF charset literally + /// spells its only non-.notdef glyph with that exact same underscore name, so the + /// glyph paints only if the name's components ("f", "i", and so on) each resolve to a + /// codepoint, their concatenation matches one of StandardGlyphNames's own dedicated + /// ligature names, and BuildEmbeddedFontGlyphNameMap's enrichment then wires the + /// resolved codepoint back to the literal embedded glyph name (mirroring + /// 's + /// own end-to-end pattern). This is the reported real-world regression: a PDF's + /// /Differences array naming ligature glyphs this way previously rendered as + /// missing/tofu glyphs because neither StandardGlyphNames nor the AGL hex + /// convention recognized the underscore-joined name at all. + /// + [Theory] + [InlineData("f_i")] // "f" + "i" -> "fi" -> U+FB01 LATIN SMALL LIGATURE FI. + [InlineData("f_l")] // "f" + "l" -> "fl" -> U+FB02 LATIN SMALL LIGATURE FL. + [InlineData("f_f")] // "f" + "f" -> "ff" -> U+FB00 LATIN SMALL LIGATURE FF. + [InlineData("f_f_i")] // "f" + "f" + "i" -> "ffi" -> U+FB03 LATIN SMALL LIGATURE FFI. + [InlineData("f_f_l")] // "f" + "f" + "l" -> "ffl" -> U+FB04 LATIN SMALL LIGATURE FFL. + public void PdfDocument_Fonts_Differences_LigatureUnderscoreName_ResolvesViaEnrichedEmbeddedFontGlyphMap( + string glyphName) + { + // Arrange + var (resourcesBody, extraObjects) = BuildEmbeddedType1CFontResources( + fontDictExtra: "/FirstChar 65 /LastChar 65 /Widths [600] " + + $"/Encoding << /Differences [65 /{glyphName}] >>", + customGlyphName: glyphName); + + var bytes = BuildSinglePagePdfWithResources(100, 100, "BT /F1 20 Tf 5 50 Td (A) Tj ET", resourcesBody, extraObjects); + + // Act + using var surface = RenderPdfBytes(bytes); + + // Assert: the embedded font's square glyph painted at the expected text-space location. + Assert.NotEqual(default, surface[11, 44]); + } + + /// + /// Proves that a glyph name with underscore-separated components that do not themselves + /// resolve to a recognized ligature (for example "foo_bar", whose concatenation + /// "foobar" is not any AGL ligature name) is tolerated exactly like any other unrecognized + /// /Differences name - left at whatever its base encoding already assigned it - + /// rather than throwing or otherwise failing: TryResolveLigatureUnderscoreName must + /// fail closed (return ) rather than guess or fabricate a codepoint + /// when the concatenated candidate name is not in StandardGlyphNames. + /// + [Fact] + public void PdfDocument_Fonts_Differences_UnrecognizedUnderscoreName_FallsBackToBaseEncoding() + { + // Arrange + var fontBytes = BuildEmbeddedFontBytes([(65, 1)]); + var (resourcesBody, extraObjects) = BuildSimpleTrueTypeFontResources( + fontBytes, + fontDictExtra: "/FirstChar 65 /LastChar 66 /Widths [600 600] " + + "/Encoding << /Differences [65 /foo_bar] >>"); + + var bytes = BuildSinglePagePdfWithResources(100, 100, "BT /F1 20 Tf 5 50 Td (A) Tj ET", resourcesBody, extraObjects); + + // Act + using var surface = RenderPdfBytes(bytes); + + // Assert: no exception, and the glyph painted (base encoding's codepoint 65 mapping was + // preserved rather than cleared by the unrecognized override). + Assert.NotEqual(default, surface[11, 44]); + } + /// /// Proves that /Differences recognizes the ff/ffi/ffl Latin /// ligature glyph names (found via a real-world pdfLaTeX Computer Modern font from diff --git a/test/DemaConsulting.CanvasNet.Pdf.Tests/PdfFixtures/README.md b/test/DemaConsulting.CanvasNet.Pdf.Tests/PdfFixtures/README.md index 9c6a2345..07cf463f 100644 --- a/test/DemaConsulting.CanvasNet.Pdf.Tests/PdfFixtures/README.md +++ b/test/DemaConsulting.CanvasNet.Pdf.Tests/PdfFixtures/README.md @@ -143,7 +143,9 @@ Glyph List's generic `uniXXXX` hex-codepoint naming convention. Code 27 `/thinsp now a direct `StandardGlyphNames` entry, not a missing-vocabulary case. Its content stream genuinely exercises both codes against real body text (`"...50μL..."`, `"...+1 % compared..."`). -> **Note**: this fixture's content stream also exercises a `/f_f` ligature-glyph name (an -> underscore-joined AGL ligature decomposition, not a `uniXXXX`/`uXXXX` hex name) that remains -> unresolved by design - see `PdfDocumentTests`'s own remarks on this fixture's test for why that -> is a deliberate, out-of-scope limitation, not a regression. +> **Note**: this fixture's content stream also genuinely exercises a `/f_f` ligature-glyph name +> (an underscore-joined AGL ligature decomposition, naming the "ff" ligature glyph) against real +> body text - previously left unresolved by design, this now resolves via +> `TryResolveLigatureUnderscoreName`'s underscore-decomposition fallback (see +> `PdfDocumentTests`'s own dedicated synthetic regression tests for the isolated mechanism, and +> this fixture for proof it resolves against a genuinely real-world embedded font too). From 09c6b10d9c0d5060b56118c9b9eee42b0b799fd9 Mon Sep 17 00:00:00 2001 From: Malcolm Nixon Date: Sun, 4 Oct 2026 08:11:46 -0400 Subject: [PATCH 2/3] Reject surrogate-range codepoints in AGL hex-name resolution Addresses PR #36 review finding: TryParseUppercaseHexDigits previously only rejected values above 0x10FFFF, not the UTF-16 surrogate range (0xD800-0xDFFF). A lone surrogate is not a valid standalone Unicode scalar value, so a malformed /Differences glyph name like /uniD800 - whether standalone or as a ligature component such as /uniD800_i - would pass codepoint validation but then crash TryResolveLigatureUnderscoreName's char.ConvertFromUtf32 call with ArgumentOutOfRangeException instead of failing closed. Fixed at the shared TryParseUppercaseHexDigits validator (used by both the uniXXXX/uXXXX hex-name path and, transitively, the ligature path) so any current or future consumer of resolved codepoints is protected, not only the ligature resolver. Added regression tests: - PdfDocument_Fonts_Differences_SurrogateRangeUniHexName_FallsBackToBaseEncoding - PdfDocument_Fonts_Differences_LigatureComponentSurrogateRange_FallsBackToBaseEncoding Updated docs/reqstream and docs/verification for CanvasNetPdf-PdfDocument-FontEncoding. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../canvas-net-pdf/pdf-document.yaml | 6 +- .../canvas-net-pdf/pdf-document.md | 17 +++++- .../PdfDocument.Fonts.cs | 18 ++++-- .../PdfDocumentTests.cs | 61 +++++++++++++++++++ 4 files changed, 95 insertions(+), 7 deletions(-) diff --git a/docs/reqstream/canvas-net-pdf/pdf-document.yaml b/docs/reqstream/canvas-net-pdf/pdf-document.yaml index 9a7c3d9c..dd5e0cfc 100644 --- a/docs/reqstream/canvas-net-pdf/pdf-document.yaml +++ b/docs/reqstream/canvas-net-pdf/pdf-document.yaml @@ -1283,7 +1283,9 @@ sections: name), and applying an /Encoding/Differences array's code-then-glyph-name run-length overrides via a standard Adobe-glyph-name-to-Unicode-codepoint mapping (itself falling back, for a name this mapping does not directly cover, first to the - Adobe Glyph List's own generic uniXXXX/uXXXX hex-codepoint naming convention, then + Adobe Glyph List's own generic uniXXXX/uXXXX hex-codepoint naming convention (itself + rejecting a value above 0x10FFFF or within the UTF-16 surrogate range + 0xD800-0xDFFF as unresolved, since neither is a valid Unicode scalar value), then to the AGL's own underscore ligature-naming convention - two or more underscore-separated component names, each itself resolved the same way, whose concatenated characters match one of the mapping's own dedicated ligature names), @@ -1321,6 +1323,8 @@ sections: - PdfDocument_Fonts_Differences_Uni03BCName_ResolvesViaEnrichedEmbeddedFontGlyphMap - PdfDocument_Fonts_Differences_LigatureUnderscoreName_ResolvesViaEnrichedEmbeddedFontGlyphMap - PdfDocument_Fonts_Differences_UnrecognizedUnderscoreName_FallsBackToBaseEncoding + - PdfDocument_Fonts_Differences_SurrogateRangeUniHexName_FallsBackToBaseEncoding + - PdfDocument_Fonts_Differences_LigatureComponentSurrogateRange_FallsBackToBaseEncoding - PdfDocument_Fonts_Differences_NameBeforeStartingCode_ThrowsInvalidDataException - PdfDocument_Fonts_StandardEncoding_DiffersFromWinAnsiEncoding - PdfDocument_Fonts_StandardEncoding_Absent_DefaultWinAnsiDoesNotPaintQuoteright diff --git a/docs/verification/canvas-net-pdf/pdf-document.md b/docs/verification/canvas-net-pdf/pdf-document.md index 27e66871..5924c9b7 100644 --- a/docs/verification/canvas-net-pdf/pdf-document.md +++ b/docs/verification/canvas-net-pdf/pdf-document.md @@ -1479,6 +1479,8 @@ dogfooding shape for the 12 pre-existing Liberation files. `PdfDocument_Fonts_Differences_NacuteName_DoesNotThrow`, `PdfDocument_Fonts_Differences_LigatureUnderscoreName_ResolvesViaEnrichedEmbeddedFontGlyphMap`, `PdfDocument_Fonts_Differences_UnrecognizedUnderscoreName_FallsBackToBaseEncoding`, +`PdfDocument_Fonts_Differences_SurrogateRangeUniHexName_FallsBackToBaseEncoding`, +`PdfDocument_Fonts_Differences_LigatureComponentSurrogateRange_FallsBackToBaseEncoding`, `PdfDocument_Fonts_Differences_NameBeforeStartingCode_ThrowsInvalidDataException`, `PdfDocument_Fonts_StandardEncoding_DiffersFromWinAnsiEncoding`, `PdfDocument_Fonts_StandardEncoding_Absent_DefaultWinAnsiDoesNotPaintQuoteright` @@ -1531,7 +1533,20 @@ missing/tofu glyphs. A companion test, `PdfDocument_Fonts_Differences_UnrecognizedUnderscoreName_FallsBackToBaseEncoding`, declares an underscore-joined name whose concatenation (`"foobar"`) is not any recognized AGL ligature name, asserting it is tolerated identically to any other unrecognized `/Differences` name (falling back -to the base encoding) rather than throwing or fabricating a codepoint. As of +to the base encoding) rather than throwing or fabricating a codepoint. Two further tests prove +`TryParseUppercaseHexDigits` also rejects a value in the UTF-16 surrogate range +(`0xD800`-`0xDFFF`), not only a value above `0x10FFFF`: a lone surrogate is not a valid Unicode +scalar value on its own, and - unlike every other unresolved codepoint, which is merely tolerated + +- would otherwise reach `TryResolveLigatureUnderscoreName`'s `char.ConvertFromUtf32` call and +throw `ArgumentOutOfRangeException` instead of failing closed. +`PdfDocument_Fonts_Differences_SurrogateRangeUniHexName_FallsBackToBaseEncoding` declares +`/uniD800` directly (the generic hex-codepoint path, with no ligature involved), asserting no +exception and a base-encoding fallback exactly like the above-`0x10FFFF` case; +`PdfDocument_Fonts_Differences_LigatureComponentSurrogateRange_FallsBackToBaseEncoding` declares +`/uniD800_i` (a ligature-underscore name whose first component is the surrogate-range value), +asserting the same no-exception fallback - proving the surrogate rejection is load-bearing for +the ligature path specifically, not merely a defensive check that happens to also work there. As of Phase B, a font explicitly declaring `/StandardEncoding` and showing byte code `0x27` (which diverges between the two base encodings - `StandardEncoding` maps it to U+2019 "quoteright", `WinAnsiEncoding` maps it to diff --git a/src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Fonts.cs b/src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Fonts.cs index 869d0f50..5094e872 100644 --- a/src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Fonts.cs +++ b/src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Fonts.cs @@ -660,17 +660,25 @@ private static bool TryParseAdobeGlyphListHexName(string name, out int codepoint /// digit uXXXX/.../uXXXXXX convention can spell a six-digit value as large as /// 0xFFFFFF, which is not a valid AGL codepoint name at all, so such a name must /// stay unresolved (per 's own tolerant handling for - /// unrecognized names) rather than being treated as resolved. + /// unrecognized names) rather than being treated as resolved. Likewise rejects a value in + /// the UTF-16 surrogate range (0xD800-0xDFFF): a lone surrogate value is not + /// a valid Unicode scalar value on its own (surrogates only exist in pairs, as a UTF-16 + /// encoding detail, never as a standalone codepoint), so resolving one here would hand + /// 's char.ConvertFromUtf32 call (and + /// any other codepoint consumer) a value it cannot legally convert, throwing + /// instead of this method's own documented + /// fail-closed "stay unresolved" contract. /// /// The candidate hex digit span to parse. /// /// The parsed value, or 0 when contains any - /// non-uppercase-hex character, or when the parsed value exceeds 0x10FFFF. + /// non-uppercase-hex character, when the parsed value exceeds 0x10FFFF, or when the + /// parsed value falls within the UTF-16 surrogate range (0xD800-0xDFFF). /// /// /// if every character in was an - /// uppercase hex digit and the parsed value is a valid Unicode codepoint (at most - /// 0x10FFFF). + /// uppercase hex digit and the parsed value is a valid Unicode scalar value (at most + /// 0x10FFFF and outside the surrogate range). /// private static bool TryParseUppercaseHexDigits(ReadOnlySpan digits, out int value) { @@ -684,7 +692,7 @@ private static bool TryParseUppercaseHexDigits(ReadOnlySpan digits, out in } if (!int.TryParse(digits, NumberStyles.HexNumber, CultureInfo.InvariantCulture, out value) || - value > 0x10FFFF) + value > 0x10FFFF || value is >= 0xD800 and <= 0xDFFF) { value = 0; return false; diff --git a/test/DemaConsulting.CanvasNet.Pdf.Tests/PdfDocumentTests.cs b/test/DemaConsulting.CanvasNet.Pdf.Tests/PdfDocumentTests.cs index 49a290a6..b0eff397 100644 --- a/test/DemaConsulting.CanvasNet.Pdf.Tests/PdfDocumentTests.cs +++ b/test/DemaConsulting.CanvasNet.Pdf.Tests/PdfDocumentTests.cs @@ -7591,6 +7591,67 @@ public void PdfDocument_Fonts_Differences_OutOfRangeUHexName_FallsBackToBaseEnco Assert.NotEqual(default, surface[11, 44]); } + /// + /// Proves that a syntactically-valid-looking four-hex-digit uniXXXX glyph name + /// whose value falls within the UTF-16 surrogate range (0xD800-0xDFFF) is + /// rejected rather than resolved: a lone surrogate value is not a valid Unicode scalar + /// value on its own, so TryParseUppercaseHexDigits must reject it (exactly like the + /// above-0x10FFFF case) and leave the affected code at whatever its base encoding + /// already assigned it, rather than propagating an invalid codepoint that would later + /// throw from char.ConvertFromUtf32 if it + /// ever reached TryResolveLigatureUnderscoreName. + /// + [Fact] + public void PdfDocument_Fonts_Differences_SurrogateRangeUniHexName_FallsBackToBaseEncoding() + { + // Arrange + var fontBytes = BuildEmbeddedFontBytes([(65, 1)]); + var (resourcesBody, extraObjects) = BuildSimpleTrueTypeFontResources( + fontBytes, + fontDictExtra: "/FirstChar 65 /LastChar 66 /Widths [600 600] " + + "/Encoding << /Differences [65 /uniD800] >>"); + + var bytes = BuildSinglePagePdfWithResources(100, 100, "BT /F1 20 Tf 5 50 Td (A) Tj ET", resourcesBody, extraObjects); + + // Act + using var surface = RenderPdfBytes(bytes); + + // Assert: no exception, and the glyph painted (base encoding's codepoint 65 mapping was + // preserved rather than overwritten by the surrogate-range override). + Assert.NotEqual(default, surface[11, 44]); + } + + /// + /// Proves that a ligature-underscore glyph name whose component resolves to a UTF-16 + /// surrogate-range value (for example uniD800_i, a deliberately pathological name + /// whose first component names a lone surrogate) does not throw + /// : TryResolveLigatureUnderscoreName + /// resolves each component via TryResolveGlyphNameToCodepoint, which (via + /// TryParseUppercaseHexDigits's surrogate-range rejection) now fails closed for that + /// component instead of handing an invalid scalar value to char.ConvertFromUtf32, + /// so the whole ligature name is left unresolved and the affected code falls back to its + /// base encoding - exactly like any other unrecognized /Differences name. + /// + [Fact] + public void PdfDocument_Fonts_Differences_LigatureComponentSurrogateRange_FallsBackToBaseEncoding() + { + // Arrange + var fontBytes = BuildEmbeddedFontBytes([(65, 1)]); + var (resourcesBody, extraObjects) = BuildSimpleTrueTypeFontResources( + fontBytes, + fontDictExtra: "/FirstChar 65 /LastChar 66 /Widths [600 600] " + + "/Encoding << /Differences [65 /uniD800_i] >>"); + + var bytes = BuildSinglePagePdfWithResources(100, 100, "BT /F1 20 Tf 5 50 Td (A) Tj ET", resourcesBody, extraObjects); + + // Act + using var surface = RenderPdfBytes(bytes); + + // Assert: no exception, and the glyph painted (base encoding's codepoint 65 mapping was + // preserved rather than overwritten by the unresolved ligature override). + Assert.NotEqual(default, surface[11, 44]); + } + /// /// Proves that each of the AGL generic uXXXX/uXXXXX/uXXXXXX /// hex-codepoint glyph-name lengths (four, five, and six uppercase hex digits) actually From a0f747ce92fc3af5d8010eea9dc9b0e7a4e38085 Mon Sep 17 00:00:00 2001 From: Malcolm Nixon Date: Sun, 4 Oct 2026 08:23:06 -0400 Subject: [PATCH 3/3] Address PR #36 review: hex-component ligature test, honest fixture claims - Add a uni0066_uni0069 InlineData case to PdfDocument_Fonts_Differences_LigatureUnderscoreName_ResolvesViaEnrichedEmbeddedFontGlyphMap so the theory also exercises AGL-hex-named components (not only literal single-letter names), proving TryResolveLigatureUnderscoreName's per-component recursion into TryResolveGlyphNameToCodepoint, not just a direct StandardGlyphNames lookup. - Correct PdfFixtures/README.md and docs/verification/canvas-net-pdf/pdf-document.md: the real-world text-type1c-differences-agl-ligatures.pdf fixture test only asserts that some pixel paints somewhere on the page, which cannot distinguish a resolved /f_f ligature glyph from an unresolved tofu glyph (the page already painted ink before this fix, from unrelated surrounding text). The isolated, glyph-specific proof is the dedicated synthetic regression test; the fixture now documented only as proof the real-world name occurs and the document still opens/renders without error. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- docs/verification/canvas-net-pdf/pdf-document.md | 16 +++++++++------- .../PdfDocumentTests.cs | 8 +++++++- .../PdfFixtures/README.md | 15 ++++++++++----- 3 files changed, 26 insertions(+), 13 deletions(-) diff --git a/docs/verification/canvas-net-pdf/pdf-document.md b/docs/verification/canvas-net-pdf/pdf-document.md index 5924c9b7..afcdc9b3 100644 --- a/docs/verification/canvas-net-pdf/pdf-document.md +++ b/docs/verification/canvas-net-pdf/pdf-document.md @@ -1236,13 +1236,15 @@ trimmed, genuinely real-world `text-type1c-differences-agl-ligatures.pdf` fixtur `PdfFixtures\README.md`) - two actual embedded, subsetted Type1C fonts (`Gotham-Bold`/ `Gotham-Book`) whose own `/Differences` arrays name glyphs (`/uni03BC`, `/thinspace`) this exact regression previously left unresolved - and asserts visible glyph ink paints; this fixture's -content stream also genuinely exercises an `/f_f` ligature-glyph name (an AGL underscore-ligature -decomposition naming the "ff" ligature glyph) against real body text, which now likewise resolves -via `TryResolveLigatureUnderscoreName` (see -`PdfDocument_Fonts_Differences_LigatureUnderscoreName_ResolvesViaEnrichedEmbeddedFontGlyphMap` -above for the precise synthetic regression test isolating that mechanism) - so this broad fixture -test deliberately only asserts "some ink paints somewhere" rather than per-glyph pixel positions, -since the precise synthetic tests already isolate each fixed mechanism exactly. +content stream also genuinely contains an `/f_f` ligature-glyph name (an AGL underscore-ligature +decomposition naming the "ff" ligature glyph) against real body text, which this fix also allows +to resolve via `TryResolveLigatureUnderscoreName` - but this broad fixture test only asserts "some +ink paints somewhere" on the whole page (a page that already painted ink before this fix, from the +unrelated text surrounding the ligature), so it cannot by itself distinguish a resolved `/f_f` +glyph from an unresolved tofu glyph; the dedicated, isolated proof that the mechanism itself works +is `PdfDocument_Fonts_Differences_LigatureUnderscoreName_ResolvesViaEnrichedEmbeddedFontGlyphMap` +above, which pixel-asserts the specific glyph position against a synthetic embedded font built +for exactly that purpose. #### CanvasNetPdf-PdfDocument-Type3FontResolution: Type3 Fonts Resolve Required Fields, Fail Closed on Malformed Ones diff --git a/test/DemaConsulting.CanvasNet.Pdf.Tests/PdfDocumentTests.cs b/test/DemaConsulting.CanvasNet.Pdf.Tests/PdfDocumentTests.cs index b0eff397..2539df3d 100644 --- a/test/DemaConsulting.CanvasNet.Pdf.Tests/PdfDocumentTests.cs +++ b/test/DemaConsulting.CanvasNet.Pdf.Tests/PdfDocumentTests.cs @@ -7702,7 +7702,12 @@ public void PdfDocument_Fonts_Differences_ValidUHexName_ResolvesViaEnrichedEmbed /// own end-to-end pattern). This is the reported real-world regression: a PDF's /// /Differences array naming ligature glyphs this way previously rendered as /// missing/tofu glyphs because neither StandardGlyphNames nor the AGL hex - /// convention recognized the underscore-joined name at all. + /// convention recognized the underscore-joined name at all. The final case, + /// uni0066_uni0069, instead spells both components using the AGL's own generic + /// uniXXXX hex-codepoint convention (rather than a literal single-letter name), + /// proving TryResolveLigatureUnderscoreName's per-component recursion into + /// TryResolveGlyphNameToCodepoint - not just a direct StandardGlyphNames + /// lookup - correctly resolves each component before concatenation. /// [Theory] [InlineData("f_i")] // "f" + "i" -> "fi" -> U+FB01 LATIN SMALL LIGATURE FI. @@ -7710,6 +7715,7 @@ public void PdfDocument_Fonts_Differences_ValidUHexName_ResolvesViaEnrichedEmbed [InlineData("f_f")] // "f" + "f" -> "ff" -> U+FB00 LATIN SMALL LIGATURE FF. [InlineData("f_f_i")] // "f" + "f" + "i" -> "ffi" -> U+FB03 LATIN SMALL LIGATURE FFI. [InlineData("f_f_l")] // "f" + "f" + "l" -> "ffl" -> U+FB04 LATIN SMALL LIGATURE FFL. + [InlineData("uni0066_uni0069")] // AGL-hex components "uni0066" + "uni0069" -> "f" + "i" -> "fi" -> U+FB01. public void PdfDocument_Fonts_Differences_LigatureUnderscoreName_ResolvesViaEnrichedEmbeddedFontGlyphMap( string glyphName) { diff --git a/test/DemaConsulting.CanvasNet.Pdf.Tests/PdfFixtures/README.md b/test/DemaConsulting.CanvasNet.Pdf.Tests/PdfFixtures/README.md index 07cf463f..cc357ee3 100644 --- a/test/DemaConsulting.CanvasNet.Pdf.Tests/PdfFixtures/README.md +++ b/test/DemaConsulting.CanvasNet.Pdf.Tests/PdfFixtures/README.md @@ -143,9 +143,14 @@ Glyph List's generic `uniXXXX` hex-codepoint naming convention. Code 27 `/thinsp now a direct `StandardGlyphNames` entry, not a missing-vocabulary case. Its content stream genuinely exercises both codes against real body text (`"...50μL..."`, `"...+1 % compared..."`). -> **Note**: this fixture's content stream also genuinely exercises a `/f_f` ligature-glyph name +> **Note**: this fixture's content stream also genuinely contains a `/f_f` ligature-glyph name > (an underscore-joined AGL ligature decomposition, naming the "ff" ligature glyph) against real -> body text - previously left unresolved by design, this now resolves via -> `TryResolveLigatureUnderscoreName`'s underscore-decomposition fallback (see -> `PdfDocumentTests`'s own dedicated synthetic regression tests for the isolated mechanism, and -> this fixture for proof it resolves against a genuinely real-world embedded font too). +> body text - previously left unresolved by design, this name now resolves via +> `TryResolveLigatureUnderscoreName`'s underscore-decomposition fallback. However, this fixture's +> own test only asserts that some pixel paints somewhere on the page - a page that already +> painted ink before this fix, from the unrelated text surrounding the ligature - so it cannot by +> itself distinguish a resolved `/f_f` glyph from an unresolved tofu glyph. The isolated proof +> that the mechanism itself works is `PdfDocumentTests`'s own dedicated synthetic regression +> tests, which pixel-assert the specific glyph position against a synthetic embedded font built +> for exactly that purpose; this fixture merely shows the real-world name occurs and the document +> still opens and renders without error.