Skip to content

fix: negated classes match runes not bytes - #180

Merged
kolkov merged 4 commits into
mainfrom
feature/issue-174-negated-class-rune-matching
Oct 9, 2026
Merged

kolkov merged 4 commits into
mainfrom
feature/issue-174-negated-class-rune-matching

Conversation

@kolkov

@kolkov kolkov commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Remove 0x80–0xFF single-byte range from coversAllNonASCII path in NFA compiler
  • This range caused negated classes (\S, \D, \W, [^x]) to match individual bytes of valid multi-byte UTF-8 runes — e.g. \S{2} matched single Cyrillic К (2 bytes)
  • Invalid UTF-8 byte handling deferred to fix: invalid UTF-8 bytes silently skipped instead of matching as U+FFFD #179 (look-ahead design)
  • Random-fuzz divergence: 0.75% → 0.21%

Test plan

  • 7 MatchString tests: single rune vs multi-repetition (2-byte Cyrillic, 3-byte CJK, 4-byte emoji)
  • 6 FindAllString tests: rune-counted matches, not byte-split fragments
  • 3 FindAllStringIndex tests: byte-correct boundaries
  • Full suite green (go test ./... -short)
  • Lint clean, gofmt clean

Fixes #174

kolkov added 3 commits October 9, 2026 10:26
Go 1.27.2 uses export data version 5, golangci-lint v2.13 supports
only version 4. CI failed on v0.12.27 merge commit.
Remove 0x80-0xFF single-byte range from coversAllNonASCII path in NFA
compiler. This range caused negated classes (\S, \D, \W, [^x]) to match
individual bytes of valid multi-byte UTF-8 runes — e.g. \S{2} matched
single Cyrillic character К (2 bytes) by treating each byte as a
separate match.

Invalid UTF-8 byte handling deferred to #179 (look-ahead design).
Skipped TestInvalidUTF8NegatedCharClass until then.

Fixes #174
@codecov

codecov Bot commented Oct 9, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@github-actions

github-actions Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

Benchmark Comparison

Comparing main → PR #180

Summary: geomean 95.17n 92.96n -2.32%

⚠️ Potential regressions detected:

geomean                               ³                +0.00%               ³
geomean                               ³                +0.00%               ³
geomean              30.01n         30.04n        +0.12%
geomean                         ³                +0.00%               ³
geomean                         ³                +0.00%               ³
AhoCorasickManyPatterns/coregex_10_patterns-4           37.12n ± ∞ ¹    39.35n ± ∞ ¹    +6.01% (p=0.008 n=5)
AhoCorasickManyPatterns/coregex_25_patterns-4           44.54n ± ∞ ¹    47.42n ± ∞ ¹    +6.47% (p=0.008 n=5)
AhoCorasickManyPatterns/coregex_50_patterns-4           38.80n ± ∞ ¹    40.71n ± ∞ ¹    +4.92% (p=0.008 n=5)
AnchoredLiteralVsStdlib/stdlib_long-4                   373.7n ± ∞ ¹    376.1n ± ∞ ¹    +0.64% (p=0.008 n=5)
ASCIIOptimization_Issue79/short_WithASCII-4             223.5n ± ∞ ¹    223.8n ± ∞ ¹    +0.13% (p=0.024 n=5)

Full results available in workflow artifacts. CI runners have ~10-20% variance.
For accurate benchmarks, run locally: ./scripts/bench.sh --compare

@kolkov
kolkov merged commit 273ff75 into main Oct 9, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix: negated classes (\S, \W, \D, [^a]) match bytes, not codepoints

1 participant