Skip to content

fix: negated classes (\S, \W, \D, [^a]) match bytes, not codepoints #174

Closed

Description

@kolkov

Negated character classes (\S, \W, \D, [^a]) match individual bytes instead of Unicode codepoints, producing false positives on multibyte input.

Repro

re := coregex.MustCompile(`\S{2}`)
fmt.Println(re.MatchString("К"))  // true — coregex (wrong: К is one rune, two bytes)

reStd := regexp.MustCompile(`\S{2}`)
fmt.Println(reStd.MatchString("К"))  // false — stdlib (correct)

More cases:

\S{2}   on "К"    → [0 2]   stdlib: no match
\W{2}   on "К"    → [0 2]   stdlib: no match
\D{2}   on "К"    → [0 2]   stdlib: no match
[^a]{3} on "日"   → match   stdlib: no match (日 is one rune, three bytes)

Both FindAllIndex and FindAllSubmatchIndex paths affected.

Context

. was fixed to match per-rune in v0.10.7. Negated classes ([^...], \S, \W, \D) still operate at byte level in the NFA. The fix should follow the same approach used for ..

This is the largest remaining class of stdlib divergence (~40% of fuzz failures at 0.75% rate).

Expected behavior

Negated classes advance by one Unicode codepoint, not one byte. A two-byte Cyrillic character is one \S, not two.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: nfaThompson NFA, PikeVM, BoundedBacktrackerpriority: criticalRelease blocker, correctness regressiontype: bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions