Skip to content

fix(refraction): bound the three quadratic tool regexes (#3222) - #3230

Merged
squid-protocol merged 1 commit into
mainfrom
fix/3222-quadratic-regexes
Sep 20, 2026
Merged

squid-protocol merged 1 commit into
mainfrom
fix/3222-quadratic-regexes

Conversation

@squid-protocol

Copy link
Copy Markdown
Owner

Closes #3222. Part of #3122.

All three baselined offenders had the same shape: an unbounded run before a required terminator, so the scan restarts from every candidate start position. That is exactly what #3205 fixed for EXEC CICS … END-EXEC..

The three

site before after
cobol_dag_architect.py OPEN operands \bOPEN\s+((?:INPUT|OUTPUT|I-O|EXTEND)\b[^.]*)\. _OPEN_ANCHOR + str.find(".")
cobol_jcl_forge.py SELECT/ASSIGN SELECT …\s+ASSIGN\s+…([A-Z0-9\-]+)[^.]*\. _FILE_ASSIGN_ANCHOR + str.find(".")
cobol_graveyard_finder.py REPLACING pairs (?:==)?([A-Z0-9_\-]+)(?:==)?\s+BY\s+… _REPLACING_PAIR, with a (?<![A-Z0-9_\-]) boundary lookbehind

Each sliced run keeps the old pattern's two rules: the terminating period is still required (an unterminated tail matched nothing before and matches nothing now), and a consumed cursor reproduces finditer's non-overlapping advance, so an anchor inside a statement already read is not a second statement.

Measured on the inputs the new tests use:

before after
20k unterminated OPEN INPUT WS-F 24.1s ~1ms
20k period-less SELECT F1 ASSIGN TO DD1 21.4s ~1ms
a 20k-character name in a REPLACING clause 10.0s 1.3ms

Why no output moves

The name run is greedy and the name class is closed under itself, so a run can never stop inside a token: a match that starts mid-token captures the same two names as the boundary start does. The lookbehind can therefore move a match's start offset but never its groups, and findall returns groups only.

Checked rather than argued: a 40,000-trial differential of old pattern vs new scan over random COBOL-ish token soup (including ==, X==A==, OPENX, SELECTX, stray periods and quotes) found 0 divergences — plus everything below.

Proof

tool_regex_redos.py --ci      3 offenders -> 0 offenders   (baseline now empty)
refraction_snapshot.py check                    0 of 87 excerpt files differ
  --corpus zopeneditor-sample                   0 of 38 differ
  --corpus cics-banking-sample-application-cbsa 0 of 166 differ
  --corpus aws-mainframe-modernization-carddemo 0 of 236 differ
mainframe_corpus.py score     byte-identical to origin/main, both corpora, both columns
pytest tests/cobol_mainframe/                   128 passed, 6 skipped
audit_check.py                                  all clear

Scores are unchanged, so no before/after table is meaningful here — the point of this PR is that the numbers do not move:

forge engine DB
zopeneditor units P 58/58 · R 58/58 P 58/58 · R 58/58
CBSA units P 680/680 · R 680/680 P 680/680 · R 680/680
CBSA dead P 62/62 · R 62/62 P 29/247 · R 29/62

Tests

Six added, two per site: an adversarial-input test with a generous absolute 5s bound (the old code takes 10–24s on the same input; per #2901 no ratio between small samples is asserted), and a semantics test pinning what the site must still extract — #3204's multi-mode OPEN INPUT A OUTPUT B. walk, the UT-/UR-S- DD strip across a multi-line SELECT, and the REPLACING pairs including the pre-existing (unchanged) fact that a :TAG: pseudo-text name matches neither pattern.

The baseline JSON is now "offenders": [], with a note that an entry may only be added alongside an issue tracking its fix.

🤖 Generated with Claude Code

All three had an unbounded run before a required terminator, so the scan
restarts from every candidate position: the shape #3205 fixed for
`EXEC CICS ... END-EXEC.`.

- `cobol_dag_architect.py` OPEN: `[^.]*\.` rescanned to the next period from
  every OPEN. The pattern is now an anchor and the operand run is sliced with
  `str.find`, with a `consumed` cursor that reproduces `finditer`'s
  non-overlapping scan. 20k unterminated OPENs: 24s -> ~1ms.
- `cobol_jcl_forge.py` SELECT/ASSIGN: same shape, same fix. The terminating
  period is still required. 20k period-less SELECTs: 21s -> ~1ms.
- `cobol_graveyard_finder.py` REPLACING pairs: an unbounded name before a
  required `BY`, so every position inside a name was a candidate start. Now
  compiled once as `_REPLACING_PAIR` with a `(?<![A-Z0-9_\-])` boundary
  lookbehind. 20k name characters: 10s -> ~1ms.

None of the three changes an output. The name run is greedy and cannot stop
inside a token, so the lookbehind can only move a match's start, never its
captures; the two sliced runs keep the old pattern's required terminator and
its non-overlapping advance. Verified by a 40k-trial differential of old
pattern vs new scan over random COBOL-ish token soup, and below.

Proof:
- `tool_regex_redos.py --ci`: 3 offenders -> 0. Baseline now empty.
- `refraction_snapshot.py check` and `--corpus <all three>`: 0 files differ
  (38 + 166 + 236 full, 87 excerpt).
- `mainframe_corpus.py score`: byte-identical to origin/main, both corpora,
  both columns.
- `pytest tests/cobol_mainframe/`: 128 passed.

Closes #3222.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

🐦‍⬛ Muninn Security Scan

✅ No security issues found.

🐦‍⬛ Powered by Muninn · Skald Lab

@squid-protocol
squid-protocol merged commit 2fb109e into main Sep 20, 2026
29 checks passed
@squid-protocol
squid-protocol deleted the fix/3222-quadratic-regexes branch September 20, 2026 00:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Refraction tools: three regexes with quadratic scaling (baselined by the #3214 sweep)

1 participant