Repository navigation
fix(refraction): bound the three quadratic tool regexes (#3222) - #3230
Merged
Merged
Conversation
All three had an unbounded run before a required terminator, so the scan restarts from every candidate position: the shape #3205 fixed for `EXEC CICS ... END-EXEC.`. - `cobol_dag_architect.py` OPEN: `[^.]*\.` rescanned to the next period from every OPEN. The pattern is now an anchor and the operand run is sliced with `str.find`, with a `consumed` cursor that reproduces `finditer`'s non-overlapping scan. 20k unterminated OPENs: 24s -> ~1ms. - `cobol_jcl_forge.py` SELECT/ASSIGN: same shape, same fix. The terminating period is still required. 20k period-less SELECTs: 21s -> ~1ms. - `cobol_graveyard_finder.py` REPLACING pairs: an unbounded name before a required `BY`, so every position inside a name was a candidate start. Now compiled once as `_REPLACING_PAIR` with a `(?<![A-Z0-9_\-])` boundary lookbehind. 20k name characters: 10s -> ~1ms. None of the three changes an output. The name run is greedy and cannot stop inside a token, so the lookbehind can only move a match's start, never its captures; the two sliced runs keep the old pattern's required terminator and its non-overlapping advance. Verified by a 40k-trial differential of old pattern vs new scan over random COBOL-ish token soup, and below. Proof: - `tool_regex_redos.py --ci`: 3 offenders -> 0. Baseline now empty. - `refraction_snapshot.py check` and `--corpus <all three>`: 0 files differ (38 + 166 + 236 full, 87 excerpt). - `mainframe_corpus.py score`: byte-identical to origin/main, both corpora, both columns. - `pytest tests/cobol_mainframe/`: 128 passed. Closes #3222. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3222. Part of #3122.
All three baselined offenders had the same shape: an unbounded run before a required terminator, so the scan restarts from every candidate start position. That is exactly what #3205 fixed for
EXEC CICS … END-EXEC..The three
cobol_dag_architect.pyOPEN operands\bOPEN\s+((?:INPUT|OUTPUT|I-O|EXTEND)\b[^.]*)\._OPEN_ANCHOR+str.find(".")cobol_jcl_forge.pySELECT/ASSIGNSELECT …\s+ASSIGN\s+…([A-Z0-9\-]+)[^.]*\._FILE_ASSIGN_ANCHOR+str.find(".")cobol_graveyard_finder.pyREPLACING pairs(?:==)?([A-Z0-9_\-]+)(?:==)?\s+BY\s+…_REPLACING_PAIR, with a(?<![A-Z0-9_\-])boundary lookbehindEach sliced run keeps the old pattern's two rules: the terminating period is still required (an unterminated tail matched nothing before and matches nothing now), and a
consumedcursor reproducesfinditer's non-overlapping advance, so an anchor inside a statement already read is not a second statement.Measured on the inputs the new tests use:
OPEN INPUT WS-FSELECT F1 ASSIGN TO DD1Why no output moves
The name run is greedy and the name class is closed under itself, so a run can never stop inside a token: a match that starts mid-token captures the same two names as the boundary start does. The lookbehind can therefore move a match's start offset but never its groups, and
findallreturns groups only.Checked rather than argued: a 40,000-trial differential of old pattern vs new scan over random COBOL-ish token soup (including
==,X==A==,OPENX,SELECTX, stray periods and quotes) found 0 divergences — plus everything below.Proof
Scores are unchanged, so no before/after table is meaningful here — the point of this PR is that the numbers do not move:
Tests
Six added, two per site: an adversarial-input test with a generous absolute 5s bound (the old code takes 10–24s on the same input; per #2901 no ratio between small samples is asserted), and a semantics test pinning what the site must still extract — #3204's multi-mode
OPEN INPUT A OUTPUT B.walk, theUT-/UR-S-DD strip across a multi-line SELECT, and the REPLACING pairs including the pre-existing (unchanged) fact that a:TAG:pseudo-text name matches neither pattern.The baseline JSON is now
"offenders": [], with a note that an entry may only be added alongside an issue tracking its fix.🤖 Generated with Claude Code