test(refraction): pinned mainframe corpora, generated-output snapshot, tool-regex ReDoS sweep (#3213, #3212, #3214)

[squid-protocol]

Closes #3213, closes #3212 (its second half; the sorted IR sets landed in #3217), closes #3214. Part of #3122.

Tests and tools only. No golden-master bless. The engine is untouched.

#3213: pinned corpora as commands

• Manifest. tests/cobol_mainframe/corpora.json records name, url, branch, ref, license, languages, answer key and CI excerpt for each corpus.
• Tool. tests/tools/mainframe_corpus.py has these commands:
  • fetch: a shallow clone at the pinned SHA. It refuses to discard local edits without --force.
  • scan: builds the master DB through galaxy_ir.scan_to_db. The DB is cached by ref, engine HEAD and a hash of any dirty gitgalaxy/ diff, so an engine edit never reuses a stale DB. The scan runs with this checkout first on PYTHONPATH, which avoids the stale-worktree trap.
  • score: the answer-key scorer on the cached clone and DB.
  • path and excerpt.
• Cache. The gitignored .mainframe_corpora/, or $GITGALAXY_MAINFRAME_CORPORA. Galaxyscope's aperture skips dot-dirs, the same precedent as .ast_accuracy_corpus/.
• Timings. A cold fetch of all three corpora takes about 5s and scan about 30s. score then reproduces the post-fix(refraction): graveyard reachability with a SECTION model (#3203) + per-path clean-room keys (#3218) #3219 numbers (CBSA dead P 62/62, R 62/62) in 1.8s from cache.

Third corpus / BMS survey. aws-mainframe-modernization-carddemo @ 59cc6c2f (Apache-2.0) has 21 .bms maps, each with its generated symbolic-map copybook; CBSA has 10. On both, the engine's mapset→class_data and map→function_data names match the source labels on 31/31 files. Two limits remain for #3122's BMS row:

• Neither corpus has a mapset with more than one DFHMDI.
• The 733 named DFHMDF fields are not named entities in the DB.

#3212: snapshot of the generated clean room + Spring Boot tree

tests/tools/refraction_snapshot.py check|update [--corpus NAME ...]:

• What it runs. cobol-refractor and then cobol-to-java, in-process, on a copy of the input.
• Normalisation. The scratch dir becomes <WORK> with / separators, the timestamp becomes <TS>, and JSON is re-serialised canonically.
• Storage. One <path>.snap file per generated file. A forge PR's diff therefore shows exactly which JCL, schema, IR or Java lines moved.

input	snapshot	where it runs
committed excerpts tests/cobol_mainframe/refraction_excerpts/ (35 upstream files, unmodified, license files alongside, listed in the manifest, regenerated by mainframe_corpus.py excerpt)	refraction_snapshot/excerpts/ (87 files)	CI, via test_refraction_snapshot.py, ~2s
full pinned corpora	refraction_snapshot/full/ (440 files, 1.27 MB)	locally, once fetched; skipped in CI

The excerpts are chosen to exercise what was just fixed:

• cobol-refractor keys every output and its dead-code state by file stem, so same-named programs overwrite each other (and merge dead code in SQLite mode) #3218's COBOL/SAM2 + multiroot/sam/SAM2;
• zapp.yaml libraries;
• CBSA's dead CALC-DAY-OF-WEEK and POPULATE-TIME-DATE sections, and the BANKDATA OPEN OUTPUT;
• CardDemo batch lineage, and dynamic CALLs with a BMS copybook.

.gitattributes keeps the excerpts byte-exact on Windows (-text) and the snapshots LF.

What the snapshot found on its first run. The output was not deterministic. Two runs with different PYTHONHASHSEED differed on CardDemo's COCRDLIC: unresolved_calls came from list(set), which reordered the IR, the audit report, the agent job and the generated Java. Fixes:

• cobol_dag_architect.py: sorted(dynamic_calls).
• Both controllers and cobol_jcl_auditor.py: sort their rglob/glob walks, because filesystem order differs per OS and orders the report, the jobs and same-name entity writes.
• The auditor's list(set)[0] becomes sorted(...)[0].

These change ordering only. Output is now byte-identical across three hash seeds.

It also shows #3221: the five zopeneditor IR dumps yield three Java services. cobol-to-java still names classes from metadata.file_name, so the two SAM2 programs collide. This is the Java side of #3218, filed rather than fixed here.

#3214: ReDoS sweep over the tool regexes

tests/tools/tool_regex_redos.py [--ci | --update-baseline]:

• Collection. It takes every re.*(...) call in cobol_to_cobol/, cobol_to_java/ and both controllers. Patterns and flags are evaluated in the module namespace, so f-strings over module constants resolve. It also takes every compiled global. The total is 54 patterns. The 4 built from function locals are listed as skipped, never dropped.
• Measurement. Each pattern runs through the method its code actually calls, over payload families derived from its own keywords, including every prefix of the keyword sequence (the Refraction JCL forge: EXEC CICS/SQL counts wrong and the unbounded lazy regex runs over the whole joined file #3205 shape). Each pattern runs in a worker process with a hard timeout.
• Offender rule.
  • One call over more than 0.25s at 4k or 40k chars is an offender outright.
  • Otherwise a family must cost ≥0.1s at 160k chars and grow ≥8x for 4x the input, confirmed best-of-3. Linear growth is ~4x, quadratic ~16x.
  • An absolute budget alone would flag the bounded .{0,600}? CICS HANDLE scan: 0.13s at 160k, but linear. There is no ratio between small single samples (flaky: ReDoS "pre-fix pattern scales quadratically" timing ratios fail on shared macOS runners #2901).
• Baseline. tests/cobol_mainframe/tool_regex_redos_baseline.json holds three offenders, each with a note → Refraction tools: three regexes with quadratic scaling (baselined by the #3214 sweep) #3222:
  • dag_architect OPEN [^.]*
  • graveyard_finder REPLACING pairs
  • jcl_forge SELECT/ASSIGN [^.]*
• CI. It runs through test_tool_regex_redos.py in every pytest leg, ~2s, because it skips re-measuring the baselined offenders.
• Revert check. Restoring cobol_jcl_forge.py from before fix(refraction): forge parser fixes (#3203 1-4, #3204, #3205, #3206, #3212 determinism) #3217 reports EXEC\s+CICS.*?END-EXEC\. and its EXEC SQL twin as 2 new offenders, so --ci fails.

tests/extraction/tools/sweep_redos_scaling.py is not extended. It scrapes assert_redos_immune payloads out of the engine's strict suites, and the tools have no such suites to scrape.

Verification

• Full suite: pytest tests/ -n 8 gives 10250 passed, 7 skipped, 9 xfailed, 3 xpassed.
• New tests: 36 pass locally with the corpora. Under Python 3.9 with no corpora (the CI situation), 30 pass and 6 skip; this caught a write_text(newline=) that is 3.10-only.
• Lint: audit_check.py is all clear (ruff, mypy, dead-key, ast-accuracy). xray-inspector . reports 0 anomalies.

Docs

• tests/cobol_mainframe/readme.md has a new section 7.
• The answer-key README and docs/refraction_engine_differential.md now give the commands in place of the manual clone steps.

Unblocks #3215 (the cobol-modernization skill), which can now document commands.

🤖 Generated with Claude Code

[github-actions]

🐦‍⬛ Muninn Security Scan

✅ No security issues found.

🐦‍⬛ Powered by Muninn · Skald Lab