Fix #4266: Extract SYNC, JUSTIFIED, BLANK WHEN ZERO, RENAMES, and resolve qualified names in layout model

[squid-protocol]

Fixes #4266. Taken over from the first draft of this branch (Gemini) and reworked from a fresh rebase onto main. The first draft's 5 CI failures, and what it got wrong, are listed below.

What #4266 changes

• Extractor (core/mainframe_boundary.py): each COBOL record entry now records sync (SYNC / SYNC LEFT / SYNC RIGHT), justified, blank_when_zero (BLANK [WHEN] ZERO[S|ES]) and, on a 66 entry, renames (A / A THRU B; qualifiers kept as X OF Y, IN read as OF, THROUGH read as THRU). These keys only appear on an entry that codes the clause, the same way sign_separate works. Each clause is read from the entry's own text (stopping at its period, with literals blanked) and must sit on COBOL-name boundaries, so WS-SYNC-FLAG and VALUE 'SYNC' are not read as clauses, and Comp sync. is.

• DB: record_data gets sync TEXT, justified INTEGER, blank_when_zero INTEGER, renames TEXT, added to old tables by _ensure_columns. state_rehydrator and the GalaxyIR loader restore them. A DB written before these columns existed still loads, with no slack bytes and no RENAMES ranges (tested).

• GalaxyIR layout (record_layout and _storage_spans): a SYNCHRONIZED elementary item now starts on its IBM Enterprise COBOL boundary, counted from the start of the record (which IBM aligns on a doubleword):

  • binary up to 4 digits: halfword
  • binary with more digits, including 8-byte S9(10)-S9(18): fullword
  • COMP-1 / INDEX / pointers: fullword (LP(32))
  • COMP-2: doubleword
  • SYNC on DISPLAY, packed or national items adds nothing.

  The slack bytes count toward the group that holds the item. Slack between the occurrences of a table is not modelled yet (noted in _slack).

• RENAMES: a 66 entry is laid out as the byte range it names (_renames_spans), from the first byte of A to the last byte of B, within the record it follows. Each operand must name exactly one placed item there, otherwise the entry is left unplaced. _operand_span resolves the entry by its name, or by its name qualified by the record, so a MOVE from or to it resolves in data_flows.

• Qualified / subscripted sources for data-move lineage: main already resolved these in data_flows (engine MOVE truncation: a qualified source (X OF Y) is never flagged as truncating #4204 qualifiers matched against storage ancestors; a subscript is stripped and occurrence_bytes rides along). The new test covers a qualified RENAMES operand (CUST-POSTAL OF CUSTOMER). The cics_mainframe_facts.md §6 sentence the issue cites is about CICS name resolution: follow data-name MOVE chains (bounded), not one hop #3578's VALUE/MOVE name-resolution chain in core/cics_resources.py, which is a different reader. It is not changed here (see "Owner decisions").

• Tests: tests/cobol_mainframe/test_sync_renames_4266.py (10 tests). They go through the full scan → DB → GalaxyIR path and include a SYNC record of 36 bytes against the same record unsynced at 30.

The first draft's CI failures: cause and fix

Check	Cause	Fix
full-suite, ground-truth	test_renames: UnboundLocalError: key. The RENAMES loop in _storage_spans used a leftover loop variable. It also called _operand_span before _span_placed existed, which would raise KeyError on a qualified operand.	RENAMES is resolved by a new _renames_spans, run inside _storage_spans before the cache is published. It matches operands against the placements directly.
crucible-audit (both legs)	(1) The draft put RENAMES into redefines ('Redefines': 'WS-TEN THRU WS-TEA'), which is semantically wrong and changed the Record Layout golden. (2) MISSING KEY: 7. Mainframe Skeleton Completeness: the branch was stale against main.	renames is its own field, and redefines is untouched. Rebased onto main. The golden diff that remains is explained below.
ruff-audit	Unformatted lines, plus an unrelated Optional[...] -> X | None sweep over two whole files. That sweep produced most of the draft's -306 lines; no working code was dropped.	The sweep is dropped. Only the changed files are run through ruff format (0.16.0). ruff_audit.py --ci passes with no new findings, so the baseline is not regenerated. It reports 3 FYI "no longer flagged" UP045 entries in galaxy_ir that could be pruned later.

The draft had three other problems: a plain SYNC (the common case) stored "", which is falsy, so it never aligned; SYNC. (period right after the clause) did not match; and the clauses never reached the DB, so a real scan never saw them. It also loosened data_moves.py to accept A (I) OF B, which is not standard COBOL. That change is dropped.

What moved (golden masters)

Both golden masters (7_mainframe_skeleton_completeness/channels.json) were regenerated with tests/tools/crucible_check.py --update --yes:

Field	Before	After
data flows / Resolved	27040	27044
data flows / Gaps / unresolved operand	99	95
data flows / Ratio	99.6%	99.7%

The 4 newly resolved flows are all in CardDemo COACTUPC.cbl, at lines 1933, 1990, 2039 and 2087:

• MOVE LIT-ALL-ALPHA-FROM-X TO LIT-ALL-ALPHA-FROM. The source is 66 ... RENAMES LIT-UPPER THRU LIT-LOWER: 26 + 26 = 52 bytes at offset 172 of WS-LITERALS. The receiver is PIC X(52).
• MOVE LIT-ALL-ALPHANUM-FROM-X TO LIT-ALL-ALPHANUM-FROM. The source is ... THRU LIT-NUMBERS: 62 bytes. The receiver is PIC X(62).

All four report truncates: false, so the computed widths agree with the receivers the program sized for them. scope_check.py --expect cobol reports exactly these three fields, as repo-wide rollup fields that it cannot attribute to a language, and nothing else. No Record Layout or per-file golden moved: the new keys are not rendered by audit_recorder, and the crucible corpus's SYNC items (genapp lgwebst5.cbl) are 01-levels, which need no slack.

Ground-truth ledger, fact cross-check, estate gate and completeness pins: no movement.

Local gates

• pr_gates.py --fast: 5/5 pass (lint, audits, secrets, xray, gauntlet).
• pr_gates.py --ratchets: 6/6 pass (ports, port-surface, estate, fact-crosscheck, completeness, ground-truth).
• Targeted tests: 326 passed across the layout, IR, answer-key, sign-separate, group-usage, data-moves, multi-program and skeleton tests and the dead-key audit.
• The full suite was not run locally (owner rule); CI runs it.

Owner decisions / follow-ups

• IBM vs other compilers: an 8-byte binary is aligned on a fullword (IBM Enterprise COBOL). GnuCOBOL and Micro Focus may use a doubleword. This PR follows IBM.
• Not modelled: slack between table occurrences (OCCURS of a group holding SYNC items).
• Census answer-key prompt: tests/tools/cross_verify_sections.py tells reviewers to "assume no slack bytes (ignore SYNCHRONIZED)". The engine now adds slack. No ledger entry moved today (the census SYNC items sit on aligned offsets or are 01s), but the zosconnect claimreq.cpy pattern (X(255) followed by COMP-2 SYNC) would differ by 7 bytes if it were ever keyed.
• Out of scope here:
  • the det translator's own layout (det/layout.py:223 "RENAMES: not laid out"), left alone to avoid colliding with the sibling det work;
  • the CICS name resolution: follow data-name MOVE chains (bounded), not one hop #3578 MOVE-chain reader for qualified or subscripted sources (cics_resources.py);
  • rendering the new keys in the audit Record Layout.

🤖 Generated with Claude Code

[github-actions]

🐦‍⬛ Muninn Security Scan

✅ No security issues found.

🐦‍⬛ Powered by Muninn · Skald Lab