Mainframe fact channel: field-level data lineage (#3452)

[squid-protocol]

Part of epic #3445. Stacked on #3483 (base feat/ims-gen-3477). Merge #3483 first, then retarget this PR to main.

What it does

Extraction (core/data_moves.py → data_move_data):

• One source → target row per MOVE / COMPUTE / ADD / SUBTRACT / MULTIPLY / DIVIDE / STRING / UNSTRING / INITIALIZE.
• Operands are kept as written, with qualifiers. Subscripts are dropped and reference modification is flagged. MOVE CORRESPONDING is marked.
• EXEC blocks are skipped, and so are control operands: STRING / UNSTRING delimiters, POINTER, COUNT IN and TALLYING.

Reader (GalaxyIR):

• data_flows() resolves both operands to storage: record, offset, bytes and one occurrence's bytes, via a COPY-expanded span walk.
  • Group moves and REDEFINES meet by storage rather than by name, including a level-01 REDEFINES.
  • A copybook item answers to the program group it expands under (COMM-SCODE OF UPDCUST-COMMAREA).
  • A procedure copybook (COPY CSUTLDPY. in the PROCEDURE DIVISION) resolves once per including program.
  • status is resolved | source_unresolved | target_unresolved | ambiguous | system (EIB / DIB / SQLCA / DFH constants / special registers).
  • truncates flags a MOVE into a shorter alphanumeric or group target.
• field_lineage(file, item, direction) follows a field forward or backward:
  • through moves (a group MOVE keeps the field's offset in the target), and through CALL USING BY REFERENCE and COMMAREA storage into other programs;
  • each hop is tagged with the channel endpoints it touches: FD records, SQL host variables, DL/I I/O areas, CICS FILE / MAP / QUEUE / CONTAINER records;
  • a name the repository doesn't declare, such as a generated BMS symbolic map, ends the trail as a named stub hop.

Example (CBSA): COMM-NAME OF UPDCUST-COMMAREA in BNK1DCS → LINK → UPDCUST DFHCOMMAREA → CUSTOMER-NAME → cics FILE CUSTOMER update. Backward, it reaches CUSTNAMI, a field of the generated map.

Engine fix found by the key

The key's independent widths disagreed with the engine on 4 CardDemo truncations. Root cause: _cobol_records read the continuation line 1 THROUGH 12. of an 88 … VALUES list as a new level-01 item THROUGH. That closed the date group early (6 bytes instead of 8) and wrongly re-parented later items.

• Such a line is now skipped, but only when it clearly continues a VALUE list (a THRU range, or an entry left open after VALUE(S) or a comma).
• A bare missing period still starts a new entry, as in genapp's 03 CA-CUSPOL-REQUEST.

Ground truth

Independent key reader: cobol_answer_key.py add-data-moves. It has its own statement cutting over the literal-blanked text, parks literals, and reads operand lists sequentially. It also computes its own PIC widths, splicing COPY members into groups.

field	CardDemo	CBSA	zOpenEditor
data moves	5171/5171	5179/5179	323/323
MOVE truncation	82/82	1/1	9/9

Ledger flag: data_moves_validated (drafted; the census is still owed). The truncations are real findings, for example MOVE 'DSNTIAC CD: ' (12 characters) into PIC X(10), and WS-MESSAGE into ERRMSGO.

Checks

• Golden masters: re-blessed. They gain a per-file "Data Moves" block (new data_moves.json). The only change to an existing file is CSUTLDWY's record layout: 3 phantom THROUGH items removed and 6 items re-parented, which is exactly the parser fix.
• Tests: unit (every verb, pseudo-text, EXEC, INVALID KEY, the VALUES continuation), recorder (including restoring a pre-Mainframe fact channel: field-level data lineage (MOVE / COMPUTE / STRING) #3452 DB), IR (storage, statuses, truncation, cross-program lineage, stub hops) and key.
• ruff and mypy audits are clean. The ledger is clean. Full suite: 11267 passed.

Scope notes

Names that only exist after COPY … REPLACING (zOpenEditor's :TAG: copybooks) stay unresolved, because record_data keeps the copybook's own names. SET, ACCEPT and READ INTO are not rows.

Closes #3452

🤖 Generated with Claude Code

https://claude.ai/code/session_017ZsVaAkb86P5r5JXDC2Y9g

[github-actions]

🐦‍⬛ Muninn Security Scan

✅ No security issues found.

🐦‍⬛ Powered by Muninn · Skald Lab