Bug sweep (COBOL engine): #4204 #4205 #4203

[squid-protocol]

Three COBOL engine bugs from the bug sweep, each fixed in its own commit with a test that fails on main and passes here.

#4204: MOVE truncation was never flagged for a qualified source

Root cause. IBM DBB EPSCMORT L231 is MOVE EPSPCOM-ERRMSG OF W-COMMUNICATION-AREA TO MSGERRO. The item comes from a nested copybook: it lives in EPSMTOUT, which EPSMTCOM COPYs, and the program COPYs EPSMTCOM twice, once under W-COMMUNICATION-AREA and once under DFHCOMMAREA.

GalaxyIR._storage_spans lays the same copybook item objects out once per COPY, but it keyed spans and ancestor paths by id(item). The second placement (DFHCOMMAREA) overwrote the first, so the qualifier OF W-COMMUNICATION-AREA never matched. The qualified lookup also searched only direct copybooks, not nested ones. The source stayed source_unresolved and truncates was None.

Fix. _storage_spans now records every placement of an item (_span_placed). _operand_span resolves a qualified operand against those placements, so the qualifier picks the right one. Unqualified lookups are unchanged.

Test. test_a_qualified_source_in_a_nested_copybook_copied_twice_resolves_and_truncates in tests/cobol_mainframe/test_galaxy_ir.py. Its fixture has the same nested-COPY, copied-twice shape.

Corpus effect. 17 qualified data flows on dbb-mortgage-application now resolve, and L231 is flagged as truncating (80 bytes into 40). The test_completeness pin for data flows moves from 124/566 to 141/568: +17 resolved here, plus 2 new rows from #4205.

#4205: MOVE ALL X'..' was not recorded

Root cause. In core/data_moves.py, the ALL operand branch only accepted a next token that starts with a quote. X'00' starts with X, so the statement produced no row.

Fix. One shared _is_literal helper (quoted, or an X/N/G/Z-prefixed literal) is now used by both the literal branch and the ALL branch.

Test. test_move_all_of_a_prefixed_literal_is_a_figurative_move in tests/core_engine/test_data_moves.py, covering ALL X'00', ALL x'FF' and ALL N'A'.

#4203: free-format data entries were read as paragraphs

Root cause. func_start has an optional 6-column sequence-area slot. In free-format source that slot swallows the indent plus the level number ( 01 of 01 VALUE-BYTES.), which steps past the regex's level-number shield, so the data name was captured as a paragraph header. On CobolCraft @ e8c420df this produced 143 phantom units, including every unit in coordinates.cob and decode.cob.

Fix. The existing cobol_sentence_start scope filter (#3197) now refuses a header on a line that has no fixed-format sequence area (the existing _COBOL_SEQUENCE_AREA test: six digits or six blanks) and that opens with a level number or FD/SD/RD/CD. Fixed-format source is untouched.

I checked this against all of CobolCraft: 419 units before, 276 after. All 143 removed units are data-entry lines, and no real paragraph was lost.

Test. tests/core_engine/test_cobol_free_format_data_entries_4203.py has CobolCraft-shaped valid and invalid fixtures (sibling programs, free-format paragraphs with FD/01/77 entries, a fixed-format control) plus a check that the func_start count and the unit list agree.

Left for a design call. The issue also proposes counting PROGRAM-ID units as function boundaries. Today each PROGRAM-ID is already counted in class_count (3 and 13 on the two files above). Making them units as well would change function_count and function-level metrics for every COBOL file in every corpus, and would need a decision on how the key models them (see #4206). This PR keeps to the false-positive fix and leaves #4203 open for that half.

Mainframe ground truth and field testing

• Ledger: the 3 dbb-mortgage-application mismatches no longer reproduce (MOVE truncation L231, MOVE ALL X'00' L1457, MOVE ALL X'FF' L3452), and their two defect causes are retired. ground_truth_ledger.py check passes with 0 new mismatches. Scoreboard: MOVE truncation is now 4/4 and data moves 578/578.
• field_testing.json: D038 (engine MOVE truncation: a qualified source (X OF Y) is never flagged as truncating #4204) now records its real root cause. D040 (cobol func_start: free-format source reads data items as paragraphs and misses PROGRAM-ID units (CobolCraft) #4203) is new, logged against CobolCraft (an unkeyed estate, after round 7). D039 (engine data moves: MOVE ALL X'..' (hex figurative) is not recorded #4205) was already logged. The report is regenerated; the units clean-fresh streak resets, as the protocol requires.

Golden masters: no drift

Neither fixture changes in this PR.

• Full precision: a clean scan of the crucible at the pinned tag, run with this branch's engine, gives 0 differences against tests/golden_master_audit at HEAD (bless_scope.py: newly parsed none, newly excluded none).
• Zero-dependency: a clean scan with galaxy_zero_venv (PyYAML only) gives 0 differences against tests/golden_master_zero_dep_audit at HEAD (newly parsed none, newly excluded none).
• The crucible has no free-format COBOL with indented level numbers and no qualified-operand data flows in the golden sections, so these fixes don't move it.
• A first bless attempt ran while the machine's load average was 42. It showed 2,911 diffs, all of them ripple from one python file (fastapi/tests/test_include_router_defaults_overrides.py) hitting a regex timeout under load. An origin/main scan run at the same time showed the same timeout. I discarded that bless.

Sibling bug-sweep PRs also re-bless golden masters, so a golden conflict after one of them merges is expected. Re-bless after merging main (crucible_check.py --update --yes or rebase_rebless.py); never hand-merge golden files.

pr_gates

python tests/tools/pr_gates.py (full): lint, audits, secrets, xray, gauntlet and golden all PASS. suite: 13,937 passed, 3 failed. All 3 failures are pre-existing, listed below.

Failing on origin/main too, so not caused by this branch: tests/cobol_mainframe/test_det_programs.py::{test_typed_lifts_only_what_every_use_allows, test_a_move_that_can_leave_negative_zero_keeps_the_item_bytes, test_typed_groups_sync_a_groups_bytes_around_its_whole_uses}.

Related

• EPSCSMRD stays in excluded_programs (from Estate 4: IBM DBB MortgageApplication -- EPSNBRVL and EPSCMORT proven; group USING items across CALL, nested COPY, CBL from column 1 #4216). answer key: model multi-program sources (sibling programs / END PROGRAM) #4206 (PR fix(answer-key): model multi-program sources (sibling / nested programs) (#4206) #4226, merged) made the answer key able to hold multi-program sources, but onboarding EPSCSMRD needs the forge to read multi-program sources per program first (about 1,000 ledger mismatches otherwise). That is follow-up work, not this PR.

Closes #4204
Closes #4205
Part of #4203 (the false-positive half; PROGRAM-ID-as-unit is left open for a design call, see above)

🤖 Generated with Claude Code

https://claude.ai/code/session_013sNMtAg6s9dyyjNgnjRHi8

[github-actions]

🐦‍⬛ Muninn Security Scan

✅ No security issues found.

🐦‍⬛ Powered by Muninn · Skald Lab

[squid-protocol]

Correction: the 3 tests/cobol_mainframe/test_det_programs.py failures listed above as pre-existing on main (test_typed_lifts_only_what_every_use_allows, test_a_move_that_can_leave_negative_zero_keeps_the_item_bytes, test_typed_groups_sync_a_groups_bytes_around_its_whole_uses) were measured against c0a7e90, which predates #4220. #4220 updated those tests to the one-line Field shape from #4202 and made CI install the translator's parser, so they no longer skip. All 3 pass on current main (0f1be1b). No follow-up issue needed.