Skip to content

Answer key: copybook record layouts + CICS RIDFLD, two new ledger fields (#3602, #3649 part 1) - #3672

Merged
squid-protocol merged 3 commits into
mainfrom
key/layouts-3602-3649
Sep 25, 2026
Merged

squid-protocol merged 3 commits into
mainfrom
key/layouts-3602-3649

Conversation

@squid-protocol

@squid-protocol squid-protocol commented Sep 25, 2026 •

Copy link
Copy Markdown
Owner

Closes #3602; part 1 of #3649 (RIDFLD). The refmod-text half of #3649 follows once #3670's engine columns merge.

Why

The Java generators rest on byte layouts: COMMAREA DTOs (#3615 / #3655), VSAM entity keys (#3617). Those layouts mostly live in copybooks, and the answer key only keyed each program's own DATA DIVISION, and only by field name. Nothing independently checked offsets or widths.

New ledger field: copybook layouts

side source
key draft_copybook_layouts: this tool's own _data_items reader + plain storage arithmetic (the same arithmetic that signs symbolic maps against IBM's DFHMAPS output, #3575)
engine GalaxyIR.record_layout per record root

The unit is ROOT/NAME @offset+bytes per elementary PIC item. The contract is the engine's:

  • a REDEFINES overlay is skipped;
  • items without a PIC (POINTER, COMP-1/2) take their storage but are not units;
  • an OCCURS item is listed once, at its first occurrence;
  • an item under a PIC item is not storage.

A copybook that COPYs another member is not keyed, since this reader doesn't expand COPY.

Key-reader errors found and fixed (each to a COBOL rule, not to "whatever the engine says")

file error rule
CardDemo CSUTLDWY.cpy 1 THROUGH 12. (continuing an 88 … VALUES) read as a level-1 item THROUGH reserved words are not data names (the engine fixed the same in #3452)
CBSA PROCISRT.cpy POINTER took 0 bytes and shifted every later field by 4 POINTER / INDEX / COMP-1 = 4, COMP-2 = 8
GENAPP soaipm1.cpy 05 laid out under an elementary 03 … PIC 9(10) an item with a PICTURE has no subordinate storage

_pic_bytes also counts CR/DB (2), E, and national N/G (2 each).

Result

Key and engine agree on all 2,759 units in 113 copybooks: CardDemo 2,131, CBSA 389, GENAPP 188, zOE 34, zECS 17; DSF has no COBOL copybooks.

  • ground_truth_ledger.py update: +0 −0, 0 untriaged; check: clean.
  • The field enters at tier draft.
  • Field-testing registry: development_rounds: 6, because it was built against every keyed estate. It stays untested until a fresh estate or a census; census suite to follow.

Also: CICS RIDFLD (#3649, part 1)

  • Unit: L<line> VERB FILE NAME RIDFLD=OPERAND per CICS FILE command, spacing normalized.
  • Key side: this tool's own EXEC CICS reader, over COBOL, PL/I and assembler sources.
  • Engine side: cics_resource_data.attributes.
  • Separate section, so the reviewed CICS resources units stay untouched.
  • Result: all 781 units agree (CardDemo 46, CBSA 17, GENAPP 7, zECS 22, DSF 689 PL/I). Tier draft, development_rounds 6.

Verified

  • Unit tests for the layout contract (REDEFINES, POINTER / COMP-2, OCCURS, CR, national, THROUGH, an item under a PIC) and for "a COPYing copybook is not keyed".
  • 847 mainframe + recorder tests. ruff / mypy / dead-key / X-Ray clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_017ZsVaAkb86P5r5JXDC2Y9g

squid-protocol and others added 3 commits September 25, 2026 15:23
The layouts the Java generators depend on mostly live in copybooks, which
no key checked. New field `copybook layouts`:
- Key: draft_copybook_layouts lays out every COBOL copybook with this
  tool's own reader and storage arithmetic, as `ROOT/NAME @offset+bytes`
  per elementary PIC item.
- Engine: GalaxyIR.record_layout per root.
- The contract is the engine's: a REDEFINES overlay is skipped; items
  without a PIC take their storage but are not units; an OCCURS item is
  listed once; an item under a PIC item is not storage.
- A copybook that COPYs another member is not keyed (this reader does
  not expand COPY).

Three key-reader errors, found against the corpora and fixed to COBOL's
rules:
- `1 THROUGH 12.` continuing an 88's VALUES was read as a level-1 item
  named THROUGH (CardDemo CSUTLDWY);
- a POINTER took 0 bytes and shifted every later field (CBSA PROCISRT);
  POINTER / INDEX / COMP-1 are 4, COMP-2 8;
- an item under an elementary PIC item was laid out (GENAPP soaipm1).
_pic_bytes also counts CR / DB (2), E, and national N / G (2 each).

Result: key and engine agree on all 2,759 units in 113 copybooks
(CardDemo 2,131, CBSA 389, GENAPP 188, zOE 34, zECS 17; DSF has no COBOL
copybooks). The ledger adds the field at tier draft (update: +0 -0, 0
untriaged; check clean). The field-testing registry marks it
development_rounds 6: it was built against every keyed estate, so it
stays `untested` until fresh estates or a census.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ZsVaAkb86P5r5JXDC2Y9g
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ZsVaAkb86P5r5JXDC2Y9g
The Java repository forge (#3617) maps each CICS FILE command's RIDFLD
onto an entity key, and no key compared it.

New field `CICS RIDFLD`: `L<line> VERB FILE NAME RIDFLD=OPERAND` per
FILE command with a RIDFLD, spacing normalized.
- Key: this tool's own EXEC CICS reader (cics_resource_ops now keeps
  the option), over COBOL, PL/I and assembler sources.
- Engine: cics_resource_data.attributes, nested parentheses included.
It is a section of its own, so the reviewed `CICS resources` units are
untouched.

Key and engine agree on all 781 units: CardDemo 46, CBSA 17, GENAPP 7,
zECS 22, DSF 689 (PL/I). The field enters at tier draft, with
development_rounds 6. Ledger update: +0 -0, 0 untriaged; check clean.

Part 2 (the reference-modification texts) needs #3670's engine columns.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ZsVaAkb86P5r5JXDC2Y9g
@squid-protocol squid-protocol changed the title Answer key: copybook record layouts, a new ledger field (#3602) Answer key: copybook record layouts + CICS RIDFLD, two new ledger fields (#3602, #3649 part 1) Sep 25, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🐦‍⬛ Muninn Security Scan

✅ No security issues found.

🐦‍⬛ Powered by Muninn · Skald Lab

@squid-protocol
squid-protocol merged commit 0b38cfc into main Sep 25, 2026
38 checks passed
@squid-protocol
squid-protocol deleted the key/layouts-3602-3649 branch September 25, 2026 19:43
squid-protocol added a commit that referenced this pull request Sep 25, 2026
…part 2) (#3676)

* Answer key: reference-modification spans, a new ledger field (#3649, part 2)

#3655 made the engine keep each reference modification as written, and
the IR reads a CICS program's COMMAREA from those texts. No key checked
them.

New field `refmod spans`: `L<line> VERB SOURCE(start:length) ->
TARGET(start:length)` per reference-modified move, with one spelling on
both sides (_norm_refmod: whitespace collapsed, no space around
+ - * / : ( )).
- Key: this tool's own data-move reader. _mv_operands now keeps the
  refmod text, the top-level parenthesis group with a colon, so a
  subscript is not a refmod. Its srm / trm flags are unchanged.
- Engine: data_move_data.source_refmod_text / target_refmod_text.
It is a section of its own, so the reviewed `data moves` units are
untouched.

Key and engine agree on all 373 units: CardDemo 174, CBSA 95, GENAPP
61, zECS 42, zOE 1. DSF's PL/I uses SUBSTR, not refmods. Tier draft,
development_rounds 6. Ledger update: +0 -0, 0 untriaged; check clean.

Closes #3649 with part 1 (#3672, RIDFLD).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ZsVaAkb86P5r5JXDC2Y9g

* Field testing: refmod spans introduced by #3676

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ZsVaAkb86P5r5JXDC2Y9g

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Answer key: copybook record layouts are not keyed (only programs' own DATA DIVISION)

1 participant