Skip to content

IMS DL/I slice 1: CALL 'CBLTDLI' through a DB PCB on both runtimes; CardDemo PAUDBUNL proven (Part of #4893) - #5232

Merged
squid-protocol merged 8 commits into
mainfrom
churn/4893-20261016T0744Z
Oct 16, 2026
Merged

squid-protocol merged 8 commits into
mainfrom
churn/4893-20261016T0744Z

Conversation

@squid-protocol

Copy link
Copy Markdown
Owner

Part of #4893 (Epic: IMS runtime), slice 1: DL/I database calls (CALL 'CBLTDLI') through a DB PCB, with SSAs and status codes, on both runtimes and against a stated DBD / PSB.

What changed

  • Oracle side. tests/equivalence/ims/ggdli.c is a new C model linked into a batch case's cobc build. It carries CBLTDLI, plus GGDLINIT / GGDLIEND for the driver.
  • Det runtime. cobolrt/dli/DetDli.java is its Java twin: the same rules and the same refusals, by name.
  • Translator.
    • det/dli.py translates CALL 'CBLTDLI' to DetDli.call and sets RETURN-CODE 0 after it (register I1).
    • stmt.py: ENTRY 'DLITCBL' as the first statement is the program's entry even when PROCEDURE DIVISION has its own USING, because IMS enters there.
    • program.py: runBatch binds the PCBs (DetDli.enter, no PARM written over PCB 1) and writes the databases at the end (DetDli.leave).
    • AIBTDLI and CEETDLI are refused by name.
  • Harness.
    • tests/tools/equivalence_ims.py handles a case's "ims" section. It reads the DBD / PSB sources through the engine's own core/ims_gen.py, and refuses by name what is not modelled.
    • The seed is the corpus's own HD unload. It is laid out per segment copybook and checked against the unload's trailer counts.
    • The databases are compared segment by segment.
    • Hooks were added in equivalence.py and equivalence_java.py. evidence.py gets harness:ims / oracle:ims components and a Target.ims flag.
  • ACCEPT FROM DATE / DATE YYYYMMDD / DAY / DAY YYYYDDD / TIME (cobolrt/Accept.java). The translator had emitted Funcs.acceptDate(clock()), a method that never existed, so any batch port with ACCEPT FROM DATE did not compile (dormant since Deterministic COBOL→Java ports: all 23 proven CardDemo programs translate with no model and prove #4089). PAUDBUNL needs it.
  • New case carddemo-paudbunl (CardDemo, burned):
    • It is job UNLDPADB STEP01 (DLI region, PSB PAUTBUNL), which unloads DBPAUTP0 with GN / GNP into OUTFIL1 / OUTFIL2.
    • Every input is the corpus's own data: its DBPAUTP0 unload, 22 roots and 202 children.
    • It has four file-fault runs.
  • Register family I (new): I1 database model and status codes; I2 sequence / qualification compare (DIFFERS for text keys, declared like the KSDS collation); I3 position and parentage; I4 the PCB mask; I5 refusals; I6 reading the HD unload.
  • Other records:
    • rule family DLI (9 rules) and RT-FUNC-ACCEPT-DATE in cobolrt_rules.json
    • MODELLED_CALLS += CBLTDLI
    • a DL/I section in det_port_design.md

Evidence

  • Level fixture first (owner rule 5). tests/equivalence/ims/fixture holds LVLDBD (three levels: DEPT, EMP, SKILL, PROJ, with C / X / P fields), LVLPSB (one PCB with PROCOPT=A, one with G), DLILVL (49 calls) and expected.txt.
    • The expected output was written from IBM's documentation and committed in aef4ca4 ("written before either runtime ran it") before any run.
    • First-time number: on its first run both runtimes printed all 51 lines exactly (2 mask lines plus 49 calls) and left byte-identical databases.
    • The two attempts before that failed to build the Java test harness (package name); neither runtime ran.
    • test_det_dli.py::test_the_level_fixture_prints_what_ibm_documents_on_both_runtimes (EQUIVALENCE_E2E=1).
  • carddemo-paudbunl proven by the det sweep: "proven on 5 runs, covering 9/10 paragraphs and 20/22 branches".
    • OUTFIL1 21/21 and OUTFIL2 202/202 records equal.
    • IMS DBPAUTP0 224/224 segments equal.
    • SYSOUT 26/26 lines equal; RETURN-CODE 0 / 0.
    • The coverage-ledger entry and the duration are committed.
  • det_port.py check vs origin/main: 112/113 ports unchanged; the only one is the new carddemo-paudbunl, re-proved with the final translator (PROVED, 73/73 statements). ACCEPT FROM DATE lives in its own runtime class (cobolrt/Accept.java), so no port that does not ACCEPT moves. A first cut in Funcs.java moved all 113.
  • Gates: pr_gates --fast 6/6 pass; pr_gates --ratchets 8 pass, 0 fail, 0 skipped. EQUIVALENCE_E2E=1 pytest tests/cobol_mainframe/test_det_dli.py 34 passed; test_evidence_components.py, test_cobolrt_rules.py, test_det_source_4462.py and test_det_coverage_ledger.py pass.

Class: a DL/I database call through a DB PCB (the CBLTDLI call interface, batch DLITCBL entry)

member status evidence
GU, GHU, GN, GHN, GNP, GHNP handled level fixture steps 1-20, 25, 28, 30, 40-49; PAUDBUNL
ISRT (root, path, by position) handled steps 21-24
REPL, DLET (with hold) handled steps 26-31
unqualified / qualified SSAs (EQ GT LT GE LE NE, symbolic forms; C / X / P) handled steps 1, 14-17
status blank GA GK GE GB GP II DJ DA AC AJ AK AD AM handled expected.txt
PCB mask feedback handled (I4 assumes what IMS leaves after an unsuccessful call) every step
ACCEPT FROM DATE / DAY / TIME (a sibling the case needed) handled Accept.from, PAUDBUNL, unit test
everything in I5 refused by name translator / harness / runtime tests

Sibling gaps (not in this PR)

  • EXEC DLI, the command-level interface with the DIB:

    • CardDemo CBPAUP0C (batch BMP: GN, GNP, DLET, CHKP)
    • CardDemo COPAUA0C, COPAUS0C, COPAUS1C (CICS: SCHD / TERM, GU / GNP / ISRT / REPL)

    It needs an EXEC DLI translator and a DIB model. The CICS ones also need equivalence_cics.py to link the model, which this slice keeps out of (COBOL translator: CICS command breadth (START/RETRIEVE, channels/containers, GETMAIN, LOAD, CONVERSE, INQUIRE, ASSIGN, counters, HANDLE ABEND PROGRAM) #4270, Epic: IBM MQ runtime — MQI calls (MQOPEN/GET/PUT/CLOSE…) and MQ structures #4892).

  • SCHD / TERM, CHKP / XRST / ROLB / SYNC: CBPAUP0C (CHKP), COPAUA0C / COPAUS* (SCHD / TERM). These are a unit-of-work and checkpoint model.

  • GSAM PCBs: CardDemo DBUNLDGS needs a GSAM model (a sequential dataset behind a PCB).

  • I/O PCB, CMPAT=YES, BMP, IMS TM and MFS: CardDemo PAUDBLOD (PSB PSBPAUTB CMPAT=YES, BMP). Its ENTRY 'DLITCBL' USING PAUTBPCB names one PCB where IMS passes the I/O PCB first.

  • Command codes, boolean SSAs, AIBTDLI / CEETDLI, FLD / POS, multiple positioning: no burned program uses them.

  • Secondary indexes (PROCSEQ), logical relationships, HDAM root order, load mode (PROCOPT L): DBPAUTX0 is the HIDAM primary index, which is not needed here. Nothing burned uses the rest.

  • CBLTDLI from a CICS program: it needs equivalence_cics.py to link ims/ggdli.c.

Notes

  • No engine or golden change: nothing under gitgalaxy/core, standards or metrics is touched.
  • field_testing.json is not touched. The ACCEPT FROM DATE defect is the det translator's, not the engine's or the forge's.
  • The plan named a separate tests/tools/test_equivalence_ims.py. Its harness tests are in tests/cobol_mainframe/test_det_dli.py with the rest.

🤖 Generated with Claude Code

squid-protocol and others added 7 commits October 16, 2026 04:05
)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…before either runtime ran it (Part of #4893)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…; level fixture passes on both runtimes (Part of #4893)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…Part of #4893)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…(Part of #4893)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@squid-protocol

Copy link
Copy Markdown
Owner Author

No description provided.

@squid-protocol
squid-protocol marked this pull request as ready for review October 16, 2026 08:40
@squid-protocol
squid-protocol enabled auto-merge (squash) October 16, 2026 08:43

def call(gen: Any, s: S.Stmt, ind: str) -> list[str]:
"""CALL 'CBLTDLI' USING function pcb io-area [ssa-1 ... ssa-15]: DetDli.call, then RETURN-CODE 0."""
from gitgalaxy.tools.cobol_to_java.det.gen import Untranslatable
from pathlib import Path
from typing import TYPE_CHECKING, Any

from gitgalaxy.tools.cobol_to_java.det import dli
@github-actions

Copy link
Copy Markdown
Contributor

🐦‍⬛ Muninn Security Scan

✅ No security issues found.

🐦‍⬛ Powered by Muninn · Skald Lab

@squid-protocol

Copy link
Copy Markdown
Owner Author

PR #5232 @ f38661f: 2 failed check(s)

== ground-truth (failure) -- real
https://github.com/squid-protocol/gitgalaxy/actions/runs/38875289735/job/116894886060
repro: python tests/tools/pr_gates.py --ratchets --only-ratchets ground-truth
errors:
##[error]Process completed with exit code 1.
log tail:
Commit: f3dc54e1fb89da6852db922499a0233137369e1d
Build Date: 2026-10-12T17:48:21Z
Worker ID: {b13e49c1-64dc-4ca0-814b-a3bb3c74914f}
Region: westus2
Cloud: Azure
##[endgroup]
##[group]Operating System
Ubuntu
24.04.5
LTS
##[endgroup]
##[group]Runner Image
Image: ubuntu-24.04
Version: 20261012.334.1
Included Software: https://github.com/actions/runner-images/blob/ubuntu24/20261012.334/images/ubuntu/Ubuntu2404-Readme.md
Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu24%2F20261012.334
##[endgroup]
##[group]GITHUB_TOKEN Permissions
Contents: read
Metadata: read
##[endgroup]
Secret source: Actions
Cache mode: write
Prepare workflow directory
Prepare all required actions
Complete job name: ground-truth
##[group]Run echo "ground-truth parts: $PARTS (tested-tree skip: $SKIP, plan mode: $MODE)"
echo "ground-truth parts: $PARTS (tested-tree skip: $SKIP, plan mode: $MODE)"
# #4935: "skipped" is a pass only when the gate proved this tree green on the PR; otherwise it is a failure
# #5164: ... or when the plan found no ground-truth input in the PR (mode none). An empty mode is not none.
test "$PARTS" = "success" || { test "$PARTS" = "skipped" && { test "$SKIP" = "true" || test "$MODE" = "none"; }; }
shell: /usr/bin/bash -e {0}
env:
PARTS: failure
SKIP:
MODE: full
##[endgroup]
ground-truth parts: failure (tested-tree skip: , plan mode: full)
##[error]Process completed with exit code 1.
Cleaning up orphan processes

== ground-truth part 1/2 (failure) -- real
https://github.com/squid-protocol/gitgalaxy/actions/runs/38875289735/job/116890195260
repro: python tests/tools/pr_gates.py --fast
failing tests: tests/cobol_mainframe/test_java_subprocess_utf8.py::test_no_java_subprocess_in_tests_decodes_with_locale_charset
errors:
fail-on-cache-miss: false
E AssertionError: tests/cobol_mainframe/test_det_dli.py:303: java missing -Dfile.encoding=UTF-8
tests/cobol_mainframe/test_java_subprocess_utf8.py:193: AssertionError
FAILED tests/cobol_mainframe/test_java_subprocess_utf8.py::test_no_java_subprocess_in_tests_decodes_with_locale_charset - AssertionError: tests/cobol_mainframe/test_det_dli.py:303: java missing -Dfile.encoding=UTF-8
##[error]Process completed with exit code 1.
log tail:
11.65s call tests/cobol_mainframe/test_det_programs.py::test_all_literal_relation_emits_compilable_java_with_and_without_a_collating_sequence
11.16s call tests/cobol_mainframe/test_trunc_opt.py::test_a_value_past_the_picture_stops_by_name[MOVE P0 TO UW MULTIPLY 100000 BY UW-9876500000 into a 9-digit binary item-True]
11.14s call tests/cobol_mainframe/test_trunc_opt.py::test_a_value_past_the_picture_stops_by_name[MOVE 99999 TO SH-99999 into a signed 4-digit binary item-True]
10.83s call tests/cobol_mainframe/test_java_subprocess_utf8.py::test_no_maven_subprocess_in_tests_decodes_with_locale_charset
10.66s call tests/cobol_mainframe/test_trunc_opt.py::test_a_value_past_the_picture_stops_by_name[MOVE 9999 TO SH MOVE SH TO NS ADD 1 TO NS MOVE NS TO SH-10000 into a signed 4-digit binary item-True]
10.58s call tests/cobol_mainframe/test_trunc_opt.py::test_a_value_past_the_picture_stops_by_name[MOVE P0 TO UW MULTIPLY 100000 BY UW-9876500000 into a 9-digit binary item-False]
10.07s call tests/cobol_mainframe/test_trunc_opt.py::test_a_value_past_the_picture_stops_by_name[COMPUTE SH = P0-98765 into a signed 4-digit binary item-False]
10.02s call tests/cobol_mainframe/test_trunc_opt.py::test_a_value_past_the_picture_stops_by_name[MOVE 99999 TO SH-99999 into a signed 4-digit binary item-False]
9.99s call tests/cobol_mainframe/test_lifted_binary_trunc.py::test_typed_binary_stores_equal_bytes_and_the_oracle[literal-BIN]
9.69s call tests/cobol_mainframe/test_trunc_opt.py::test_a_value_past_the_picture_stops_by_name[COMPUTE SH = P0-98765 into a signed 4-digit binary item-True]
9.49s call tests/cobol_mainframe/test_trunc_opt.py::test_a_value_past_the_picture_stops_by_name[MOVE 9999 TO SH MOVE SH TO NS ADD 1 TO NS MOVE NS TO SH-10000 into a signed 4-digit binary item-False]
9.45s call tests/cobol_mainframe/test_tool_regex_redos.py::test_catches_a_mild_quadratic_by_growth
9.37s call tests/cobol_mainframe/test_lifted_binary_trunc.py::test_a_synced_groups_bytes_are_not_re_truncated[STD]
9.13s call tests/cobol_mainframe/test_uow_transactional.py::test_uow_transactional
9.12s call tests/cobol_mainframe/test_uow_transactional.py::test_uow_transactional_plain_classes
=========================== short test summary info ============================
FAILED tests/cobol_mainframe/test_java_subprocess_utf8.py::test_no_java_subprocess_in_tests_decodes_with_locale_charset - AssertionError: tests/cobol_mainframe/test_det_dli.py:303: java missing -Dfile.encoding=UTF-8
tests/cobol_mainframe/test_det_dli.py:303: java missing -Dstdout.encoding=UTF-8
tests/cobol_mainframe/test_det_dli.py:358: java missing -Dfile.encoding=UTF-8
tests/cobol_mainframe/test_det_dli.py:358: java missing -Dstdout.encoding=UTF-8
tests/cobol_mainframe/test_det_dli.py:358: java/javac uses text= without encoding='utf-8'; capture bytes and decode, or pass encoding
assert not ['tests/cobol_mainframe/test_det_dli.py:303: java missing -Dfile.encoding=UTF-8', 'tests/cobol_mainframe/test_det_dli....frame/test_det_dli.py:358: java/javac uses text= without encoding='utf-8'; capture bytes and decode, or pass
1 failed, 4048 passed, 562 skipped, 1 warning in 425.91s (0:07:05)
##[error]Process completed with exit code 1.
Post job cleanup.
[command]/usr/bin/git version
git version 2.55.0
Temporarily overriding HOME='/home/runner/work/_temp/89a7523e-78b4-487a-aa35-a0d75f53df11' before making global git config changes
Adding repository directory to the temporary git global config as a safe directory
[command]/usr/bin/git config --global --add safe.directory /home/runner/work/gitgalaxy/gitgalaxy/gitgalaxy
Removing SSH command configuration
[command]/usr/bin/git config --local --name-only --get-regexp core.sshCommand
[command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
Removing HTTP extra header
[command]/usr/bin/git config --local --name-only --get-regexp http.https://github.com/.extraheader
[command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http.https://github.com/.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :"
Removing includeIf entries pointing to credentials config files
[command]/usr/bin/git config --local --name-only --get-regexp ^includeIf.gitdir:
[command]/usr/bin/git submodule foreach --recursive git config --local --show-origin --name-only --get-regexp remote.origin.url
Cleaning up orphan processes

@squid-protocol squid-protocol added the shepherd:needs-fix Bot: a real failure on this head; see the digest comment. Guide: docs/ci.md label Oct 16, 2026
…8, output decoded as UTF-8 (Part of #4893)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@squid-protocol squid-protocol added shepherd:merge Maintainer: merge when green (auto-merge, squash). Guide: docs/ci.md and removed shepherd:merge Maintainer: merge when green (auto-merge, squash). Guide: docs/ci.md shepherd:needs-fix Bot: a real failure on this head; see the digest comment. Guide: docs/ci.md labels Oct 16, 2026
@squid-protocol

Copy link
Copy Markdown
Owner Author

No description provided.

1 similar comment
@squid-protocol

Copy link
Copy Markdown
Owner Author

No description provided.

@squid-protocol
squid-protocol merged commit ea3b513 into main Oct 16, 2026
57 checks passed
@squid-protocol
squid-protocol deleted the churn/4893-20261016T0744Z branch October 16, 2026 09:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

shepherd:merge Maintainer: merge when green (auto-merge, squash). Guide: docs/ci.md

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants