Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/language_status/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ The other COBOL-to-Java pages here: [`det_port_design.md`](det_port_design.md) (
runtime contract and every proven result), [`det_survey.md`](det_survey.md) (how much of six estates it translates),
[`statement_inventory.md`](statement_inventory.md) (what a deterministic translator could take over, by statement
kind), [`porting_loop.md`](porting_loop.md) (model-written ports), [`mutation_testing.md`](mutation_testing.md) and
[`cics_crucible.md`](cics_crucible.md). The wiki's summary is `docs/wiki/05-19-proven-cobol-to-java-ports.md`.
[`cics_crucible.md`](cics_crucible.md). [`estate4_dbb_mortgage.md`](estate4_dbb_mortgage.md): the fourth estate (IBM DBB MortgageApplication), what was proven and what onboarding it needed. The wiki's summary is `docs/wiki/05-19-proven-cobol-to-java-ports.md`.

## Signature-bearing languages (47)

Expand Down
21 changes: 12 additions & 9 deletions docs/language_status/cics_field_testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,37 +21,37 @@ counted in it (their fact counts are not recorded here).
| field | tested on: public | private | fresh | facts (public) | engine defects | forge defects | clean fresh | clean fresh facts | bound | status | needs |
|---|---|---|---|---|---|---|---|---|---|---|---|
| units of work and handlers | 6 | 0 | 3 | 2,492 | 0 | 0 | 3 | 2,111 | 0.1% | field-tested | - |
| data moves | 6 | 0 | 3 | 12,583 | 2 | 0 | 2 | 517 | 0.6% | field-tested | - |
| BMS screen fields | 3 | 0 | 1 | 1,865 | 0 | 0 | 1 | 286 | 1.0% | open | 1 more clean fresh round(s), 14 more clean fresh facts |
| JCL job flow | 5 | 0 | 2 | 1,620 | 1 | 0 | 1 | 261 | 1.1% | open | 1 more clean fresh round(s), 39 more clean fresh facts |
| CSD resources | 4 | 0 | 2 | 457 | 0 | 0 | 2 | 223 | 1.3% | open | 77 more clean fresh facts |
| JCL resolved DSNs | 5 | 0 | 2 | 752 | 1 | 0 | 1 | 102 | 2.9% | open | 1 more clean fresh round(s), 198 more clean fresh facts |
| MOVE truncation | 6 | 0 | 3 | 165 | 0 | 0 | 3 | 73 | 4.1% | open | 227 more clean fresh facts |
| file I/O moves | 3 | 0 | 1 | 247 | 0 | 0 | 1 | 38 | 7.9% | open | 1 more clean fresh round(s), 262 more clean fresh facts |
| dead | 4 | 0 | 2 | 143 | 0 | 6 | 2 | 34 | 8.8% | open | 266 more clean fresh facts |
| dead | 5 | 0 | 3 | 144 | 0 | 6 | 3 | 35 | 8.6% | open | 265 more clean fresh facts |
| entry transactions | 4 | 0 | 2 | 70 | 0 | 0 | 2 | 30 | 10.0% | open | 270 more clean fresh facts |
| DD names | 4 | 0 | 1 | 89 | 0 | 0 | 1 | 22 | 13.6% | open | 1 more clean fresh round(s), 278 more clean fresh facts |
| units | 7 | 0 | 4 | 2,058 | 6 | 3 | 1 | 21 | 14.3% | open | 1 more clean fresh round(s), 279 more clean fresh facts |
| DB2 table access | 3 | 0 | 1 | 58 | 0 | 0 | 1 | 21 | 14.3% | open | 1 more clean fresh round(s), 279 more clean fresh facts |
| dynamic call targets | 5 | 0 | 2 | 232 | 0 | 0 | 2 | 16 | 18.8% | open | 284 more clean fresh facts |
| inputs | 3 | 0 | 1 | 55 | 0 | 2 | 1 | 13 | 23.1% | open | 1 more clean fresh round(s), 287 more clean fresh facts |
| dead (non-trivial) | 4 | 0 | 2 | 27 | 0 | 0 | 2 | 9 | 33.3% | open | 291 more clean fresh facts |
| dead (non-trivial) | 5 | 0 | 3 | 28 | 0 | 0 | 3 | 10 | 30.0% | open | 290 more clean fresh facts |
| copybook paths | 6 | 0 | 3 | 431 | 3 | 1 | 1 | 9 | 33.3% | open | 1 more clean fresh round(s), 291 more clean fresh facts |
| outputs | 4 | 0 | 1 | 37 | 0 | 1 | 1 | 9 | 33.3% | open | 1 more clean fresh round(s), 291 more clean fresh facts |
| program_id | 7 | 0 | 4 | 128 | 2 | 0 | 1 | 5 | 60.0% | open | 1 more clean fresh round(s), 295 more clean fresh facts |
| CALL USING | 5 | 0 | 2 | 103 | 0 | 0 | 2 | 5 | 60.0% | open | 295 more clean fresh facts |
| VSAM defines | 3 | 0 | 1 | 30 | 0 | 0 | 1 | 4 | 75.0% | open | 1 more clean fresh round(s), 296 more clean fresh facts |
| call targets | 7 | 0 | 4 | 144 | 1 | 1 | 1 | 2 | - | open | 1 more clean fresh round(s), 298 more clean fresh facts |
| dynamic CALLs | 3 | 0 | 2 | 5 | 0 | 0 | 2 | 2 | - | open | 298 more clean fresh facts |
| DL/I calls | 2 | 0 | 1 | 36 | 0 | 0 | 1 | 1 | - | open | 1 more clean fresh round(s), 299 more clean fresh facts |
| IMS access check | 2 | 0 | 1 | 15 | 0 | 0 | 1 | 1 | - | open | 1 more clean fresh round(s), 299 more clean fresh facts |
| IMS segment access | 2 | 0 | 1 | 21 | 0 | 0 | 1 | 1 | - | open | 1 more clean fresh round(s), 299 more clean fresh facts |
| async children | 2 | 0 | 1 | 6 | 0 | 0 | 1 | 1 | - | open | 1 more clean fresh round(s), 299 more clean fresh facts |
| dynamic CALLs | 2 | 0 | 1 | 4 | 0 | 0 | 1 | 1 | - | open | 1 more clean fresh round(s), 299 more clean fresh facts |
| call targets | 6 | 0 | 3 | 142 | 1 | 1 | 0 | 0 | - | open | 2 more clean fresh round(s), 300 more clean fresh facts |
| program_id | 6 | 0 | 3 | 123 | 2 | 0 | 0 | 0 | - | open | 2 more clean fresh round(s), 300 more clean fresh facts |
| MOVE truncation | 6 | 0 | 3 | 165 | 1 | 0 | 0 | 0 | - | open | 2 more clean fresh round(s), 300 more clean fresh facts |
| data moves | 6 | 0 | 3 | 12,583 | 3 | 0 | 0 | 0 | - | open | 2 more clean fresh round(s), 300 more clean fresh facts |
| record fields | 6 | 0 | 3 | 5,475 | 3 | 2 | 0 | 0 | - | open | 2 more clean fresh round(s), 300 more clean fresh facts |
| units | 6 | 0 | 3 | 2,037 | 6 | 3 | 0 | 0 | - | open | 2 more clean fresh round(s), 300 more clean fresh facts |
| CICS RIDFLD | 5 | 0 | 0 | 781 | 0 | 0 | 0 | 0 | - | open | 2 more clean fresh round(s), 300 more clean fresh facts |
| CICS resources | 5 | 0 | 3 | 3,588 | 1 | 0 | 0 | 0 | - | open | 2 more clean fresh round(s), 300 more clean fresh facts |
| CICS task control | 5 | 0 | 3 | 206 | 1 | 0 | 0 | 0 | - | open | 2 more clean fresh round(s), 300 more clean fresh facts |
| copybook layouts | 5 | 0 | 0 | 2,759 | 1 | 0 | 0 | 0 | - | open | 2 more clean fresh round(s), 300 more clean fresh facts |
| copybook paths | 5 | 0 | 2 | 422 | 3 | 1 | 0 | 0 | - | open | 2 more clean fresh round(s), 300 more clean fresh facts |
| refmod spans | 5 | 0 | 0 | 373 | 1 | 0 | 0 | 0 | - | open | 2 more clean fresh round(s), 300 more clean fresh facts |
| JCL runner programs | 4 | 0 | 0 | 86 | 0 | 0 | 0 | 0 | - | open | 2 more clean fresh round(s), 300 more clean fresh facts |
| file control | 4 | 0 | 1 | 89 | 1 | 3 | 0 | 0 | - | open | 2 more clean fresh round(s), 300 more clean fresh facts |
Expand Down Expand Up @@ -79,6 +79,7 @@ counted in it (their fact counts are not recorded here).
| 4 | cics-genapp | public | 2,294 | 11 | 4 | 1 | 0 |
| 5 | zecs | public | 1,293 | 43 | 0 | 0 | 2 |
| 6 | dsf | public | 2,018 | 4 | 7 | 1 | 0 |
| 7 | dbb-mortgage-application | public | 45 | 0 | 2 | 0 | 0 |

## Defect log

Expand Down Expand Up @@ -125,6 +126,8 @@ counted in it (their fact counts are not recorded here).
| D035 | review, after 6 | forge | fact | inputs, outputs, file control | #3998 | The refractor without `--scan` read SELECT with its own regex (cobol_dag_architect): `ASSIGN TO "x"` gave DD `TO`, `SELECT OPTIONAL f` and Japanese / full-width file names were missed, so lineage differed from `--scan`. It now uses the engine's SELECT reader. Found while fixing #3992, not by the estate scan. |
| D036 | unkeyed estate, after 6 | forge | fact | units, dead | #4026 / #4028 | cobol_graveyard_finder's unit reader took only ASCII names, so Japanese / full-width paragraphs and sections (13 opensourcecobol4j programs) were not units, and it rejected any `END-...` name as a scope terminator, so the legal paragraph `END-IPROC1.` was not one either: those units were neither counted nor judged dead or alive. A GnuCOBOL trace entered them. |
| D037 | 4 | engine | fact | units | #4031 | The engine's COBOL function-start rule excludes every END-... word as a scope terminator, so GENAPP's legal paragraph `End-Program.` (5 programs, reached by PERFORM END-PROGRAM) is not a unit. Surfaced when #4028 fixed the same rule in the forge and the key reader, and the key gained the paragraph. |
| D038 | 7 | engine | fact | MOVE truncation | #4204 | A MOVE whose source is qualified (EPSPCOM-ERRMSG OF W-COMMUNICATION-AREA, 80 bytes, into the BMS field MSGERRO) is recorded but its truncation is not flagged: the sizer does not resolve the qualified source's length. IBM DBB EPSCMORT L231. |
| D039 | 7 | engine | fact | data moves | #4205 | MOVE ALL X'..' (a hexadecimal figurative) is not recorded as a data move: IBM DBB EPSCSMRD L1457 (ALL X'00') and L3452 (ALL X'FF'). |

Key errors (the census's findings against the answer key itself) are counted from each key's
rulings, not logged by hand. The engine agreeing with a key is only as good as the key: a key
Expand Down
8 changes: 8 additions & 0 deletions docs/language_status/det_port_design.md
Original file line number Diff line number Diff line change
Expand Up @@ -319,6 +319,14 @@ up to and including the LINK (its events and the COMMAREA bytes it passes, byte
their error paths, ABNDPROC's DUPREC among them. No case here browses, so EBCDIC vs ASCII key order is still not
exercised.

### Estate 4: IBM DBB MortgageApplication (2026-10-03)

A fourth estate, from a different IBM team: [estate4_dbb_mortgage.md](estate4_dbb_mortgage.md). 99.0% of its 204
statements translate. EPSNBRVL (a CALL with a group USING item; 22/22 branches) and EPSCMORT (CICS, a generated BMS map,
Db2) are proven. EPSMPMT is not proven on purpose (`NUMPROC(MIG)`, C5), and EPSMLIST waits for ESDS browse by RBA
(#4213). Onboarding it needed general fixes: compiler cards from column 1, `ID DIVISION`, group USING items across a
CALL as contract DTOs, and nested COPY in a copybook's record (the IR).

## The combined method (B): deterministic first, a model refactors under proof

**B1 -- structured style** (`det_port.py --style structured`, no model). A program with no GO TO, no EXEC CICS HANDLE
Expand Down
82 changes: 82 additions & 0 deletions docs/language_status/estate4_dbb_mortgage.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# Estate 4: IBM DBB MortgageApplication

*2026-10-03. A fourth public COBOL/CICS estate, from a different IBM team than CBSA and GenApp, onboarded to test how
far the deterministic translator and the equivalence harness generalise.*

## Why this estate

| candidate | licence | COBOL | verdict |
|---|---|---|---|
| **IBM DBB MortgageApplication** (`IBM/dbb`, `zBuilder/MortgageApplication`) | Apache-2.0 | 6 sources: CALLed subprograms, a CICS/BMS/Db2 screen, a VSAM browse, a generated web-service wrapper | **chosen** |
| Galasa SimBank (`galasa-dev/simplatform`) | EPL-2.0 | no COBOL source in the repository (a simulator; one JCL skeleton) | nothing to translate |
| Rocket / Micro Focus BankDemo | Rocket EULA ("internal demonstration purposes with other Rocket products") | 81 sources | licence does not allow redistributing results or fixtures |

MortgageApplication is small but different in kind: lower-case members, `CBL` cards starting in the sequence area,
`ID DIVISION`, a `PROGRAM-ID` without its period, group USING items, copybooks that COPY other copybooks, a symbolic map
the repository does not ship, Db2's own `SYSIBM.SYSDUMMY1`, an ESDS browsed by RBA, and one source holding nine
programs. Corpus `dbb-mortgage-application` (`tests/cobol_mainframe/corpora.json`), pinned at `ce2c1be5`.

## Ground truth

The answer key (`tests/cobol_mainframe/answer_key/dbb-mortgage-application.json`) covers 5 programs. A blind second
model (Gemini, via `agy`, given only the brief: paths in the public corpus, no key) answered all 45 census questions
the same way as the key (`cross_verify.py census`). EPSCSMRD is left out: nine sibling programs in one source, which
the key cannot yet represent (#4206). The ground-truth ledger triages the engine's 13 disagreements: 10 are the
deliberate unreferenced-by-name census (#2806), and 3 are engine defects, now filed: a qualified MOVE source's
truncation (#4204) and `MOVE ALL X'..'` (#4205).

## Translation survey

`tests/tools/det_survey.py --corpus dbb-mortgage-application`: **202 of 204 statements (99.0%) translate
deterministically.** Of 6 programs, 3 translate whole and 2 with one hole each: EPSCSMRT's dynamic `CALL
WS-CALLED-PROGRAM` and EPSMPMT's `FUNCTION ANNUITY`, in A300-TRY2, which is dead code. EPSCSMRD is refused: its
DATA DIVISION is nine programs'. Before the fixes below, all 6 were refused.

## Cases

| case | program | kind | result | coverage |
|---|---|---|---|---|
| `mortgage-nbrvl` | EPSNBRVL, number validator | CALL, group USING item | **proven**, 15 calls | 3/3 paragraphs, 22/22 branches |
| `mortgage-cmort` | EPSCMORT, mortgage screen | CICS, BMS (generated map), Db2 `SYSIBM.SYSDUMMY1`, CALL EPSNBRVL | **proven**, 7 scenarios (6 written, plus the automatic SQL-fault task on its SELECT, #4200) | 3/6 paragraphs, 12/21 branches |
| `mortgage-mpmt` | EPSMPMT, payment calculator | CALL, binary arguments as field values | **not proven on purpose**: `CBL NUMPROC(MIG)` (C5) | — |

**Not reached, and why:**
- EPSCMORT's ENTER (indicator '3') and WHEN OTHER paths LINK EPSCSMRT, which, like EPSMPMT, compiles with
`NUMPROC(MIG)` (C5).
- Its PF9 and ENTER-from-list paths LINK EPSMLIST, whose ESDS browse by RBA the CICS model refuses (#4213).
- EPSMLIST has no case for the same reason.
- EPSCSMRD has no case: nine programs in one source, LE condition handlers, channels and containers.

## What it took

**Det translator:**
- Compiler-option cards are read by the engine's own card reader. A card may start in any column from 1:
` CBL NUMPROC(MIG)` sits in columns 4–6.
- `ID DIVISION.` reads as IDENTIFICATION DIVISION.
- A group USING item crosses a CALL as the call forge's contract DTO, through the COMMAREA codec a LINK uses: the
callee's `handleCall(EpsNumberValidation)`, and the caller's CALL site builds the DTO from its storage and writes the
filled object back (BY REFERENCE). `CobolRef` is imported only where a CALL passes text.

**IR:** a COPY that follows a copied elementary root inside its copybook expands as that root's siblings. EPSMTCOM
is `10 PROCESS-INDICATOR` then `COPY EPSMTINP.` and `COPY EPSMTOUT.`; the COMMAREA DTO was 1 byte and is now the whole
106-byte record.

**Harness (CICS):** the generator emits the CICS exception package only when some program throws or handles one,
and this estate has none. The CICS equivalence test now catches `CicsAbendException` only when the project has it.

**Harness:**
- A CALL case may declare a group USING item's `record`. The Java side gets a DTO built field by field and returns
it, compared by value as a LINKed COMMAREA is. An argument may be given as field values (binary and packed inputs),
set on the COBOL side with hexadecimal MOVEs.
- Copybooks are staged under their member names, so `COPY EPSNBRPM` finds `epsnbrpm.cpy`.
- IBM's compiler assumes the period after a `PROGRAM-ID` name; GnuCOBOL does not, so the harness supplies it.
- A screen may name a symbolic map generated from the corpus's BMS (`@bms/EPSMORT.cpy`), on both sides.
- X12 (register): EPSCMORT copies DFHCOMMAREA before it tests EIBCALEN. In its first task, the COMMAREA fields that
came from nowhere are undefined, so they are left out and counted: 1 field.

## Issues filed

- #4204: engine MOVE truncation misses a qualified source.
- #4205: engine data moves miss `MOVE ALL X'..'`.
- #4206: the answer key can't hold multi-program sources.
- #4213: ESDS browse by RBA, which blocks EPSMLIST.
19 changes: 16 additions & 3 deletions docs/language_status/oracle_assumptions.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ verdict, but it limits what the proof says about inputs outside the scenarios.
| C2 | compiler | Arithmetic intermediates: exact decimal vs IBM's precision rules | ASSUMED | yes (INTCALC, POSTTRAN …) |
| C3 | compiler | An integer literal truncated to zero keeps no sign | DIFFERS | no |
| C4 | compiler | An unsigned binary taken below zero by ADD/SUBTRACT wraps | DIFFERS | no |
| C5 | compiler | `INTDATE(LILIAN)`, `ARITH(EXTEND)`, `NUMPROC(PFD)`, `TRUNC(OPT)` | REFUSED | — |
| C5 | compiler | `INTDATE(LILIAN)`, `ARITH(EXTEND)`, `NUMPROC(PFD)`, `NUMPROC(MIG)`, `TRUNC(OPT)` | REFUSED | — |
| C6 | compiler | COMP-1 / COMP-2: IEEE vs IBM hexadecimal floating point | DIFFERS | no (CBSA uses them) |
| C7 | compiler | COMP-5 byte order: little-endian vs z/OS big-endian | DIFFERS | read as numbers only |
| C8 | compiler | DISPLAY of signed zoned, binary and packed items | MATCHED | yes |
Expand All @@ -59,6 +59,7 @@ verdict, but it limits what the proof says about inputs outside the scenarios.
| X9 | CICS | Named counters (GET COUNTER) | MATCHED | yes (GenApp LGACDB01) |
| X10 | CICS | A LINK target's COMMAREA bytes past the end of the caller's record | DIFFERS | no |
| X11 | CICS | ASKTIME ABSTIME into a field narrower than S9(15) COMP-3 (GenApp's WS-ABSTIME) | DIFFERS | yes (GenApp error paths, #4173) |
| X12 | CICS | A task with no COMMAREA that MOVEs DFHCOMMAREA anyway | UNDEFINED, masked | yes (DBB EPSCMORT) |
| L1 | LE | CEEDAYS: documented pictures only | MATCHED / REFUSED | yes |
| L2 | LE | CEE3ABD abend codes | MATCHED | yes |
| L3 | LE | WORKING-STORAGE with no VALUE clause: GnuCOBOL's spaces vs LE's STORAGE option on z/OS | ASSUMED | yes (CardDemo READACCT OUTFILE, 2 bytes) |
Expand Down Expand Up @@ -113,8 +114,10 @@ verdict, but it limits what the proof says about inputs outside the scenarios.
- **Reached.** No case reaches it. `test_det_programs.py` keeps its unsigned item above zero.

### C5. Options GnuCOBOL cannot honour — REFUSED
- **What.** `INTDATE(LILIAN)`, `ARITH(EXTEND)`, `NUMPROC(PFD)` and `TRUNC(OPT)`, from a CBL/PROCESS card or a case's
`compiler_options`, stop the run (`UnsupportedOption`).
- **What.** `INTDATE(LILIAN)`, `ARITH(EXTEND)`, `NUMPROC(PFD)`, `NUMPROC(MIG)` and `TRUNC(OPT)`, from a CBL/PROCESS
card or a case's `compiler_options`, stop the run (`UnsupportedOption`).
- **NUMPROC(MIG)** (OS/VS COBOL migration sign processing): IBM DBB MortgageApplication's EPSMPMT and EPSCSMRT carry
`CBL NUMPROC(MIG)`. Their cases (`mortgage-mpmt`) are written and listed in KNOWN_UNPROVEN until an oracle models it.
- **Note.** CBSA's build JCL passes `TRUNC(OPT)`, but its programs' PROCESS cards override it with `TRUNC(STD)` (C1).
- `TRUNC(OPT)` is undefined for out-of-range values by IBM's own description, so no oracle could be faithful to it.

Expand Down Expand Up @@ -310,6 +313,16 @@ The length READQ TS returns on ITEMERR or QIDERR is not documented, so it is not
messages is not z/OS's.
- **Reached.** Yes, by GenApp's SQL-fault tasks (M2).

### X12. DFHCOMMAREA referenced with EIBCALEN = 0 — UNDEFINED, masked
- **What.** IBM DBB EPSCMORT does `MOVE DFHCOMMAREA TO W-COMMUNICATION-AREA` before it tests EIBCALEN, so its first
task (no COMMAREA) copies storage it was never given. On z/OS that is undefined: CICS establishes no addressability
for an absent COMMAREA, and what the MOVE reads (or whether it abends) depends on the region.
- **Harness.** The CICS model gives LOW-VALUES there. A numeric field the task then returns still holding all
LOW-VALUES was set by nothing (a MOVE leaves digits, never X'00'), and the Java side's COMMAREA DTO cannot hold an
invalid number. In a scenario with `"commarea": null` only, such a field is left out of the comparison on both
sides and counted (`undefined_commarea_fields` in the report): `equivalence_cics.mask_absent_commarea`. Every other
field of that COMMAREA, and every field of every other scenario, is compared as usual.

## Language Environment

### L1. CEEDAYS — MATCHED where documented, REFUSED otherwise
Expand Down
Loading
Loading