[squid-protocol]

Part of #3834 (epic #3819). Parts 1 (#3907) and 2 (#3936) are merged. This part adds dimension D (target DB collation), dimension E (input data) and the rest of dimension C (runtime locales). Only the gauntlet changes: tests/tools/cultural_gauntlet.py, tests/cultural_gauntlet/. No engine code.

D: target database collation

Why: VSAM returns records in the order of the key's bytes in the data page: lower case before upper case, letters before digits. Target databases order differently. MySQL's utf8mb4_0900_ai_ci even treats a, A and á as equal.

The estate (rows collation_*) is a PIC X(8) KSDS. A CICS program browses it (STARTBR / READNEXT / READPREV) and a batch step reads it sequentially. It is generated for H2, PostgreSQL and MySQL, in every column.

collation-static (always, in CI) checks #3822's contract:

• Key and sort columns compare byte by byte: COLLATE "C" on PostgreSQL, utf8mb4_bin on MySQL, H2's own code-point default.
• The @PrePersist sort key uses the declared page.
• The finders order by the sort column, and the services pass sortKey(from, page).
• A sequential READ comes back in key order.
• The ticket carries the key-order rule.

collation-db (opt-in, --db) runs against real databases, but only the ones this machine has:

1. Creates the table from the generated entity, rendered as Hibernate's ddl-auto would render its @Column mappings.
2. Writes 22 keys through the generated CobolRecords, with sortKey filling the sort column as @PrePersist would. The keys cover mixed case, ÆØÅ, Ä, Ñ, é, digits and letters, punctuation, and ABC next to ABC.
3. Reads them back with the generated finders' SQL and with findAll().

The oracle is computed here, independently of the converter: each key space-padded and encoded in the page, sorted by bytes. ABC is a duplicate WRITE. A canary checks that the database's own ORDER BY CUST_KEY differs from the mainframe's order, so the keys can actually catch a wrong collation.

What really ran:

• H2 2.2.224, from ~/.m2.
• PostgreSQL 16 as a throwaway cluster (initdb, trust auth, localhost on a free port, stopped on exit), with one database created LOCALE 'C' and one LOCALE 'en_US.UTF-8'. Each cell reports the collation the database names itself.
• MySQL: not run. There is no MySQL server or docker image here, so those cells are skipped and say so. Its utf8mb4_bin is checked statically only.

Result:

• Every browse matches the mainframe on H2, PostgreSQL C and PostgreSQL en_US.UTF-8, in all 8 columns: from LOW-VALUES, from a key, backwards, and the duplicate.
• Bug Generated batch sequential READ of a KSDS is findAll(): records come back in database order, not key order #3945: the batch step's sequential READ is a bare findAll() with no ORDER BY. PostgreSQL returns heap order and H2 returns primary-key code-point order, not the mainframe's order. Ledgered: 32 collation-static cells and 24 collation-db cells.

E: input data

The estate (rows input_eur, input_usa, input_iso, one per DB2 DATE format) has:

• a KSDS whose record has DATUM PIC 9(8) and FECHA PIC X(10) next to ACCT-DATE PIC 9(8);
• a DB2 table with DATE columns DATUM and FECHA, TIME ZEIT, and ACCT_DATE CHAR(10);
• COMPUTE ... ROUNDED and ROUNDED MODE NEAREST-EVEN on the same product;
• FUNCTION NUMVAL.

input-static checks:

• Fields are typed by PICTURE and SQL type, not by the language of their name: the codec, the entity, the clean-room schema, the DB2 row (LocalDate datum / fecha, String acctDate), and the repository's Db2Dates note.
• Db2Dates.FORMAT is the declared format.
• The ticket's rounding is HALF_UP / HALF_EVEN, and the ticket carries the NUMVAL rule (Java runtime accepts non-ASCII digits as numbers (Arabic-Indic, Devanagari, full-width) where COBOL NUMERIC/NUMVAL would not #3831).

runtime-run (--run) checks the generated Db2Dates and CobolRecords in all 15 JVM environments against literals:

• DB2's EUR / USA / ISO character forms (26.09.2026 14.30.05, 09/26/2026 02:30 PM, 2026-09-26 14.30.05), and parsing all three.
• ROUNDED ties: 0.025 -> 0.03 / 0.02, -0.025 -> -0.03 / -0.02, 2.5 / -2.5 / 3.5 -> 3 -3 4 / 2 -2 4.
• NUMVAL, decimal() and zoned reads of full-width, Arabic-Indic, Thai and Devanagari digits must be invalid. This includes numbers printed by a th-TH-u-nu-thai / ar-EG formatter and a UTF-8 record's full-width DATUM.

The COBOL answers were decided by running GnuCOBOL 3 -std=ibm (the harness image) and are pinned in the tool:

• ROUNDED ties: 3 / -3; NEAREST-EVEN: 2 / -2.
• TEST-NUMVAL('１２３') and TEST-NUMVAL('١٢٣') return 1.
• The NUMERIC class test is false.
• Arithmetic on such a zoned field is "not numeric" under -debug (IBM S0C7).

Result:

• Everything passes and is identical across all 15 environments, except one probe.
• Bug Generated Db2Dates.parseTimestamp accepts Arabic-Indic / full-width digits in the time (Integer.parseInt) #3946: Db2Dates.parseTimestamp("2026-09-26-١٤.٣٠.٠٥...") parses, because Integer.parseInt takes Arabic-Indic digits; DB2 rejects that string. Ledgered: 24 runtime-run cells.

C: the remaining runtime locales

• New --run environments: tr-TR, ar-EG, th-TH-u-nu-thai and de-DE, each with the JVM zone set to UTC, Asia/Kolkata and Europe/Berlin (12 new, 15 in all). PINNED_CLOCK is Berlin's spring-forward day, so a clock that used the JVM zone would read 03:30+0200.
• Thai digits: jvm_flags now passes -Duser.extensions=u-nu-thai. Before, the Thai environment silently ran as plain th-TH with ASCII digits. The canary now checks that Thai digits appear, and prints the Calendar year (2569 under Thai).
• New runtime-static traps:
  • DateTimeFormatter.ofPattern without a Locale, when the pattern has a text or week field. A numeric-only pattern is locale-proof in java.time, and --run confirms that under th/ar for MainframeClock's yyyyMMddHHmmssSSZ.
  • ofLocalized*
  • SimpleDateFormat without a Locale
  • Collator.getInstance()
  • Character.isDigit / digit / getNumericValue
• Results: no generated code trips the new traps. Every seed, D and E cell stays identical across all 15 environments.

Cells (after #3940)

run	cells pass	skipped	ledgered (all part 3)	time
--ci (sampled)	185/189	14	4	7.5 s
--full --ci	549/581	91	32	26 s
--full --run --ci	616/672	104	56	72 s
--full --db --ci	549/605	99	56	31 s
--full --run --db --ci	616/696	112	80	77 s

• All five exit 0, and parts 1 and 2 are fully green after #3933 / #3910 / #3911: floating currency editing, PICTURE SYMBOL precision, decimal-comma VALUEs -- the India slice passes #3940.
• The 8 skipped collation-db cells are the MySQL ones.
• The baseline is exactly #3933 / #3910 / #3911: floating currency editing, PICTURE SYMBOL precision, decimal-comma VALUEs -- the India slice passes #3940's empty one plus the 80 part-3 cells (Generated batch sequential READ of a KSDS is findAll(): records come back in database order, not key order #3945: 56, Generated Db2Dates.parseTimestamp accepts Arabic-Indic / full-width digits in the time (Integer.parseInt) #3946: 24).
• The sampled CI run went from about 6.9 s to 7.5 s on 12 cores, so +0.7 s.

Gates

• pr_gates.py --fast: 5/5 pass (lint, audits, secrets, xray, gauntlet 186 s).
• pytest tests/cultural_gauntlet/: 123 passed in 21.3 s (main: 103 in 11.5 s). The added time is mostly the real H2 and PostgreSQL runs and the 15-environment E run; each skips cleanly where its database or JDK is missing.

Not done / limits

• MySQL ai_ci is not executed, because there is no server here. Its cells say so.
• The DDL is the harness's rendering of the generated @Columns, not Hibernate itself.
• Trailing spaces are tested through the codec path (keys arrive space-padded, as fromRecord reads them). sortKey itself does not pad, so a port that passes a trimmed key to READPREV would miss the equal record. That is a porting-contract question, not ledgered.
• The seed cells are still not executed under GnuCOBOL; E's COBOL answers are pinned from one GnuCOBOL run.

🤖 Generated with Claude Code

[github-actions]

🐦‍⬛ Muninn Security Scan

✅ No security issues found.

🐦‍⬛ Powered by Muninn · Skald Lab