Skip to content

Target config: data.code_page silently accepts non-EBCDIC pages (shift_jis -> katakana 'sign characters'); a PC-COBOL (ASCII / Shift_JIS) estate cannot be declared #3958

Closed

Description

@squid-protocol

Found while preparing the opensourcecobol4j fresh estate (#3806, epic #3803). This is a pre-trial finding; no porting was run.

Defect

data.code_page is documented as "the EBCDIC code page for zoned-decimal sign overpunch". validate() checks it with zoned_sign_characters(), which says it rejects anything that is "not a known EBCDIC code page". In fact it only checks codecs.lookup(cp), so any Python codec passes silently, and the "sign characters" are whatever that codec maps 0xC0-0xC9 / 0xD0-0xD9 to:

>>> zoned_sign_characters("shift_jis")
('タチツテトナニヌネノ', 'ミムメモヤユヨラリル')      # half-width katakana, not sign characters
>>> zoned_sign_characters("cp1252")
('ÀÁÂÃÄÅÆÇÈÉ', 'ÐÑÒÓÔÕÖרÙ')

CobolRecords gets those characters baked in (records_source), and EbcdicDecoderUtil / sortKey use the page as if it were EBCDIC. So a config naming the ASCII-family page an estate really uses produces wrong zoned-decimal codecs, and nothing warns about it. The estate here is opensourcecobol4j, GnuCOBOL or Micro Focus data: Shift_JIS or UTF-8 text with ASCII zoned decimals.

It also means the Japanese PC estate (#3806) cannot be declared honestly. There is no way to say "the data is Shift_JIS, zoned decimals are ASCII (0x30-0x39, with the dialect's negative convention)". The only accepted values are EBCDIC pages, or a non-EBCDIC page that is silently misread.

Minimal reproduction (origin/main 3f8bdb6)

import tempfile
from pathlib import Path
from gitgalaxy.tools.cobol_to_java.java_target import load_target
p = Path(tempfile.mkdtemp()) / "t.yml"
p.write_text("data:\n  code_page: shift_jis\n")
print(load_target(p).data.code_page)      # -> shift_jis   (accepted; expected: ConfigError, or an ASCII-family mode)
p.write_text("data:\n  code_page: nosuch\n")
load_target(p)                            # -> ConfigError: not a known EBCDIC code page  (only unknown codecs fail)

Expected

Either:

  • reject non-EBCDIC pages with the error the code already promises; or
  • (better, for PC-COBOL estates) support an ASCII-family data encoding: the text codec (shift_jis / cp932 / utf-8 / cp1252), ASCII zoned-decimal signs (the dialect's own convention), and ASCII byte-order collation.

🤖 Generated with Claude Code

https://claude.ai/code/session_01TmgE1CwLgaqxqKEgQn5Dw5

Activity

  1. added 2 commits that reference this issue on Sep 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugUnintended behavior or logic failure in the enginelegacy-modernizationCOBOL refractor, dead-code extraction, and JCL forgingpriority: medium

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions