Refractor without --scan: cobol_dag_architect reads ASSIGN TO "x" as DD 'TO', misses SELECT OPTIONAL and non-ASCII file names (lineage differs from --scan) #3998

Found while fixing #3992 (PR #3994), and left out of that PR. It affects the refractor without --scan. There, cobol_dag_architect supplies the program's file lineage (analysis.lineage.inputs/outputs) from its own SELECT reader, not the engine's.

Defect

_SELECT in gitgalaxy/tools/cobol_to_cobol/cobol_dag_architect.py:36:

_SELECT = re.compile(rf"{_V}SELECT\s+([A-Z0-9\-]+)\s+ASSIGN\s+(?:TO\s+)?([A-Z0-9@#$\-]+)")

• ASSIGN TO "x" reads TO as the DD name. Quoted text is blanked before matching, so (?:TO\s+)? backtracks and TO itself matches the name class.
• SELECT OPTIONAL f is missed. OPTIONAL is taken as the file name, so the real file never maps and it drops out of the lineage.
• Non-ASCII file names are missed. The class is [A-Z0-9\-], so Japanese names are not read, even though COBOL: Japanese user-defined words are invisible -- a Japanese PROGRAM-ID drops the program from refraction; data names truncated or missing; sections missing #3955 / COBOL: full-width hyphens (－ U+FF0D / − U+2212) and full-width digits still truncate Japanese data names and drop sections (#3955 follow-up) #3991 made the engine accept them.

The engine's own reader (core/file_control.py, used with --scan) and, since #3994, analyze_cobol_intent's files_requested get all three right. So the same program gets different lineage depending on whether --scan is passed.

Minimal reproduction (origin/main 71d33c5)

mkdir -p estate
cat > estate/ASGTO.cbl <<'COB'
       IDENTIFICATION DIVISION.
       PROGRAM-ID. ASGTO.
       ENVIRONMENT DIVISION.
       INPUT-OUTPUT SECTION.
       FILE-CONTROL.
           SELECT IN-FILE ASSIGN TO "./IN-FILE"
                  ORGANIZATION LINE SEQUENTIAL.
           SELECT OPTIONAL OUT-FILE ASSIGN TO OUTDD.
           SELECT 顧客ファイル ASSIGN TO CUSTDD.
       DATA DIVISION.
       FILE SECTION.
       FD IN-FILE.
       01 IN-REC PIC X(10).
       FD OUT-FILE.
       01 OUT-REC PIC X(10).
       FD 顧客ファイル.
       01 顧客レコード PIC X(10).
       PROCEDURE DIVISION.
           OPEN INPUT IN-FILE 顧客ファイル OUTPUT OUT-FILE.
           READ IN-FILE.
           WRITE OUT-REC FROM IN-REC.
           CLOSE IN-FILE OUT-FILE 顧客ファイル.
           STOP RUN.
COB
python -c "import sys; sys.argv=['r','estate']; from gitgalaxy.cobol_refractor_controller import main; main()"
python -c "import json,glob; print(json.load(open(glob.glob('estate_gitgalaxy_clean_*/04_ir_state_dumps/*_ir.json')[0]))['analysis']['lineage'])"

Actual (no --scan): inputs: ['TO'], outputs: []
With --scan: inputs: ['./IN-FILE', 'CUSTDD'], outputs: ['OUTDD'] (expected)

Expected

Both paths give the same lineage. The cleanest fix is probably for the no-scan path to use the engine's SELECT reader (core/file_control.py) rather than keep a second regex. Otherwise _SELECT has to read the ASSIGN target before literals are blanked, accept OPTIONAL, never take TO as a name, and use the COBOL word class (national, Japanese, full-width). A test should check that --scan and no-scan lineage agree on these SELECT forms.

🤖 Generated with Claude Code