fix(cobol): sentence-anchored paragraph headers (#3197) + census identifier lexicon (#3198)

[squid-protocol]

Closes #3197, closes #3198. Part of #3122. Engine PR, with a golden-master bless.

#3197 — a paragraph header begins a sentence

func_start accepted any lone NAME. indented into Area B, so the last line of a multi-line statement or data description became a paragraph, and split the real paragraph's span while it was at it.

The proposed fix was file-level fixed/free-format detection (#2538's deferred item). The measurement says it isn't needed, and that an Area-A column anchor would be wrong. Over language-crucible v1.3.0, keyword-rosetta and all three pinned corpora:

	Area A (cols 8-11)	Area B (12+)
previous sentence ended	10,766 real headers	299 real headers, e.g. crucible CBL0601v01InOutLineLoop.cbl puts every paragraph in column 13
previous sentence still open	0	656, every one a statement/declaration tail

So the rule is the language's own: a header can only start where the previous sentence ended. It is format-independent, and it keeps the Area-B paragraphs that accepted source contains (a compiler warns and carries on).

The deciding context is the previous line, which a Python lookbehind cannot span, so this is a registry-declared scope filter (_scope_filters: {"func_start": "cobol_sentence_start"}), honoured by both func_start consumers: the count in coding_analysis and the unit list in _slice_by_labels. A filter only one of them honoured would make the same file report two different paragraph counts (#2753's unfiltered-consumer trap); a test asserts they agree.

One extra phantom shape came with it: an IDENTIFICATION/ENVIRONMENT operand on its own line (PROGRAM-ID. then COACTUPC., OBJECT-COMPUTER. then XXXXX083.). The period ends a sentence, so the operand needs its own keyword list.

#3198 — the census judges occurrences by the language's lexicon

docs/unreferenced_by_name_contract.md settles most of this issue. Corollary 3 already says the census means "nothing else in this file names this unit", and explicitly not "dead". So the issue's "fuller" option (a reachability pass) would change what the signal means, not fix it. What IS broken is the name test:

• COBOL names are case-insensitive: perform a-para did not name A-PARA.
• - is a COBOL name character: B-PARA-EXIT counted as a mention of B-PARA, so a paragraph read as referenced because a different paragraph's name began with its name (13 units across the corpora).

Two optional top-level keys carry it, both defaulting to today's reading, so every language that declares neither is unchanged by construction:

"identifier_case": "insensitive",
"identifier_extra_chars": "-",

cobol is the only language that declares either. The other case-insensitive languages have the same defect latent; declaring a key moves that language's census and needs its own evidence, so they are audited per language in #3225. A closed-set test pins the family, the same way invocation_model is pinned.

The issue's micro-repro now reads:

unit	before	after	why
MAIN-PARA	1	1	entry, named nowhere — correct, and not a dead-code claim
A-PARA	1	0	perform a-para is a reference
B-PARA	0	1	only ever named as the prefix of B-PARA-EXIT
B-PARA-EXIT	0	0	PERFORMed
C-PARA	1	1	unchanged

#3120's dead-code switch is re-scoped, not pending

The entry paragraph stays flagged, because nothing names it. usage_status therefore cannot be COBOL dead code under its own contract. The engine supplies the unit inventory (now exact) and the forge keeps reachability, which the answer key scores 62/62 · 10/10 non-trivial on CBSA. A future engine-side reachability signal would be a new field beside usage_status, with its own issue and evidence — not a redefinition. Recorded in docs/refraction_engine_differential.md.

Measured (answer key, engine column)

python tests/tools/mainframe_corpus.py score (#3213), before → after:

field	zopeneditor-sample	CBSA
units	P 58/67 → 58/58 · R 58/58	P 680/706 → 680/680 · R 680/680
usage_status claims	5 → 4	248 → 247

Recall was already complete on both, so the phantoms were pure precision loss.

The bless

crucible_check.py --update --yes, after scoping the uncapped diff (the tool's own output is capped twice):

• 4,196 raw differences; 2,333 are 3D topological coordinates (the layout re-solves corpus-wide when any node's mass changes).
• All 1,863 real differences are inside the cobol language group, plus the corpus-wide aggregate rows that necessarily follow (ecosystem summary, archetype mix, health averages).
• Direction: Method Declarations down in all 183 moved files; Unreferenced By Name 122 down / 44 up (the hyphen fix flags, the case fix and the phantom removal clear).
• Hand-verified on BANKDATA.cbl, 36 → 28 units: every dropped one is an OCCURS … DEPENDING ON operand, a MOVE … TO target or a DIVIDE … REMAINDER operand. No real paragraph is lost.

Verification

check	result
pytest tests/	10270 passed, 7 skipped, 9 xfailed, 3 xpassed
crucible_check.py (both fixtures, after bless)	PASS / PASS
rosetta_audit.py	51 languages, 0 regressions — no corpus re-bless owed
tri_comparison_chart.py --all --ci	all OK (3 languages)
tree_sitter_accuracy_audit.py --ci --all	all OK (30 languages; cobol has no grammar)
audit_check.py	all clear (ruff, mypy, dead-key, ast-accuracy)
refraction_snapshot.py check (+ all 3 full corpora)	0 differ — an engine change must not move forge output

New tests: test_cobol_sentence_paragraphs_3197.py (free format, fixed format with a column-7 continuation, real Area-B paragraphs, four statement-tail shapes, count/unit-list agreement, and the span no longer truncating). test_unreferenced_by_name_contract_2806.py gains corollary 7's rows. test_orphan_hyphen_fast_path_3182.py is updated: its parity mechanism targeted the old admission guard and would have passed vacuously, and its fixture's census is now asserted outright.

🤖 Generated with Claude Code

[github-actions]

🐦‍⬛ Muninn Security Scan

✅ No security issues found.

🐦‍⬛ Powered by Muninn · Skald Lab