Repository navigation
rosetta[cobol]: cross-language consistency — 4 red / 8 amber metrics, 5 ledgered shapes #2567
Description
Part of #2560 — per-language tracking for the keyword-rosetta control corpus (built for #1096). Planted intent is identical in every language, so every deviation below is measured bias vs. the 46-language median, validated in the corpus's deviation ledger. 🔴 = beyond ±50% of the cross-language median, 🟡 = ±25–50%.
Triage order (per the epic): structure detection first, then signal/keyword rules, then risk-score consequences — §3's numbers are downstream of §1–§2 and re-baseline as those fix. Cross-cutting engine issues (e.g. #2535/#2546) fix many languages at once; this issue tracks this language's residual after each cross-cutting fix lands, and closes when the language sits inside the ±25% band on every comparable metric (or every remaining deviation is ledgered as intended morphology).
1. Structure detection (fix first)
- 🔴
args: 1 vs median 13 (-92%)
2. Signal / keyword deviations
- 🔴
fragile_debt: 3 vs median 1 (+200%) - 🔴
planned_debt: 2 vs median 1 (+100%) - 🔴
state_mutation: 2 vs median 6 (-67%) - 🟡
comment_lines: 15 vs median 30 (-50%) - 🟡
globals: 3 vs median 2 (+50%) - 🟡
high_risk_execution: 3 vs median 2 (+50%) - 🟡
io: 4 vs median 3 (+33%) - 🟡
keyword_hits: 178 vs median 247.5 (-28%)
3. Risk-exposure consequences (downstream)
- 🟡
risk_api_exposure: 6.81696 vs median 5.31104 (+28%) - 🟡
risk_cognitive_load: 8.11636 vs median 15.1444 (-46%) - 🟡
risk_safety_score: 53.1215 vs median 37.297 (+42%)
Ledgered shapes touching cobol
| defect | type | issue | evidence | summary |
|---|---|---|---|---|
census-requires-git-tracked |
engine-semantic | — | — | GalaxyScope's census enumerates git-tracked files only; an untracked folder scans as '0 files mapped' with no per-file warning |
cobol-working-storage-globals |
keyword-overlap | — | (notes) | cobol globals pattern includes WORKING-STORAGE\s+SECTION as an alternative — every fixed-format program with a working-storage section records globals >= 1 |
cobol-hyphen-identifier-debt-leak |
upstream-bug | #2537 | c.cpy | Debt rules are whole-text, not comment-anchored, and '-' is a regex word boundary: HACK-LEVEL (data item, 2 code occurrences) counts fragile_debt, PROBE-TODO (paragraph … |
cobol-area-b-bare-token-paragraph |
upstream-question | #2538 | (notes) | A bare token + period in Area B (CEE3DMP |
string-literal-selective-shielding |
upstream-question | #2535 | a.cpy | Keywords inside one string literal ('if eval fails, try open') count toward branch (+2), safety (+1), io (+1) but eval does NOT count toward high_risk_execution (routes … |
Decoy outcomes (file-level evidence)
- main.cbl ·
never ALTER this flow and no PERFORM loop(comment): stripped: contributed 0 - a.cpy ·
DISPLAY 'IF ALTER FAILS TRY OPEN AGAIN'(string): ALL counted: IF (branch +1), ALTER (high_risk +1), OPEN (io +1) — cobol does NOT shield the high-risk family in literals, unlike python/javascript. Ledger: string-literal-selective-shielding; gitgalaxy#2535 comment - c.cpy ·
77 HACK-LEVEL PIC 9. / DISPLAY HACK-LEVEL.(code): COUNTED (+2): hyphen is a word boundary so \bHACK\b matches inside the identifier, both occurrences; the PROBE-TODO paragraph name likewise adds planned_debt +1. Ledger: cobol-hyphen-identifier-debt-leak; gitgalaxy#2537
Reproduce / evidence
- Corpus folder:
data/cobol/—expected_signals.jsoncarries the shape-by-shape accounting - Gate:
GALAXYSCOPE_BIN=<venv>/bin/galaxyscope python tools/verify_language.py cobolin the corpus repo - Full context: findings_by_language.md#cobol · bias_report.md · bias chart
Activity
squid-protocol commentedon Aug 31, 2026
Sweep complete (rosetta-language-sweep): 6🔴 / 4🟡 → 3🔴 / 4🟡
Worked per the corpus's
rosetta-language-sweepskill. Note the livelanguage_deviations.pytable had already superseded this issue's title (4🔴/8🟡): the post-#2618 api re-baseline moved
keyword_hitsandrisk_api_exposurein-band and re-bandedcomment_lines/risk_cognitive_loadto red before this sweep started — the sweep's true starting point was 6🔴/4🟡.
The five-bucket classification
1. Real engine bug — fixed.
fragile_debt+200% /planned_debt+100% = #2537: the shareddebt patterns' bare
\b(...)\b+--as-word-boundary matched debt keywords inside hyphenatedidentifiers (
HACK-LEVEL×2, thePROBE-TODOparagraph name; scheme'sprobe-todosymbolreproduced it). Fixed by PR #2622 (hyphen-adjacency guards on
GLOBAL_FRAGILE_DEBT/GLOBAL_PLANNED_DEBT; golden masters re-blessed — the same FP class died corpus-wide, e.g. anSSN mask
xxx-xx-xxxxcounted asXXXand aroles-path-bugdirectory name counted ×4).cobol → 1/1 exact, scheme planned_debt → 1. Re-baselined in keyword-rosetta PR #10.
2. Missing rule w/ morphology. None — cobol's rule surface was already the registry's deepest.
3. Corpus authoring gap.
comment_lines14 vs 30 (−53%, lowest of all 46): the cobol shellswere the only corpus language authored with zero blank lines (fortran/ada siblings carry ~12;
blanks are legal COBOL). Re-authored with idiomatic spacing + neutral banners (signal placement
unchanged) → 33, in-band (PR #10).
4. Intended morphology — ledgered.
args1 vs 13: new validated entrycobol-args-clause-match-morphology. Bucket-3 test runfirst (the jcl
PARM=precedent): verified empirically the file-level metric countsUSING/RETURNING clause matches, not captured names — a 13-name
PROCEDURE DIVISION USINGstill records
struct_args1 — and every per-probe alternative (CALL/ENTRY/INVOKE … USING) is also anapikeyword → overlap-cost rejection (jclDISP=precedent).globals+50%:WORKING-STORAGE SECTIONin the globals rule — deliberate shared-statedesign, already ledgered (
cobol-working-storage-globals); keeping the rule confirmed.5. Median inflation / cross-cutting — cobol is the honest one.
state_mutation2 vs 6: matches planted intent exactly; median inflated by state-flux ×3 branch-context weighting: undocumented, and phantom-triggered by branch keywords inside string literals #2546's ×3 flux.high_risk_execution+50% /io+33%: the a.cpy string decoy (ALTER/OPENin a DISPLAYliteral) — blocked on prism/detector: string-literal shielding is selective by signal family (if/try/open count inside a literal, eval does not) #2535's selective-shielding design question.
risk_cognitive_load−57% /risk_safety_score+42% are downstream shadows; re-baselineas prism/detector: string-literal shielding is selective by signal family (if/try/open count inside a literal, eval does not) #2535/kotlin: branch rule counts 'return' — inconsistent with java/swift/c siblings #2545/state-flux ×3 branch-context weighting: undocumented, and phantom-triggered by branch keywords inside string literals #2546 land.
Artifacts
priority: highper the COBOL/JCL convention).ENGINE_REF=pull/2622/head; cobol gate 75 assertions PASS,scheme 72 PASS; bias artifacts regenerated — the regen also folds in stale fix(detector): exclude synthetic slicer bucket names from orphan/duplicate census (#2547) #2618 cache drift,
documented in the PR).
docs/language_status/cobol.md§10).Close criterion
Every remaining deviation is ledgered as intended morphology (
args,globals) or blocked on anamed cross-cutting issue (
state_mutation→ #2546;high_risk_execution/io→ #2535; §3 →downstream). Per the epic, that satisfies the close condition — recommending close once #2622 and
rosetta PR #10 merge; closing is your call.
docs(language-status): cobol §10 rosetta consistency capstone (#2567 …
squid-protocol commentedon Sep 1, 2026
Closing per the sweep summary above: 6🔴/4🟡 → 3🔴/4🟡, and every remaining deviation is ledgered as intended morphology (args → cobol-args-clause-match-morphology, globals → cobol-working-storage-globals) or blocked on a named cross-cutting issue (state_mutation → #2546, high_risk_execution/io → #2535, §3 → downstream re-baseline). Choreography landed: gitgalaxy#2622 + #2623 merged, keyword-rosetta#10 merged, KEYWORD_ROSETTA_REF → 89911a8, rosetta-audit green on main against the new pin.
squid-protocol commentedon Sep 3, 2026
Status: 0🔴 / 1🟡 unexplained, 8 further out-of-band cell(s) accounted for
Generated from
docs/bias_data.json+deviation_ledger.jsonbytools/issue_status.py, using the same verdict machinerybias_report.py --gateruns on (gitgalaxy#2669 E.1), so this comment and the gate cannot disagree about what is left.Still unexplained — the work this issue tracks
encapsulation_ratioOut of band, but accounted for
derived — the deviation entered through an input that is itself out of band -- the same finding counted twice
structural_mass— inherits argsledgered — a validated ledger entry names this language and this metric
args— cobol-args-clause-match-morphology, args-no-parameter-surface-morphologyavg_func_args— args-no-parameter-surface-morphologyavg_func_loc— function-body-length-morphologycontrol_flow_ratio— control-flow-ratio-denominator-is-a-vocabulary-tallyfunc_internal_density— function-body-length-morphologyglobals— cobol-working-storage-globalsrisk_safety_score— path-and-extension-modifiersClose criterion (gitgalaxy#2669 Batch E.4)
python tools/language_deviations.py cobolexits 1: 1 unexplained cell(s) remain. Each needs a fix, or a validated ledger entry saying why it is correct, before this issue can close.squid-protocol commentedon Sep 5, 2026
Reopen flag resolved (E.4 had noted this issue was closed pre-E.1 but no longer passed the machine gate): as of keyword-rosetta#54's n/a sweep,
language_deviations.py cobolexits 0 again — every cobol deviation carries a recorded reason. No reopen owed; F.6's loop reopens automatically on future drift.