Repository navigation
rosetta[m4]: cross-language consistency — 8 red / 6 amber metrics, 4 ledgered shapes #2585
Description
Part of #2560 — per-language tracking for the keyword-rosetta control corpus (built for #1096). Planted intent is identical in every language, so every deviation below is measured bias vs. the 46-language median, validated in the corpus's deviation ledger. 🔴 = beyond ±50% of the cross-language median, 🟡 = ±25–50%.
Triage order (per the epic): structure detection first, then signal/keyword rules, then risk-score consequences — §3's numbers are downstream of §1–§2 and re-baseline as those fix. Cross-cutting engine issues (e.g. #2535/#2546) fix many languages at once; this issue tracks this language's residual after each cross-cutting fix lands, and closes when the language sits inside the ±25% band on every comparable metric (or every remaining deviation is ledgered as intended morphology).
1. Structure detection (fix first)
- 🔴
dependency_links: 0 vs median 3 (-100%) - 🟡
args: 18 vs median 13 (+38%)
2. Signal / keyword deviations
- 🟡
branch: 3 vs median 5 (-40%) - 🟡
comment_lines: 21 vs median 30 (-30%) - 🟡
keyword_hits: 174 vs median 247.5 (-30%) - 🟡
state_mutation: 4 vs median 6 (-33%)
3. Risk-exposure consequences (downstream)
- 🔴
risk_api_exposure: 0 vs median 5.31104 (-100%) - 🔴
risk_cognitive_load: 4 vs median 15.1444 (-74%) - 🔴
risk_documentation: 22.9356 vs median 57.1208 (-60%) - 🔴
risk_safety_score: 57.4043 vs median 37.297 (+54%) - 🔴
risk_spec_match: 41.3333 vs median 96 (-57%) - 🔴
risk_state_flux: 40 vs median 19.9999 (+100%) - 🔴
risk_tech_debt: 80 vs median 39.9576 (+100%) - 🟡
risk_verification: 1.28776 vs median 2.3628 (-45%)
Ledgered shapes touching m4
| defect | type | issue | evidence | summary |
|---|---|---|---|---|
census-requires-git-tracked |
engine-semantic | — | — | GalaxyScope's census enumerates git-tracked files only; an untracked folder scans as '0 files mapped' with no per-file warning |
batch4-dual-keyword-overlaps |
keyword-overlap | — | — | Collective entry for batch 4's finditer-confirmed dual-rule keywords, each detailed in its language's manifest notes: agc RESUME (cleanup+branch), livecode global (globa… |
no-dependency-capture-languages |
upstream-question | — | — | m4 and scheme have _dependency_capture = None: include()/(import ...) count the import signal but can never produce a DAG edge, so popularity stays 0 and orphans never c… |
string-literal-selective-shielding |
upstream-question | #2535 | — | Keywords inside one string literal ('if eval fails, try open') count toward branch (+2), safety (+1), io (+1) but eval does NOT count toward high_risk_execution (routes … |
Decoy outcomes (file-level evidence)
- b.m4 ·
m4_append(note, [if eval fails, try open])(string): NOTHING counted from the quoted text: m4's branch keywords are macro names (ifelse/ifdef), bare if is not a keyword, and eval is not m4's risk keyword — state stayed exactly 2 with no flux (no branch context in b). The cleanest file in the corpus.
Reproduce / evidence
- Corpus folder:
data/m4/—expected_signals.jsoncarries the shape-by-shape accounting - Gate:
GALAXYSCOPE_BIN=<venv>/bin/galaxyscope python tools/verify_language.py m4in the corpus repo - Full context: findings_by_language.md#m4 · bias_report.md · bias chart
Activity
squid-protocol commentedon Sep 3, 2026
Status: 3🔴 / 3🟡 unexplained, 4 further out-of-band cell(s) accounted for
Generated from
docs/bias_data.json+deviation_ledger.jsonbytools/issue_status.py, using the same verdict machinerybias_report.py --gateruns on (gitgalaxy#2669 E.1), so this comment and the gate cannot disagree about what is left.Still unexplained — the work this issue tracks
keyword_hitsargscoding_loctotal_locrisk_safety_scorerisk_state_fluxOut of band, but accounted for
derived — the deviation entered through an input that is itself out of band -- the same finding counted twice
avg_func_loc— inherits total_loc, coding_locfunc_internal_density— inherits argsledgered — a validated ledger entry names this language and this metric
control_flow_ratio— control-flow-ratio-denominator-is-a-vocabulary-tallystate_mutation— string-literal-selective-shielding, batch4-dual-keyword-overlapsUnreviewed n/a cells
The registry defines no rule here, and no validated ledger entry records why — each is either real morphology to ledger or a missing-rule engine gap of the kind gitgalaxy#2610 turned out to be:
risk_concurrency.Close criterion (gitgalaxy#2669 Batch E.4)
python tools/language_deviations.py m4exits 1: 6 unexplained cell(s) and 1 unreviewed n/a cell(s) remain. Each needs a fix, or a validated ledger entry saying why it is correct, before this issue can close.squid-protocol commentedon Sep 5, 2026
Status: clean — every out-of-band metric is accounted for
Generated from
docs/bias_data.json+deviation_ledger.jsonbytools/issue_status.py, using the same verdict machinerybias_report.py --gateruns on (gitgalaxy#2669 E.1), so this comment and the gate cannot disagree about what is left.Out of band, but accounted for
derived — the deviation entered through an input that is itself out of band -- the same finding counted twice
risk_safety_score— inherits state_mutationrisk_state_flux— inherits state_mutationledgered — a validated ledger entry names this language and this metric
args— m4-parameters-are-use-sitesraw_state_slop_orphans— orphan-detection-is-name-recurrencerisk_api_exposure— api-double-count-inflates-scored-apistate_mutation— string-literal-selective-shielding, batch4-dual-keyword-overlapsReported, not gated (program length, unplanted risk inputs, commit age -- gitgalaxy#2669 F.1/F.3):
avg_func_loc1 vs median 5.104 (-80%),coding_loc7 vs median 17.5 (-60%),control_flow_ratio0.25 vs median 0.09375 (+167%),keyword_hits79 vs median 146.5 (-46%),total_loc12 vs median 25 (-52%). None of these is a planted signal; a derived risk cell above may inherit from one.Close criterion (gitgalaxy#2669 Batch E.4)
python tools/language_deviations.py m4exits 0: nothing is unexplained and no n/a cell is unreviewed. That is not the same as "no metric deviates" — deviations remain, and each is named above with the reason it is not a defect. This issue has nothing language-specific left to do.squid-protocol commentedon Sep 5, 2026
Closing per the F.6 criterion (gitgalaxy#2669):
language_deviations.py m4exits 0 — every out-of-band metric carries a recorded reason (validated ledger entry, derivation edge, or reported-not-gated group). This was the last language family blocked on unreviewed n/a cells; the n/a review landed in keyword-rosetta#54 (all 26 cells dispositioned; engine gaps filed as gitgalaxy#2732/#2733).To be explicit about what closed does NOT mean: not all-green — out-of-band metrics remain, each with its reason recorded above; the nightly regen + F.6 loop reopens this issue automatically if the language drifts back out of compliance. §10 capstone owed per the E.3/#2696 standing rule (F.6 owns it).