Repository navigation
fix(core-engine): an export construct that names many functions is still a declaration (#2823, roadmap Phase 3) - #2828
Conversation
…ill a declaration (#2823, roadmap Phase 3) `unreferenced_by_name` read 0.25 per keyword-rosetta file for haskell and scheme against a 2.50 corpus median -- 12 of each language's 13 probe functions reading as REFERENCED while nothing called any of them. That is #2806's defect with the sign flipped: there the census called a function unreferenced that the language reaches without naming it; here it called a function referenced on an occurrence that is a declaration, which corollary 2 of the stated contract says must not clear the flag. One outside occurrence each, and it is the export: module A (probeGlobals, probeTest, probeSafety) where -- outside the span probeGlobals :: Int -> Int -- INSIDE the span probeGlobals env = env -- INSIDE the span (export probe-globals) -- outside the span (define (probe-globals env) env) The Haskell type signature needed no rule of its own: haskell's slicer span STARTS at the signature, so that repetition is already inside the function's own definition. Measured -- declaring `^([a-z]\w*)[ \t]*::` as an export form moves the count not at all, which is what the issue found and could not explain. #2774's `_visibility_export` captures ONE name per match, which is the whole statement for `export -f foo` and its four siblings and cannot express a construct that names every exported function at once. Adds `_visibility_export_list`, the plural form: its capture groups are REGIONS holding exported names, and `_export_declaration_offsets` records the start offset of every name token inside a region. Two keys rather than one generalised key, deliberately -- the singular form records its group's own start offset verbatim, and assembly exports `.foo` while ruby exports `save!`, whose first and last characters a region tokenizer does not treat as part of a name; tokenizing those captures would put the offset one character off and silently stop discounting them. A language declares one key or the other, and a test is the gate on that. haskell declares the module header's parenthesised list (bounded by the `) where` anchor, since `[^)]*` stops at the first `Opt(..)`); scheme declares both of its clause forms, R7RS `(export a b c)` and Guile `#:export (a b c)`. Measured, both corpora: * keyword-rosetta: haskell and scheme 0.25 -> 3.00 per file, and NO other language moves -- structurally guaranteed, since the singular path is unchanged and only these two declare the new key. 3.00 rather than the 2.50 median because both carry the same plant gap #2806 found in abap/m4/makefile/objective-c, hidden until now behind the export false-negative: `main`'s `entry` dispatched one of its three probes. The corpus PR plants the other two and both land on 2.50. * language-crucible: haskell 1.4% -> 27.9% unreferenced (2 of 147 units -> 41). Every one of the 39 new hits is a pandoc utility whose name occurs exactly three times in its own file -- export list, type signature, equation -- with no caller there. That is corollary 5 working as designed: `Text.Pandoc.Shared` is a library module of 100 exported helpers, 36 of which nothing in that file calls, and saying so is the input galaxyscope.py's orphan -> api conversion needs. Real-world scheme does not move: neither crucible file uses an export clause outside a macro template. Golden masters re-blessed. NINE value differences, identical in both fixtures, every one of them haskell/pandoc: three files' Unreferenced By Name (Shared 0 -> 36, Filter 2 -> 4, Options 0 -> 1), their Tech Debt Exposure (the census is weighted 2.0 in slop_stress), the directory group's average 3.76 -> 17.51, and global avg_tech_debt 27.332 -> 27.386. No topological X/Y/Z coordinates moved at all -- unusual for a bless, and the reason is that no node's structural mass changed. One neighbour moved and it is a PRE-EXISTING defect this made visible, filed as #2827 rather than fixed here: with scheme finally producing orphans, its `api_orphan_credit` went 0 -> 3, the double count #2731/#2734 exists to prevent. `api_declared_orphans` tokenises the api-matched line with `\b\w+\b`, which cannot produce a token holding a hyphen, so `probe-globals` is never recognised in `(export probe-globals)`. haskell is the control that proves the diagnosis -- its probe names are all-\w, its api_declared_orphans reads 3, and its credit correctly stays 0. cobol carries the same double count on main today, and the crucible makes it much larger than the control corpus does: 4983 -> 5122 api_declared_orphans under a name-aware test, +139 across zig, lua and powershell, whose count is 0 for the ENTIRE language because a Verb-Noun name always has a hyphen. Docs: corollary 2 gains the plural form, the audit table's haskell/scheme row moves 0.25 -> 2.50, and a new section states the whole finding including why two keys and what the crucible says. how_to_add_a_language rule 19 gains the opposite-mistake bullet -- a census that reads too CLEAN is the same defect as one that reads too dirty. Closes #2823. Part of #2812. Cross-repo: keyword-rosetta PR (branch plant/2823-haskell-scheme-dispatch) plants the two dispatch calls and re-blesses scheme's credit cell against the #2827 ledger entry. This merges FIRST; that one goes green against gitgalaxy main once it lands. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
…#2804) Regenerated both golden masters on top of the merge with main (which had itself re-blessed them for #2828's export-construct change). The fixtures now carry both main's baseline and this branch's cobol deltas; diff vs origin/main is cobol-only (Global State Dependencies phantom removals + Function Parameters operand counts) plus sanitized scan metadata. Both modes pass crucible_check.py. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GVJXkEphGEjmupV9j5mdSr
… operands not clauses (#2804) (#2830) * fix(cobol): globals drops WORKING-STORAGE header (#2805); args counts operands not clauses (#2804) Two COBOL rosetta cells, both corpus-morphology rejections that no longer hold. #2805 -- globals: dropped `WORKING-STORAGE SECTION` from the `globals` rule. `GLOBAL`/`EXTERNAL`/`COMMON` are the item-level clauses that publish shared state; `WORKING-STORAGE SECTION` is a region header, mandatory in every program that declares any state, so counting it stacked an unconditional phantom `globals >= 1` on every real COBOL file (and inflated `encapsulation_ratio`). `SECTION` still counts under `structural_boundaries`, so no structural signal is lost. (The issue floated a fallback shape (b) too, but the rosetta corpus keeps the WS header and the item clauses in *different* files -- main.cbl vs a.cpy -- so only dropping the header reaches the planted 2; a fallback would leave it 3.) #2804 -- args: the file-level `args` signal counted USING/RETURNING *clauses*, not declared parameters -- `USING A, B, C` read the same 1 as `USING A`. Two fixes together: * the `args` regex captured only the first operand of a comma-SPACE list (`WS-A, WS-B, WS-C` -> `WS-A,`) because each operand iteration had no leading-whitespace allowance; widened the inter-operand separator so the whole list is captured. Newlines are crossed only when operands are comma-joined, and the same-line run is capped (`[ \t]{1,4}`), so a period-less list does not over-read into the next statement or bridge the wide gap to fixed-format columns 73-80 (both real regressions caught on the NIST CCVS / carddemo corpora: CSUTLDPY 3->5, IC2014 44->45). * new `_count_cobol_using_operands` turns each captured list into a real operand count (dropping BY REFERENCE/CONTENT/VALUE phrase words), and the file-level tally uses it for COBOL's `args` rule -- same "one match, real operand count" shape as the agc/assembly register counters. Verification: 215 cobol extraction/mainframe tests pass; new operand-count and over-capture-guard cases added to test_cobol_strict.py; args regex confirmed ReDoS-immune. Golden-crucible cobol deltas all move in the correct direction (346 globals -1/-2/-4 phantom removals; 41 args increases to real operand counts, extremes hand-verified: IC2014=44, IC2244=48); the 64 non-cobol golden diffs are pre-existing main drift, not from this change. ruff/mypy baselines clean. Cross-repo (rosetta sweep -- engine PR first): keyword-rosetta owes a corpus re-bless against engine main -- data/cobol/expected_signals.json main.cbl globals 1->0 (#2805). #2804's chart-cell movement is the optional corpus shape (a) (append ` USING <item>` to the nine ENTRY lines); the engine change here is the general-correctness half and is invisible on the single-operand rosetta shell. Golden-crucible cobol values also owe a re-bless at the next crucible pin bump. Label: rosetta:rebless-owed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GVJXkEphGEjmupV9j5mdSr * test(golden-crucible): re-bless cobol baselines for #2805 globals + #2804 args Regenerated both golden masters (full-precision and zero-dependency) against the pinned language-crucible v1.2.0 corpus via `tests/tools/crucible_check.py --update --yes`. All 387 changed cells in each fixture are COBOL and are the intended effect of this branch's engine change: * Global State Dependencies (globals): 346 files drop the phantom that the `WORKING-STORAGE SECTION` region header used to add (Expected 1 -> Got 0, and Expected N -> N-1/N-2/N-4 on multi-program NIST files). #2805. * Function Parameters (args): 41 files rise from the old clause-first-operand undercount to the real declared-operand count -- e.g. CSUTLDPY.cpy 1 -> 3, IC2014.2.cbl 11 -> 44, IC2244.2.cbl 12 -> 48. #2804. Both modes now pass `crucible_check.py`. No non-COBOL structural cell moved (the only other diff lines are the fixtures' sanitized scan-path/timestamp metadata, which the comparison ignores). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GVJXkEphGEjmupV9j5mdSr * test(golden-crucible): re-bless cobol baselines on merged main (#2805, #2804) Regenerated both golden masters on top of the merge with main (which had itself re-blessed them for #2828's export-construct change). The fixtures now carry both main's baseline and this branch's cobol deltas; diff vs origin/main is cobol-only (Global State Dependencies phantom removals + Function Parameters operand counts) plus sanitized scan metadata. Both modes pass crucible_check.py. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GVJXkEphGEjmupV9j5mdSr --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Closes #2823. Part of #2812.
unreferenced_by_nameread 0.25 per keyword-rosetta file for haskell and scheme against a 2.50 corpus median — 12 of each language's 13 probe functions reading as referenced while nothing called any of them. That is #2806's defect with the sign flipped: there the census called a function unreferenced that the language reaches without naming it; here it called a function referenced on an occurrence that is a declaration, which corollary 2 of the stated contract says must not clear the flag.One outside occurrence each, and it is the export
The Haskell type signature needed no rule of its own: haskell's slicer span starts at the signature, so that repetition is already inside the function's own definition. That explains the issue's own unexplained measurement — declaring
^([a-z]\w*)[ \t]*::as an export form moves the count not at all._visibility_export_list#2774's
_visibility_exportcaptures one name per match, which is the whole statement forexport -f fooand its four siblings and cannot express a construct that names every exported function at once. This adds the plural form: its capture groups are regions holding exported names, and_export_declaration_offsetsrecords the start offset of every name token inside a region.Two keys rather than one generalised key, deliberately. The singular form records its group's own start offset verbatim, and assembly exports
.foowhile ruby exportssave!— whose first and last characters a region tokenizer does not treat as part of a name. Tokenizing those captures would put the offset one character off and silently stop discounting them. A language declares one key or the other, andtest_the_two_export_keys_never_disagree_about_a_languageis the gate on that.haskell declares the module header's parenthesised list (bounded by the
) whereanchor, since[^)]*stops at the firstOpt(..), and every one of pandoc's seven headers terminates exactly right); scheme declares both of its clause forms, R7RS(export a b c)and Guile#:export (a b c).Measured, both corpora
No other language can move: the singular path is byte-identical and only these two declare the new key.
3.00, not the 2.50 median, because both languages also carry the same plant gap #2806 found in abap/m4/makefile/objective-c — hidden until now behind the export false-negative — where
main'sentrydispatched one of its three probes. The corpus PR plants the other two and both land on 2.50.The crucible's 39 new hits are correct. Every one is a pandoc utility whose name occurs exactly three times in its own file — export list, type signature, equation — with no caller there. That is corollary 5 working as designed:
Text.Pandoc.Sharedis a library module of 100 exported helpers, 36 of which nothing in that file calls, and saying so is the inputgalaxyscope.py's orphan → api conversion needs. Same shape as #2774's shell precedent (0 → 3 per file).The bless
Nine value differences, identical in both fixtures, every one of them
haskell/pandoc:Shared.hsUnreferenced By NameFilter.hsUnreferenced By NameOptions.hsUnreferenced By Namehaskell/pandocavg tech debtavg_tech_debtThe tech-debt rows are the census's own weight arriving downstream (
slop_stresscounts each unit at 2.0). No topologicalX/Y/Zcoordinates moved at all — unusual for a bless, and the reason is that no node's structural mass changed.One neighbour moved — a pre-existing defect this made visible
With scheme finally producing orphans, its
api_orphan_creditwent 0 → 3 in a/b/c: the double count #2731/#2734 exists to prevent.api_declared_orphanstokenises the api-matched line with\b\w+\b, which cannot produce a token holding a hyphen, soprobe-globalsis never recognised in(export probe-globals).haskell is the control that proves the diagnosis — its probe names are all-
\w, itsapi_declared_orphansreads 3, and its credit correctly stays 0.Not new, and much larger than the control corpus suggests: cobol carries it on main today, and over the crucible
api_declared_orphansmoves 4,983 → 5,122 under a name-aware test, +139 across zig (@"..."quoted identifiers), lua (bit.bor,M.start) and powershell — whose count is 0 for the entire language, because aVerb-Nounname always contains a hyphen. Filed as #2827 rather than fixed here: it moves five languages and needs its own bless.Docs
docs/unreferenced_by_name_contract.md: corollary 2 gains the plural form, the audit table's haskell/scheme row moves 0.25 → 2.50, and a new section states the whole finding — why two keys, what the crucible says, and what the bless moved.how_to_add_a_language.mdrule 19 gains the opposite-mistake bullet: a census that reads too clean is the same defect as one that reads too dirty.Verification
crucible_check.py: both fixtures re-blessed, diff scoped above (9 values each, no unattributed mover).rosetta_audit.py --baseline-bin <engine main>: 46 languages, 0 regressions.tri_comparison_chart.py --all --ci: 3 languages, all OK.audit_check.py(ruff / mypy / dead-key / ast-accuracy): all clear.Cross-repo
plant/2823-haskell-scheme-dispatch) — plants the two dispatch calls, re-blesses scheme'sapi_orphan_creditcell against a new api_declared_orphans tokenizes api lines with\b\w+\b, so a hyphenated or qualified name is never recognised as already-declared (#2731's double count returns for cobol and scheme) #2827 ledger entry, and narrowsorphan-detection-is-name-recurrencefrom 8 languages to 6.mainonce it lands.\b\w+\b, so a hyphenated or qualified name is never recognised as already-declared (#2731's double count returns for cobol and scheme) #2827 returns scheme's and cobol'sapi_orphan_creditcells to 0 and owes another corpus re-bless.🤖 Generated with Claude Code