Skip to content

fix(core-engine): an export construct that names many functions is still a declaration (#2823, roadmap Phase 3) - #2828

Merged
squid-protocol merged 1 commit into
mainfrom
fix/2823-export-list-declarations
Sep 6, 2026
Merged

squid-protocol merged 1 commit into
mainfrom
fix/2823-export-list-declarations

Conversation

@squid-protocol

Copy link
Copy Markdown
Owner

Closes #2823. Part of #2812.

unreferenced_by_name read 0.25 per keyword-rosetta file for haskell and scheme against a 2.50 corpus median — 12 of each language's 13 probe functions reading as referenced while nothing called any of them. That is #2806's defect with the sign flipped: there the census called a function unreferenced that the language reaches without naming it; here it called a function referenced on an occurrence that is a declaration, which corollary 2 of the stated contract says must not clear the flag.

One outside occurrence each, and it is the export

module A (probeGlobals, probeTest, probeSafety) where   -- outside the span
probeGlobals :: Int -> Int                              -- INSIDE the span
probeGlobals env = env                                  -- INSIDE the span
(export probe-globals)                                  -- outside the span
(define (probe-globals env) env)

The Haskell type signature needed no rule of its own: haskell's slicer span starts at the signature, so that repetition is already inside the function's own definition. That explains the issue's own unexplained measurement — declaring ^([a-z]\w*)[ \t]*:: as an export form moves the count not at all.

_visibility_export_list

#2774's _visibility_export captures one name per match, which is the whole statement for export -f foo and its four siblings and cannot express a construct that names every exported function at once. This adds the plural form: its capture groups are regions holding exported names, and _export_declaration_offsets records the start offset of every name token inside a region.

Two keys rather than one generalised key, deliberately. The singular form records its group's own start offset verbatim, and assembly exports .foo while ruby exports save! — whose first and last characters a region tokenizer does not treat as part of a name. Tokenizing those captures would put the offset one character off and silently stop discounting them. A language declares one key or the other, and test_the_two_export_keys_never_disagree_about_a_language is the gate on that.

haskell declares the module header's parenthesised list (bounded by the ) where anchor, since [^)]* stops at the first Opt(..), and every one of pandoc's seven headers terminates exactly right); scheme declares both of its clause forms, R7RS (export a b c) and Guile #:export (a b c).

Measured, both corpora

corpus before after
keyword-rosetta, haskell 0.25 /file 3.00 /file
keyword-rosetta, scheme 0.25 /file 3.00 /file
keyword-rosetta, other 44 languages — unchanged
language-crucible, haskell 2 of 147 units (1.4%) 41 of 147 (27.9%)
language-crucible, scheme 0 0 — neither file uses an export clause outside a macro template

No other language can move: the singular path is byte-identical and only these two declare the new key.

3.00, not the 2.50 median, because both languages also carry the same plant gap #2806 found in abap/m4/makefile/objective-c — hidden until now behind the export false-negative — where main's entry dispatched one of its three probes. The corpus PR plants the other two and both land on 2.50.

The crucible's 39 new hits are correct. Every one is a pandoc utility whose name occurs exactly three times in its own file — export list, type signature, equation — with no caller there. That is corollary 5 working as designed: Text.Pandoc.Shared is a library module of 100 exported helpers, 36 of which nothing in that file calls, and saying so is the input galaxyscope.py's orphan → api conversion needs. Same shape as #2774's shell precedent (0 → 3 per file).

The bless

Nine value differences, identical in both fixtures, every one of them haskell/pandoc:

what was is
Shared.hs Unreferenced By Name 0 36
Filter.hs Unreferenced By Name 2 4
Options.hs Unreferenced By Name 0 1
those three files' Tech Debt Exposure 0.0% / 41.35% / 0.0% 97.04% / 85.83% / 9.7%
haskell/pandoc avg tech debt 3.76 17.51
global avg_tech_debt 27.332 27.386

The tech-debt rows are the census's own weight arriving downstream (slop_stress counts each unit at 2.0). No topological X/Y/Z coordinates moved at all — unusual for a bless, and the reason is that no node's structural mass changed.

One neighbour moved — a pre-existing defect this made visible

With scheme finally producing orphans, its api_orphan_credit went 0 → 3 in a/b/c: the double count #2731/#2734 exists to prevent. api_declared_orphans tokenises the api-matched line with \b\w+\b, which cannot produce a token holding a hyphen, so probe-globals is never recognised in (export probe-globals).

haskell is the control that proves the diagnosis — its probe names are all-\w, its api_declared_orphans reads 3, and its credit correctly stays 0.

Not new, and much larger than the control corpus suggests: cobol carries it on main today, and over the crucible api_declared_orphans moves 4,983 → 5,122 under a name-aware test, +139 across zig (@"..." quoted identifiers), lua (bit.bor, M.start) and powershell — whose count is 0 for the entire language, because a Verb-Noun name always contains a hyphen. Filed as #2827 rather than fixed here: it moves five languages and needs its own bless.

Docs

  • docs/unreferenced_by_name_contract.md: corollary 2 gains the plural form, the audit table's haskell/scheme row moves 0.25 → 2.50, and a new section states the whole finding — why two keys, what the crucible says, and what the bless moved.
  • how_to_add_a_language.md rule 19 gains the opposite-mistake bullet: a census that reads too clean is the same defect as one that reads too dirty.

Verification

  • Full suite: 7,914 passed, 4 skipped, 9 xfailed.
  • crucible_check.py: both fixtures re-blessed, diff scoped above (9 values each, no unattributed mover).
  • rosetta_audit.py --baseline-bin <engine main>: 46 languages, 0 regressions.
  • tri_comparison_chart.py --all --ci: 3 languages, all OK.
  • audit_check.py (ruff / mypy / dead-key / ast-accuracy): all clear.

Cross-repo

🤖 Generated with Claude Code

…ill a declaration (#2823, roadmap Phase 3)

`unreferenced_by_name` read 0.25 per keyword-rosetta file for haskell and
scheme against a 2.50 corpus median -- 12 of each language's 13 probe
functions reading as REFERENCED while nothing called any of them. That is
#2806's defect with the sign flipped: there the census called a function
unreferenced that the language reaches without naming it; here it called
a function referenced on an occurrence that is a declaration, which
corollary 2 of the stated contract says must not clear the flag.

One outside occurrence each, and it is the export:

    module A (probeGlobals, probeTest, probeSafety) where   -- outside the span
    probeGlobals :: Int -> Int                              -- INSIDE the span
    probeGlobals env = env                                  -- INSIDE the span

    (export probe-globals)                                  -- outside the span
    (define (probe-globals env) env)

The Haskell type signature needed no rule of its own: haskell's slicer
span STARTS at the signature, so that repetition is already inside the
function's own definition. Measured -- declaring `^([a-z]\w*)[ \t]*::` as
an export form moves the count not at all, which is what the issue found
and could not explain.

#2774's `_visibility_export` captures ONE name per match, which is the
whole statement for `export -f foo` and its four siblings and cannot
express a construct that names every exported function at once. Adds
`_visibility_export_list`, the plural form: its capture groups are
REGIONS holding exported names, and `_export_declaration_offsets` records
the start offset of every name token inside a region. Two keys rather than
one generalised key, deliberately -- the singular form records its group's
own start offset verbatim, and assembly exports `.foo` while ruby exports
`save!`, whose first and last characters a region tokenizer does not treat
as part of a name; tokenizing those captures would put the offset one
character off and silently stop discounting them. A language declares one
key or the other, and a test is the gate on that.

haskell declares the module header's parenthesised list (bounded by the
`) where` anchor, since `[^)]*` stops at the first `Opt(..)`); scheme
declares both of its clause forms, R7RS `(export a b c)` and Guile
`#:export (a b c)`.

Measured, both corpora:

  * keyword-rosetta: haskell and scheme 0.25 -> 3.00 per file, and NO
    other language moves -- structurally guaranteed, since the singular
    path is unchanged and only these two declare the new key. 3.00 rather
    than the 2.50 median because both carry the same plant gap #2806
    found in abap/m4/makefile/objective-c, hidden until now behind the
    export false-negative: `main`'s `entry` dispatched one of its three
    probes. The corpus PR plants the other two and both land on 2.50.
  * language-crucible: haskell 1.4% -> 27.9% unreferenced (2 of 147 units
    -> 41). Every one of the 39 new hits is a pandoc utility whose name
    occurs exactly three times in its own file -- export list, type
    signature, equation -- with no caller there. That is corollary 5
    working as designed: `Text.Pandoc.Shared` is a library module of 100
    exported helpers, 36 of which nothing in that file calls, and saying
    so is the input galaxyscope.py's orphan -> api conversion needs.
    Real-world scheme does not move: neither crucible file uses an export
    clause outside a macro template.

Golden masters re-blessed. NINE value differences, identical in both
fixtures, every one of them haskell/pandoc: three files' Unreferenced By
Name (Shared 0 -> 36, Filter 2 -> 4, Options 0 -> 1), their Tech Debt
Exposure (the census is weighted 2.0 in slop_stress), the directory
group's average 3.76 -> 17.51, and global avg_tech_debt 27.332 -> 27.386.
No topological X/Y/Z coordinates moved at all -- unusual for a bless, and
the reason is that no node's structural mass changed.

One neighbour moved and it is a PRE-EXISTING defect this made visible,
filed as #2827 rather than fixed here: with scheme finally producing
orphans, its `api_orphan_credit` went 0 -> 3, the double count #2731/#2734
exists to prevent. `api_declared_orphans` tokenises the api-matched line
with `\b\w+\b`, which cannot produce a token holding a hyphen, so
`probe-globals` is never recognised in `(export probe-globals)`. haskell
is the control that proves the diagnosis -- its probe names are all-\w,
its api_declared_orphans reads 3, and its credit correctly stays 0. cobol
carries the same double count on main today, and the crucible makes it
much larger than the control corpus does: 4983 -> 5122 api_declared_orphans
under a name-aware test, +139 across zig, lua and powershell, whose count
is 0 for the ENTIRE language because a Verb-Noun name always has a hyphen.

Docs: corollary 2 gains the plural form, the audit table's haskell/scheme
row moves 0.25 -> 2.50, and a new section states the whole finding
including why two keys and what the crucible says. how_to_add_a_language
rule 19 gains the opposite-mistake bullet -- a census that reads too CLEAN
is the same defect as one that reads too dirty.

Closes #2823. Part of #2812.

Cross-repo: keyword-rosetta PR (branch plant/2823-haskell-scheme-dispatch)
plants the two dispatch calls and re-blesses scheme's credit cell against
the #2827 ledger entry. This merges FIRST; that one goes green against
gitgalaxy main once it lands.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

🐦‍⬛ Muninn Security Scan

✅ No security issues found.

🐦‍⬛ Powered by Muninn · Skald Lab

@squid-protocol

Copy link
Copy Markdown
Owner Author

rosetta-audit failure: the companion re-bless, labelled

REG  scheme             FAIL scheme: 3 mismatch(es)
       - a.scm: api_orphan_credit expected 0, got 3
       - b.scm: api_orphan_credit expected 0, got 3
       - c.scm: api_orphan_credit expected 0, got 3
rosetta_audit: 46 language(s) checked -- 1 regression(s), 0 pre-existing, 0 broken.

This is the exact mirror of squid-protocol/keyword-rosetta#83's own red CI, and the two together are the whole cross-repo state: rosetta-audit checks out the corpus at main, whose scheme manifest still expects api_orphan_credit: 0 — the value it reads when (export probe-globals) is still wrongly clearing the orphan flag. With this build there are orphans again, galaxyscope.py's Contextual Baseline Fix converts them, and the cell reads 3.

The corpus PR blesses those three cells to 3 against a new ledger entry (api-declared-orphans-blind-to-non-word-names, #2827) and cannot go green until this merges; this cannot go green until that one does. Applied rosetta:rebless-owed — the label the workflow documents for exactly this shape.

Worth stating explicitly: only scheme moves, and only on this one downstream cell. haskell passes here, on corpus main, with no plant — its api_orphan_credit is 0 in both engine states, for two different reasons (no orphans before; all three suppressed by api_declared_orphans after). The census itself is not gated in any manifest, so haskell's and scheme's 0.25 → 3.00 move is invisible to this audit; what it caught is the single consequence two layers downstream, which is #2827.

@squid-protocol
squid-protocol merged commit ab06ea1 into main Sep 6, 2026
44 of 45 checks passed
@squid-protocol
squid-protocol deleted the fix/2823-export-list-declarations branch September 6, 2026 22:48
squid-protocol added a commit that referenced this pull request Sep 6, 2026
…#2804)

Regenerated both golden masters on top of the merge with main (which had itself
re-blessed them for #2828's export-construct change). The fixtures now carry
both main's baseline and this branch's cobol deltas; diff vs origin/main is
cobol-only (Global State Dependencies phantom removals + Function Parameters
operand counts) plus sanitized scan metadata. Both modes pass crucible_check.py.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GVJXkEphGEjmupV9j5mdSr
squid-protocol added a commit that referenced this pull request Sep 6, 2026
… operands not clauses (#2804) (#2830)

* fix(cobol): globals drops WORKING-STORAGE header (#2805); args counts operands not clauses (#2804)

Two COBOL rosetta cells, both corpus-morphology rejections that no longer hold.

#2805 -- globals: dropped `WORKING-STORAGE SECTION` from the `globals` rule.
`GLOBAL`/`EXTERNAL`/`COMMON` are the item-level clauses that publish shared
state; `WORKING-STORAGE SECTION` is a region header, mandatory in every program
that declares any state, so counting it stacked an unconditional phantom
`globals >= 1` on every real COBOL file (and inflated `encapsulation_ratio`).
`SECTION` still counts under `structural_boundaries`, so no structural signal is
lost. (The issue floated a fallback shape (b) too, but the rosetta corpus keeps
the WS header and the item clauses in *different* files -- main.cbl vs a.cpy --
so only dropping the header reaches the planted 2; a fallback would leave it 3.)

#2804 -- args: the file-level `args` signal counted USING/RETURNING *clauses*,
not declared parameters -- `USING A, B, C` read the same 1 as `USING A`. Two
fixes together:
  * the `args` regex captured only the first operand of a comma-SPACE list
    (`WS-A, WS-B, WS-C` -> `WS-A,`) because each operand iteration had no
    leading-whitespace allowance; widened the inter-operand separator so the
    whole list is captured. Newlines are crossed only when operands are
    comma-joined, and the same-line run is capped (`[ \t]{1,4}`), so a
    period-less list does not over-read into the next statement or bridge the
    wide gap to fixed-format columns 73-80 (both real regressions caught on the
    NIST CCVS / carddemo corpora: CSUTLDPY 3->5, IC2014 44->45).
  * new `_count_cobol_using_operands` turns each captured list into a real
    operand count (dropping BY REFERENCE/CONTENT/VALUE phrase words), and the
    file-level tally uses it for COBOL's `args` rule -- same "one match, real
    operand count" shape as the agc/assembly register counters.

Verification: 215 cobol extraction/mainframe tests pass; new operand-count and
over-capture-guard cases added to test_cobol_strict.py; args regex confirmed
ReDoS-immune. Golden-crucible cobol deltas all move in the correct direction
(346 globals -1/-2/-4 phantom removals; 41 args increases to real operand
counts, extremes hand-verified: IC2014=44, IC2244=48); the 64 non-cobol
golden diffs are pre-existing main drift, not from this change. ruff/mypy
baselines clean.

Cross-repo (rosetta sweep -- engine PR first): keyword-rosetta owes a corpus
re-bless against engine main -- data/cobol/expected_signals.json main.cbl
globals 1->0 (#2805). #2804's chart-cell movement is the optional corpus
shape (a) (append ` USING <item>` to the nine ENTRY lines); the engine change
here is the general-correctness half and is invisible on the single-operand
rosetta shell. Golden-crucible cobol values also owe a re-bless at the next
crucible pin bump. Label: rosetta:rebless-owed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GVJXkEphGEjmupV9j5mdSr

* test(golden-crucible): re-bless cobol baselines for #2805 globals + #2804 args

Regenerated both golden masters (full-precision and zero-dependency) against the
pinned language-crucible v1.2.0 corpus via `tests/tools/crucible_check.py
--update --yes`. All 387 changed cells in each fixture are COBOL and are the
intended effect of this branch's engine change:

  * Global State Dependencies (globals): 346 files drop the phantom that the
    `WORKING-STORAGE SECTION` region header used to add (Expected 1 -> Got 0, and
    Expected N -> N-1/N-2/N-4 on multi-program NIST files). #2805.
  * Function Parameters (args): 41 files rise from the old clause-first-operand
    undercount to the real declared-operand count -- e.g. CSUTLDPY.cpy 1 -> 3,
    IC2014.2.cbl 11 -> 44, IC2244.2.cbl 12 -> 48. #2804.

Both modes now pass `crucible_check.py`. No non-COBOL structural cell moved (the
only other diff lines are the fixtures' sanitized scan-path/timestamp metadata,
which the comparison ignores).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GVJXkEphGEjmupV9j5mdSr

* test(golden-crucible): re-bless cobol baselines on merged main (#2805, #2804)

Regenerated both golden masters on top of the merge with main (which had itself
re-blessed them for #2828's export-construct change). The fixtures now carry
both main's baseline and this branch's cobol deltas; diff vs origin/main is
cobol-only (Global State Dependencies phantom removals + Function Parameters
operand counts) plus sanitized scan metadata. Both modes pass crucible_check.py.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GVJXkEphGEjmupV9j5mdSr

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

rosetta:rebless-owed Intentionally moves keyword-rosetta counts; audit warns, corpus re-blesses after merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

unreferenced_by_name reads 0.25/file for haskell and scheme: a module export list and a type signature are declarations, not references

1 participant