Repository navigation
fix(core-engine): the orphan census counts references, not name repetitions (#2727, #2728) - #2754
Merged
Merged
Conversation
…itions (#2727, #2728) `orphaned_logic` was decided by `token_counts[name] <= 1` over the whole code stream, so whether a function read as orphaned depended on how many times the language's syntax writes its name -- not on whether anything calls it. Three distinct defects came out of that one line. 1. #2727 -- syntax that repeats the name masked every orphan. Ada closes with `end Probe_Globals;`, LiveCode ends a handler by name, Haskell writes the type signature above the equation: the count is already 2 before anything calls the function. livecode reported 3 orphans across the crucible where the scoped test finds 127; ada and livecode reported 0 per file in keyword-rosetta against a 2.50 baseline. The test is now scoped to each function's own span, with a correction the issue's fix shape missed. The span does NOT reliably contain the declaration: shell's Mode-B spans start at the `{`, so a K&R `make_module()` on the previous line sits outside its own body, and ruby's spans are offset outright (66 of 132 sampled functions). Counting outside the span naively makes the declaration itself read as a reference: ruby fell 51 -> 44 orphans that way, html 5 -> 2, shell 8 -> 3, losing real ones. So when the span holds no occurrence of the name, the definition site is outside it by construction and exactly one outside occurrence is discounted. With the correction no language decreases for this reason. 2. A third defect neither issue named: `\b\w+\b` cannot produce a token containing `-`, `:` or `.`, so any name holding one counted ZERO and satisfied `<= 1` unconditionally. Every C++ `Class::method`, PowerShell `Verb-Noun`, Tcl `::ns::proc`, Scheme hyphenated define and COBOL paragraph was an orphan however many times it was called: cpp 1054 -> 720, tcl 210 -> 148, powershell 56 -> 3, scheme 60 -> 10, abap 41 -> 8. This also corrects part of #2727's own analysis -- for the hyphenated members of its "name never recurs" row the cause was tokenization, not invocation form. 3. #2728 -- igniter-keyword buckets entered the checks as ordinary functions. dockerfile's four same-bodied `RUN` slices scored `state_slop_duplicates` 1, the only nonzero duplicate cell across 46 corpus languages and a phantom; css's `keyframes` slices cleared each other's orphan flag by repeating the keyword. The bucket set is DERIVED from each language's own `func_start` (`_closed_literal_capture`), never hand-listed, so it cannot drift: exactly css, dockerfile and html have a capture that is provably a closed literal set, and a test pins that membership. Deliberately NOT added to `_UNCOUNTABLE_SLICE_NAMES`: that set governs the function POPULATION (#2691), and removing css's ~150 at-rule slices would take `functions_found` to 0 and make every per-function descriptor undefined (#2689's markdown/html shape). That is a real question, owned by keyword-rosetta's `slicer-segments-statements-not-functions` entry, and it is not this issue's measured harm. Net crucible movement: 7397 -> 7803 orphans (+5.5%). Both golden masters re-blessed: 2278 / 2277 differences, and the scoping proof is that inside "Structural Signatures" the only signals that moved are Orphaned Logic (581) and Public Exports (71) -- 652 of 652. Everything else is that signal's known consumers: tech debt 750, api/documentation 180, mass and the corpus-wide topology re-solve 560, roll-ups 388. Full suite 7632 passed serially; tree-sitter 30/30 OK; tri-comparison 3/3 OK; audit_check clear. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WfeHRWra1d6Z5uSReFajSq
The local `tri_comparison_chart.py --ci` verification run rewrote 24 `last_reconciled_at` stamps and nothing else (verified: zero non-timestamp diff lines). The push-to-main companion workflow regenerates this file, so the churn does not belong in this PR. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WfeHRWra1d6Z5uSReFajSq
Contributor
squid-protocol
added a commit
that referenced
this pull request
Sep 5, 2026
…an-census fix Main re-blessed for #2754 after this branch's first bless, so the fixtures conflicted. Merged main (taking its fixtures) and regenerated both from the merged tree. Scoped against main: 893 differences, 428 topological, 465 substantive -- all jcl files / jcl aggregates plus the two corpus-wide health averages they feed; zero off-target files. Same shape as the first bless. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
squid-protocol
added a commit
that referenced
this pull request
Sep 5, 2026
…ution narrowed to the command executors (#2748–#2751) (#2755) * feat(core-engine): jcl api, cleanup and globals rules, high_risk_execution narrowed to executors (#2748, #2749, #2750, #2751) Four rosetta gaps in one language, found by the 2026-09-05 pass over every non-green cell of the keyword-rosetta bias report and filed as #2748-#2751. - api: `//name PROC` -- a cataloged or in-stream procedure is the callable surface `EXEC name` / `EXEC PROC=name` invokes (api contract, fallback family). 13 declarations in 13 crucible files; 185 of 376 EXEC steps call one. - cleanup: DELETE as a dataset's normal-termination disposition (`DISP=(MOD,DELETE,DELETE)`, `(OLD,DELETE)`, `(,DELETE)`) -- JCL's teardown idiom. #2610 declined it for the io overlap; #2742 reversed that posture for sync_locks and the same narrow-subset measurement applies (36 of 533 DISP=). The abend-only positional of an allocation is excluded. - globals: JOBLIB (vs STEPLIB), a job-level SET symbol (vs a PROC parameter) and EXPORT SYMLIST -- the scoped-vs-global distinction the ledger said JCL lacked. SET is dual globals+state_mutation, dockerfile's ENV shape. - high_risk_execution: a bare `PGM=` counted every step (188 of 376 crucible EXECs, IEFBR14 and the compilers included). Narrowed to the programs that execute caller-supplied commands: IKJEFT01/1A/1B, BPXBATCH/BPXBATSL/BPXBATA2/ BPXBATA8/AOPBATCH, IRXJCL, SDSF. 61 hits in 48 files after. 20 new strict cases: per-rule partitions, overlap pins (io/sync_locks/state), an end-to-end deck through prism + splice, ReDoS detonations. Docs: jcl status sections 1/3/4/6/10 and the api contract's fallback family + audit table. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(core-engine): re-bless the golden masters for the jcl rule changes (#2748-#2751) Both fixtures regenerated with crucible_check.py --update. Every substantive difference is in jcl files or jcl directory aggregates (zero off-target files); the rest is the corpus-wide topological re-solve. Field breakdown and direction of change are in the PR description. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(core-engine): re-bless the golden masters on top of #2754's orphan-census fix Main re-blessed for #2754 after this branch's first bless, so the fixtures conflicted. Merged main (taking its fixtures) and regenerated both from the merged tree. Scoped against main: 893 differences, 428 topological, 465 substantive -- all jcl files / jcl aggregates plus the two corpus-wide health averages they feed; zero off-target files. Same shape as the first bless. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> --------- Co-authored-by: Joe Esquibel <squid-protocol@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This was referenced Sep 5, 2026
squid-protocol
added a commit
that referenced
this pull request
Sep 5, 2026
…the TS structural_mass dampener (#2778) `raw_state_slop_orphans` is the corpus's third-weakest metric (61%). Four independent bugs each made it under-report; they share one golden-master bless, and #2768 is unsafe without #2777, so they land together. #2777 (filed during this work): `_is_orphan` anchored its reference test with `\b`, which asserts a `\w`<->`\W` transition rather than "no adjacent word character". A name ending in a non-word character matched NOTHING -- not even its own declaration -- so the declaration discount fired and the function was reported an orphan unconditionally. 32 of 32 ruby predicate/bang/setter methods and 12 of 12 C++ operator overloads in the crucible were dead code regardless of use. Replaced with lookarounds, exactly equivalent for an all-word name. #2768: the `len(func_name) > 3` guard was a proxy from the pre-#2727 whole-file token-frequency test. Against #2754's span-scoped test it only meant a function named in three characters or fewer could never be reported unused, in any language -- 3.6% of functions corpus-wide, 29.4% of lua's. #2774: naming a function in an export statement counted as a use, so a declared- public uncalled function could never be reported dead -- the exact population the census exists to find. Fixed with an opt-in `_visibility_export` rule on the five languages that publish a function by naming it; `_is_orphan` discounts only the captured name's own span, never a whole line, so a genuine call sharing a line with an export still counts. The other 41 languages are unchanged by construction. Not the `api` rule, whose broad visibility modifiers (javascript's bare `export`) would have swallowed real calls. #2767: yaml's `func_start` had no capture group, so every extracted step in every scanned workflow was named `run` -- killing the orphan census, the duplicate census, and per-step identity. Steps now take their name from the adjacent `name:` key with a bounded step-over for intervening keys. The keyword alternation is group 1 and excludes its colon, so `_closed_literal_capture` recognises it and an unnamed step is censused like dockerfile's `RUN`. #2769: a missing group made `tsx?$` a top-level alternative in PATH_MODIFIERS, so every .ts/.tsx file in every scan took a 0.1 structural_mass dampener meant for icons and assets. Verified: 7696 passed; rosetta_audit 46/46 clean, 0 regressions; audit_check clear; golden masters re-blessed with the uncapped diff scoped per field. Closes #2767 Closes #2768 Closes #2769 Closes #2774 Closes #2777 Co-authored-by: Joe Esquibel <squid-protocol@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This was referenced Sep 6, 2026
Closed
squid-protocol
added a commit
that referenced
this pull request
Sep 9, 2026
…#2827) (#2914) detector.py's overlap test (#2731) decided whether the api rule had already counted an unreferenced function by tokenising each api-matched line with \b\w+\b and testing the orphan's name for membership. That tokenizer cannot produce a token containing `-`, `:`, `.`, `!` or `?`, so every function whose name holds one -- cobol PROBE-GLOBALS in its own `ENTRY 'PROBE-GLOBALS'` line, scheme `(export probe-globals)`, lua `M.start`, perl `Pkg::method`, zig `@"..."`, every powershell Verb-Noun -- was invisible to it, the overlap read 0, and galaxyscope's Contextual Baseline Fix credited the orphan a second time: the exact double count #2731 removed, back for every language whose names are not plain \w. Same tokenizer family as #2754, which fixed the census and left this consumer behind. The test now searches the orphan's real name with the census's own _name_boundary_pattern lookarounds over the api-matched lines, so both halves of the census agree about what an occurrence of a name is. The "no false positives" argument in the comment still holds: an orphan's name occurs exactly once in the file, so a match on an api line is its own declaration. Corpus (rosetta_audit against keyword-rosetta origin/main): cobol and scheme a/b/c api_orphan_credit 3 -> 0, nothing else moves (c's a.c is #2907's pending re-bless). Label rosetta:rebless-owed. Golden masters re-blessed in both modes; scoped with a full golden_diff pass: 14 files in lua (freebsd-src, cosmopolitan), perl (exiftool) and zig (Compilation.zig) move on exactly Public Exports / API Exposure / Documentation Exposure / Structural Magnitude, their aggregates follow, the rest is the coordinate re-solve. Phase 1 of #2908. Co-authored-by: Joe Esquibel <squid-protocol@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #2727.
Closes #2728.
Part of #2669 (F.4).
orphaned_logicwas decided by one line —token_counts[name] <= 1over the whole code stream —so whether a function read as orphaned depended on how many times the language's syntax writes its
name, not on whether anything calls it. Three distinct defects fall out of that, two filed and one
found while fixing them.
1. #2727 — syntax that repeats the name masked every orphan
Ada closes with
end Probe_Globals;, LiveCode ends a handler by name, Haskell writes the typesignature above the equation. The count is already 2 before anything calls the function, so those
languages could never report one:
livecodefound 3 orphans across the crucible where thescoped test finds 127;
adaandlivecoderead 0 per file in keyword-rosetta against a 2.50baseline.
The fix shape as filed was unsafe, and I corrected it. "Count outside
[start_idx, end_idx)"assumes the span contains the function's declaration. It often doesn't:
shell— Mode-B spans start at the{, so a K&Rmake_module()on the previous line sitsoutside its own body (
curl/initscript.sh);ruby— spans are offset outright; 66 of 132 sampled functions have the declaration outside.Counting naively there makes the declaration itself read as a reference and clears the flag —
measured:
ruby51 → 44 orphans,html5 → 2,shell8 → 3, all real ones lost. So when the spanholds no occurrence of the name, the definition site is outside it by construction and exactly one
outside occurrence is discounted. With the correction, no language decreases for that reason.
I also tried
span ∪ start_line; it fixes ruby and html but not shell, becausestart_linepointsat the
{rather than the line carrying the name — a separate metadata observation worth its ownissue.
The Haskell caveat the issue raised is moot: the span does start at the type signature
(
probeGlobals's span begins atprobeGlobals :: Int -> Int). Haskell still reads ~0 orphans, butfor a reason the issue didn't anticipate — its module export list names every probe outside every
span. That's keyword-rosetta's already-ledgered
api-export-names-suppress-orphan-creditshape,corpus-side, not an engine gap. The one orphan it does find (
entry, unexported and uncalled) iscorrect.
2. A third defect neither issue named
\b\w+\bcannot produce a token containing-,:or.. Any name holding one therefore countedzero and satisfied
<= 1unconditionally — it was an orphan however many times it was called:cppClass::methodtcl::ns::procpowershellVerb-NounschemedefineabapcobolThis also corrects part of #2727's own analysis. Its third table row — abap, cobol, jcl, m4,
makefile, objective-c, scheme at 3.25 orphans/file — is attributed there to names "written once,
invoked by other means". For the hyphenated members the cause was tokenization: the counter
could not represent the name at all.
schemegoes 3.25 → 0.25 orphans/file in the corpus on thatalone.
3. #2728 — igniter-keyword buckets entered the checks as ordinary functions
dockerfile's four same-bodied
RUNslices scoredstate_slop_duplicates1 — the only nonzeroduplicate cell across all 46 corpus languages, and a phantom. css's
keyframesslices cleared eachother's orphan flag by repeating the keyword.
The bucket set is derived from each language's own
func_start, never hand-listed, so it cannotdrift:
_closed_literal_capture()returns names only when the capture is provably a fixed literalalternation and the empty set otherwise. Exactly three languages qualify —
css,dockerfile,html— andtest_closed_literal_capture_is_generated_not_hand_listedpins that membership, so anew keyword in any of those alternations is picked up automatically and a rule that can produce a
real identifier can never opt in by accident.
Deliberately not added to
_UNCOUNTABLE_SLICE_NAMES, as the issue suggested. That set governsthe function population (#2691); removing css's ~150 at-rule slices would take
functions_foundto 0 and make every per-function descriptor undefined — #2689's markdown/html shape. That is a real
question, owned by keyword-rosetta's
slicer-segments-statements-not-functionsentry, and it is not#2728's measured harm. Recorded in the code rather than silently skipped.
Verification
tree_sitter_accuracy_audit --ci --all30/30 OK;tri_comparison_chart --all --ci3/3 OK;audit_check.pyclear; ruff and mypy at baseline.Golden masters re-blessed: 2278 / 2277 differences. The scoping proof is what moved inside
7. Structural Signatures: Orphaned Logic 581 + Public Exports 71 = 652 of 652. No otherstructural signal moved at all, which is exactly what a change confined to the orphan census should
look like. Everything else is that signal's known consumers:
tech_debt(orphans are dead weight)Public ExportsNet crucible orphans 7397 → 7803 (+5.5%): increases where the span fix removes a mask
(
typescript+333,livecode+124,php+100,jcl+50,go/rust+46), decreases where thetokenization fix stops manufacturing them.
Cross-repo
Companion: squid-protocol/keyword-rosetta re-bless, owed after this merges — carries the
rosetta:rebless-owedlabel perkeyword-rosetta/docs/GATING.md's cross-repo flow.Corpus impact is unusually small for a change this size: 4 cells,
api_orphan_creditondockerfile/a.dockerfile(1 → 0) andscheme/{a,b,c}.scm(3 → 0). The rosetta manifests do notgate orphan counts directly — they see them only through the orphan→api conversion — so the
languages whose orphans move but whose api rule already declares them (ada, livecode, haskell)
record no manifest change at all.
🤖 Generated with Claude Code
https://claude.ai/code/session_01WfeHRWra1d6Z5uSReFajSq