fix(detection): shebang tokenisation, ambiguous triggers, trampolines, prefix anchors (#3116, #3118, #3133, #3134)

[squid-protocol]

Fixes #3116, closes #3118.

The bug

A shebang trigger was tested as a bare substring of the whole line (if trigger in first_line), so shell's sh fired on tclsh/wish/jimsh/swift-sh/racketsh and javascript's node fired on ts-node. The bogus shebang language then contradicted the file's own extension, and the Identity Conflict Trap forced the file to Tier 5 — Absolute Distrust: undeterminable, intensity 0.0, plus an Identity Masking anomaly flag. A customer with an ordinary Tcl script didn't see a classification error; they saw the scanner accusing their file of masquerading.

Measured on main before the fix:

file	shebang	before	after
hello.tcl	#!/usr/bin/tclsh	Tier 5 undeterminable	tcl Tier 0
gui.tcl	#!/usr/bin/wish	Tier 5 undeterminable	tcl Tier 0
j.tcl	#!/usr/bin/env jimsh	Tier 5 undeterminable	tcl Tier 0
s.swift	#!/usr/bin/env swift-sh	Tier 5 undeterminable	swift Tier 0
r.scm	#!/usr/bin/env racketsh	Tier 5 undeterminable	scheme Tier 0
t.ts	#!/usr/bin/env ts-node	Tier 5 undeterminable	typescript Tier 0

#!/usr/bin/pwsh only ever passed by accident of registry order (powershell precedes shell), not by design.

The fix

Resolve the interpreter basename and match it as a token: handles the plain path form, env, env -S, and VAR=value prefixes. A trigger matches exactly, or with a pure version suffix — so python still names python3 and python3.12 while never naming micropython.

Two more bugs the #3118 invariant test then found

Writing "every trigger must resolve to the language that declares it" surfaced two further defects, both fixed here:

1. Three triggers are claimed by two languages each. deno and bun both run TypeScript and JavaScript; csi is both Chicken Scheme's interpreter and the C# script runner. Registry order picked a winner, so the other claimant's files contradicted their own extension and hit Tier 5 — the same bug via a second route. Measured on main: .ts + #!/usr/bin/env deno run and .scm + #!/usr/bin/csi -s both returned undeterminable with an Identity Masking flag, and deno+TypeScript is a mainstream combination.

Such triggers now yield no shebang verdict and the extension decides — correct for every claimant, and the same refuse-rather-than-guess posture Tier 4's margin requirement already takes. All four claimants (.ts/.js/.scm/.csx) now classify correctly.

2. tcl's path-shaped bin/expect trigger only matched under the old substring behaviour. Basename-normalising triggers at boot restores it, and upgrades it: #!/usr/bin/expect on a .tcl file now reaches Tier 0 consensus instead of extension-only Tier 2.

The trigger table is now built once in _calibrate_lookup_maps (normalised, longest-first, ambiguities dropped) rather than rebuilt per file.

Also: source_proof now names the mechanism that fired

_tier_2_fingerprint_check has always resolved shebangs and internal discriminators, but every caller labelled the result Shebang. So a .asm resolved by ACCTPGM CSECT reported Single Indicator (Shebang) with no #! anywhere in the file, and an identity conflict raised against a discriminator match reported a contradiction with a shebang that didn't exist. source_proof is a load-bearing claim in this engine — the mechanism now travels with the verdict (Shebang vs Internal Signature).

Tests

tests/core_engine/test_detection_resolution_order.py, 81 cases:

• the claimant list per contested extension in registry order (a pinned literal, so adding or reordering a claimant is a reviewable diff)
• the invariant that every multi-claimant extension is a registered collision — an unregistered one lets Tier 1 lock the last registrant and can raise a false Tier 5 against the other claimant's discriminator
• per-claimant behavioural resolution for 18 pairs across 9 contested extensions
• the shebang order-independence invariant, the ambiguity pin, the bin/expect case, and the six Shebang matching is unanchored substring search: canonical tclsh/wish/ts-node shebangs trigger false Identity Conflicts (Tier 5) #3116 regressions

9,594 core_engine+extraction tests pass; the single failure is the pre-existing fidelity-table pin staleness. ruff/mypy audits baseline-clean.

🤖 Generated with Claude Code

---

Update: two more fixes, found by the #3117 harness

Once the harness (#3130) existed, it was run against this branch and surfaced four more defects. Two are fixed here; the third is disproven twice and documented; the fourth is left open.

#3133 — the portable-trampoline idiom

#!/bin/sh followed by exec tclsh "$0" ${1+"$@"} is the canonical Tcl portability trick, and the crucible's two cases are sqlite's own test-harness header verbatim. The shebang genuinely is sh, so it legitimately contradicts the .tcl extension — and the Identity Conflict Trap read that as masquerading and refused the file at Tier 5 with an Identity Masking flag. A customer would see the scanner call a stock sqlite script a disguised payload.

A generic shell shebang is the standard bootstrap vehicle; an exec <interpreter> in the opening lines is what the file actually runs as. The escape hatch is deliberately narrow: only an interpreter the extension's own language claims clears the conflict, and only when the conflicting shebang resolved to the shell family. A .tcl that re-execs python3 still trips the trap — pinned by a test.

Note this is not the #3116 bug and was not fixed by the original commits here: there the matcher was wrong, here it is right.

#3134 — a real extension outranks a filename prefix

BUILD.mk locked to python at Tier 1 via Prefix Anchor (BUILD) before its own .mk was consulted. Now an extension outranks a prefix, while an exact filename match (Makefile, Dockerfile) still wins outright — those have no extension to defer to.

Scoped so a template extension does not outrank a prefix either: .in names a template, not a language, so Makefile.pre.in (which is Makefile syntax) stays makefile rather than following .in to m4. The unscoped version flipped that file, and the crucible caught it before it shipped.

#3132 — not fixed, and two candidate fixes measured and rejected

Both plausible fixes make detection materially worse, and both are now recorded as comments at the relevant lines so nobody re-derives them:

candidate fix	overall	contested subset
baseline (this branch)	0.9974	0.9859
gravity abstains on same-extension collisions	0.9840	0.9034
exclude the contested extension from discriminator scoring	0.9850	0.9095

Both break 32 sqlite files to db2_sql to recover 7 MicroPython files. The root cause is sharper than the issue's original framing: python lists .py as its own discriminators entry, which is literally where the reported "72% Local Dominance" comes from (base 14 + 14×2 = 42 against embedded_python's 14 + 1×2 = 16) — and that same accidental self-reference is what currently holds .sql together. It is load-bearing by accident, and .sql needs a real content signal before it can be removed.

I would have shipped the first of those two without the harness. That is the clearest argument for #3130 I can offer.

Measured effect of what did land

metric	before	after
overall accuracy	0.9964	0.9974
auto subset	0.9984	0.9996
refusals	3	1
Tier 5 conflicts	3	1
misclassified files	11	8

The 8 remaining are 7 × #3132 and 1 × the sh/PowerShell polyglot (download.ps1, which has no exec line and so is genuinely a different shape from the trampoline).

Golden masters

Regenerated in both environments, drift fully explained: 462 diffs but only three identity-level changes — LICENSE.adoc's proof and tier (same language), and the excluded-artifacts queue shrinking 548 → 546 as the two trampoline files join the analysed population. tcl composition goes 29 → 31, and the remaining ~459 diffs are that group's aggregates and population-relative z-scores recomputing around two new members. Zero language flips.

87 resolution-order tests (up from 81); 9,600 core_engine+extraction pass.

[github-actions]

🐦‍⬛ Muninn Security Scan

✅ No security issues found.

🐦‍⬛ Powered by Muninn · Skald Lab