Skip to content

fix(redact): relocate ^-anchored spans so egress masking stops dropping whole files (#2912) - #2929

Closed
aclansley wants to merge 1 commit into
garrytan:mainfrom
aclansley:fix/locate-span-anchored-patterns
Closed

aclansley wants to merge 1 commit into
garrytan:mainfrom
aclansley:fix/locate-span-anchored-patterns

Conversation

@aclansley

Copy link
Copy Markdown

What

locateSpan() cannot relocate an ^-anchored pattern, so redactFindingSpans() returns null and its caller drops the whole payload. This is the mechanism behind #2912.

Why it happens

A finding's col points at the reported span — the capture group when the pattern has one — which sits after the start of the whole match. locateSpan resumed the scan two characters before that span:

re.lastIndex = Math.max(0, offset - 2);
const m = re.exec(input);

env.kv is ^-anchored:

regex: /^[ \t]*(?:export[ \t]+)?[A-Z][A-Z0-9_]*(?:KEY|TOKEN|SECRET|...)[ \t]*=[ \t]*['"]?([^\s'"]{8,})['"]?/

The only position its ^ can match is the line start, and that is already behind lastIndex. exec scans forward, never matches the line the finding came from, and locateSpan returns -1. redactFindingSpans then returns null — which, per its own contract, the caller must read as "drop the whole payload".

The scope is wider than #2912 reports

#2912 frames this as a false positive on lines that read a key from the environment. The trigger is actually any env.kv match at all, so a file is dropped just as readily when the match is a genuine secret:

redactFindingSpans('DB_PASSWORD = "s3cr3t-Kx9mQ2vLp7Wn"', { repoVisibility: "private" })
// → null   (before this change)

That is the fourth case in the added test file. The practical effect is that the files most likely to contain credentials are the ones most likely to be withheld.

The fix

Resume the scan at the line start so an anchored match is reachable, then walk matches and accept only the one whose span lands exactly at the reported offset.

Fail-closed behaviour is unchanged: when no match lands at that offset, locateSpan still returns -1 and the payload is still dropped. The change makes a correct relocation possible; it does not make masking more permissive.

Verification

test/locate-span-anchored.test.ts (new) test/redact-engine.test.ts
before 0 pass / 5 fail 98 pass / 0 fail
after 5 pass / 0 fail 98 pass / 0 fail

Verified against main at v1.87.5.0.

test/gstack-redact-cli.test.ts, test/telemetry.test.ts and test/redact-audit-log.test.ts report an identical 124 pass / 37 fail both with and without this change — those 37 are pre-existing failures in my sandbox (they shell out through execSync) and are unrelated to it.

Deliberately not addressed

The other half of #2912 — env.kv firing on os.environ[ in the first place — is left alone. Tightening that validator trades against false negatives on real secrets, which felt like a maintainer's call rather than mine. This change stops the file loss either way, and the two are independent.

🤖 Generated with Claude Code

…ng files

locateSpan resumed the scan at the finding's span, but a finding's col
points at the capture group, which sits after the start of the whole
match. For an ^-anchored pattern (env.kv) the only position ^ can match is
the line start, already behind the resume point, so exec found nothing and
locateSpan returned -1.

redactFindingSpans treats -1 as "cannot mask" and returns null, which its
contract requires the caller to read as "drop the whole payload" (garrytan#2912).
Any file carrying an env-style assignment was therefore withheld in full,
including files whose match was a genuine secret rather than the reported
false positive.

Resume the scan at the line start so an anchored match is reachable, and
keep requiring the span to land exactly at the reported offset, so the
fail-closed contract is unchanged.

Adds test/locate-span-anchored.test.ts: five cases, all failing before this
change and passing after. test/redact-engine.test.ts stays at 98/98.

Refs garrytan#2912

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@trunk-io

trunk-io Bot commented Sep 22, 2026

Copy link
Copy Markdown

Merging to main in this repository is managed by Trunk.

  • To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

garrytan added a commit that referenced this pull request Sep 23, 2026
* fix(settings): preserve symlinked settings targets

Resolve the selected target for locking, mutation, backup, and rollback; refuse target changes and preserve private modes. Addresses #2830.

* fix(redact): bind masking to original detected spans

Inspired by #2929's anchored-span diagnosis; independently implemented using normalization offsets. Addresses #2930 and the relocation portion of #2912 without changing detection sensitivity.

* fix(evals): exclude operator credentials from prefix admission

Adapts the credential-suffix screen proposed in #2636, with real launched-child regression coverage and deliberate provider-auth exceptions.

* fix(artifacts): retain custom allowlist rules on reinitialization

Preserve the exact user-owned suffix and publish only a successfully assembled replacement. Independently implements the repair reported in #2907.

* test(cso): verify exact masked reads and unmaskable payload refusal

* fix(cso): preserve exact filesystem identities through lease recovery

Preserve 64-bit device/inode identity and nanosecond race checks. Add native NTFS lifecycle coverage for #2927; retain ambiguous legacy-state refusal without claiming Windows PID-reuse recovery is resolved.

* fix(redact): bind pre-push scans to destination and preserve seam context

Uses #2935 (bd07318) as source evidence for push-target range and slice-overlap defects. Independently implemented; no cherry-pick or release metadata adoption.

* test(ci): gate native agent ownership and settings links on macOS

* fix(browse): bind agent lifetimes and cleanup to owned generations

Uses #2931 by Chris Hutton / Claude Fable 5.1 as attributed design input; independently implemented without broad sweeps or copied code. Keep uncertain children and locks rather than deleting foreign state.

* test(ci): include concurrent shutdown controls in the native macOS gate

* v1.88.1.0 fix: harden credential boundaries and owned state

* fix(redact): preserve target provenance and scan boundary semantics

* test(artifacts): read managed rules from atomic allowlist assembly

* fix: preserve native exit observations and fixture prerequisites

* fix: preserve UTF-16 offsets through redaction normalization
@capy-ai

capy-ai Bot commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

Thank you @aclansley for isolating the anchored-match relocation failure and proposing the exact-position check. Your diagnosis and anchored-span approach are explicitly credited in the maintainer integration, #2942, merged as b9706f3 for v1.88.1.0.

The shipped implementation binds masking to the original spans captured during scanning, rather than searching the raw text again. It also preserves normalization offsets, overlapping matches, and fail-closed behavior. See lib/redact-engine.ts, the span-binding regressions, and the Unicode-offset regressions.

Closing this PR as superseded by that integration, not as a merge or wholesale adoption of this branch. As you correctly separated in your proposal, detector false positives are a different problem; #2912 remains open for that and its outstanding size-limit concerns. Thanks for keeping the original fix focused and fail-closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant