fix(review): reduce secret-scan false positives and surface finding locations - #3178
Conversation
…ocations Exclude lowercase-hyphenated word compounds (e.g. the "installation-token" test fixture used 351+ times in this repo's own suite) from the generic secret-assignment heuristic, and surface file:line locations in the secret_leak finding so a hard-blocking match can be verified without re-deriving it from the whole diff. Fixes #3041
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #3178 +/- ##
=======================================
Coverage 94.72% 94.72%
=======================================
Files 269 269
Lines 29619 29658 +39
Branches 10806 10814 +8
=======================================
+ Hits 28056 28095 +39
Misses 917 917
Partials 646 646
🚀 New features to boost your workflow:
|
|
Warning 🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 ⏸️ Gittensory review result - manual review recommendedReview updated: 2026-07-04 19:13:48 UTC
⏸️ Suggested Action - Manual Review
Review summary Nits — 6 non-blocking
Concerns raised — review before merging
Review context
Contributor next steps
Signal definitions
🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed 💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →. Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.
|
) PR #5346 (a resubmission of #5341) was auto-closed over two inert test-fixture strings that matched the generic_secret_assignment keyword-plus-quoted-value SHAPE but weren't real credentials -- the same heuristic has now caused at least eight prior false-positive incidents (#2613, #3178, #3673, #3866, #4587, #4733, plus several fixture-rewording commits), each patched by narrowing an allowlist rather than fixing the underlying design. REES's own copy of this rule already rates it "medium confidence" ("catches real keys but also the occasional long opaque non-secret"), and content-lane/security-scan.ts's own header states the design principle this violated: a gate that auto-closes with no human queue may only hard-close on a signal unambiguous enough that a false positive is essentially impossible. Split generic_secret_assignment out of HARD_SECRET_KINDS into a new ADVISORY_ONLY_SECRET_KINDS: it still surfaces (a warning-severity possible_secret_assignment finding / a "manual" content-lane verdict), but never auto-blocks or auto-closes on its own. Concrete credential formats (github_token, aws_access_key, private_key_block, ...) are unaffected and remain unconditional hard blockers. Also add a structural placeholder heuristic (looksLikeDescriptive PlaceholderPhrase, mirrored in REES): a value with 5+ lowercase-only hyphen/underscore segments containing an English function word reads as written prose describing the value, not a credential or a chosen passphrase -- this independently resolves both PR #5346 literals without weakening detection of a genuine human-chosen passphrase like "correct-horse-battery-secret" (no function words, by design).
Summary
generic_secret_assignmentrule: a captured value madeentirely of lowercase words joined by hyphens (2+ segments, e.g.
installation-token— the test-fixturemock token used 351+ times across this repo's own test suite) is now treated as a non-secret. Real
secrets/tokens are essentially always alphanumeric, mixed-case, or base64/hex, never a pure
lowercase-hyphenated phrase, so this exclusion stays narrow (requires ≥1 hyphen) and doesn't broaden into
excluding arbitrary single lowercase words. This is the confirmed root cause of PR fix(review): key the durable CI-state cache on resolved required contexts #3036's wrongful
hard-block (
test/unit/queue.test.ts'sResponse.json({ token: "installation-token" })fixture).secret_leakfinding previously carried no file:linelocation at all, forcing a maintainer to re-derive it from the whole diff. Added
scanDiffForSecretsWithLocationsinsrc/review/secrets-scan.ts, which walks abuildSecretScanDiff-shapeddiff line by line (tracking file headers, hunk headers, and +/-/context line types) and returns each
pattern hit with its file path and 1-based line number in the new/post-change file (or
line: 0for asecret-shaped filename on an added/renamed file header, preserving the previous header-scanning behavior).
secretLeakFindinginsrc/review/safety.tsnow calls this directly on the raw diff (no morepre-filtering to an added-only text blob) and appends up to 5
path:linelocations to the finding'sdetail, noting how many more were omitted beyond that cap.SECRET_PATTERNS+hasGenericSecretAssignment) was factored into one sharedmatchedKindsInhelper that bothscanForSecrets(unchanged signature/behavior) and the newscanDiffForSecretsWithLocationsdelegate to, so there is exactly one place the pattern list is applied.path for a confirmed false positive. That's a separate, larger design decision; left as a follow-up rather
than built here.
Fixes #3041
Scope
type(scope): short summaryConventional Commit format, for examplefix(api): restore profile access checks.CONTRIBUTING.mdand does not reintroduce GitHub Pages, VitePress,site/, orCNAME.Validation
git diff --checknpm run actionlintnpm run typechecknpm run test:coveragelocally — both changed files are at 100%:src/review/secrets-scan.ts55/55lines, 34/34 branches, 8/8 functions;
src/review/safety.ts19/19 lines, 12/12 branches, 7/7functions. Full suite: 8630 passed, 7 skipped.
npm run test:workersnpm run build:mcpnpm run test:mcp-packnpm run ui:openapi:checknpm run ui:lintnpm run ui:typechecknpm run ui:buildnpm audit --audit-level=moderate— 0 vulnerabilitiesAlso ran (not template-listed, but part of this repo's full local gate):
npm run db:migrations:check,npm run db:schema-drift:check,npm run cf-typegen:check,npm run selfhost:validate-observability,npm run ui:test,npm run ui:version-audit,npm run rees:test(677 passed) — all green.If any required check was skipped, explain why:
npm run selfhost:env-reference:check(part of the localtest:ciaggregator, but not part of thevalidate-codeGitHub Actions job) fails on a cleanorigin/maincheckout with no changes from this PR —apps/gittensory-ui/src/lib/selfhost-env-reference.tsdrifted from a prior, unrelated merge. Verified viagit stashthat this failure is present before any of this PR's changes are applied. This PR touchesneither the self-host env-var surface nor that generated file, so regenerating it here would be out of
scope; flagging as a pre-existing, separate issue rather than fixing it in this PR.
Safety
UI Evidencesection below with JPG/JPEG or PNG screenshots arranged as organized, captioned, clickable thumbnails. SVG screenshots are not used as review evidence. Review-only screenshots or recordings are not committed to the repository.This is a deterministic, security-relevant scanner change, so extra care was taken to verify it doesn't
regress detection of real secrets:
test/unit/secrets-scan.test.tsandtest/unit/safety-wiring.test.tstests passunmodified — including every format-specific pattern (github_token, github_pat, aws_access_key,
slack_token, google_api_key, gitlab_token, npm_token, stripe_secret_key, sendgrid_key,
huggingface_token, jwt, private_key_block) and the existing
generic_secret_assignmentpositive case(
sk_live_+ mixed-case/digit value), none of which are touched by the new exclusion.still flags after the fix, proving the new lowercase-hyphenated-compound exclusion is narrow and doesn't
broaden past its intended shape.
confirming the regex specifically requires 2+ hyphen-joined segments.
SECRET_PATTERNSregexes (github_token, aws_access_key, etc.) are completelyuntouched — the fix only changes
isPlaceholderSecretValue, which exclusively gates thegeneric_secret_assignmentheuristic path.secretLeakFinding's existing "scans only added lines / added-or-renamed file headers, neverremoved/context lines" behavior is preserved exactly — now implemented via
scanDiffForSecretsWithLocations's own line-type handling instead of a pre-filter — and all of thatbehavior's original tests pass unmodified, plus a new explicit test that a removed line's secret-shaped
content never appears in the finding.
UI Evidence
N/A — backend-only change (
src/review/**), no UI/frontend/docs/extension surface touched.Notes
before this PR was written.