Skip to content

feat(review): capture bounded raw context in configured-blocker fired events, permanently excluding secret_leak - #8135

Merged
loopover-orb[bot] merged 1 commit into
JSONbored:mainfrom
cleanjunc:feat/8130-blocker-raw-context
Jul 23, 2026
Merged

feat(review): capture bounded raw context in configured-blocker fired events, permanently excluding secret_leak#8135
loopover-orb[bot] merged 1 commit into
JSONbored:mainfrom
cleanjunc:feat/8130-blocker-raw-context

Conversation

@cleanjunc

Copy link
Copy Markdown
Contributor

Summary

Closes #8130.

Extends #8104's generic recordConfiguredGateBlockerSignals (src/rules/advisory.ts) so every non-excluded configured-blocker code captures the raw context its detection actually evaluated — making those corpora rich enough to backtest logic/detection changes, not just thresholds — while permanently excluding secret_leak by design.

  • RAW_CONTEXT_EXCLUDED_CODES — the exported, named constant the issue mandates (secret_leak inside), with the full security reasoning attached: capturing the diff that triggered a secret_leak finding would store the leaked credential itself in the calibration audit trail. The comment also carries the issue's Boundaries note (additions require an explicit, maintainer-reviewed redaction design — never a quiet edit).
  • RAW_CONTEXT_MAX_DIFF_CHARS = 120000 — mirrors src/services/ai-review.ts's own input.diff.slice(0, 120000) bound exactly (the same number, with a keep-in-sync comment), never a newly invented bound.
  • AI-judgment codes (ai_consensus_defect/ai_review_split): the metadata gains the AI review's own diff, threaded from the one caller (src/queue/processors.ts) via the already-memoized getReviewFiles() + the same buildAiReviewDiff the review consumed — never re-fetched or re-derived.
  • Every other non-excluded code — audited individually per the issue's requirement (the audit is recorded in the function's doc comment): missing_linked_issue reads linkage state, duplicate_pr_risk reads sibling-PR overlap, pre_merge_check_required/cla_check_unresolved read check-run conclusions, manifest_missing_tests reads changed paths vs manifest expectations, the review-thread code reads unresolved-thread state — none evaluates raw diff content, so each captures its detection's own recorded detail (the narrative of exactly that evaluated signal) as rawSignal, same bound.
  • linked_issue_scope_mismatch remains skipped entirely (calibration: wire linked_issue_scope_mismatch into the shared signal-tracking module (fired + reversal) #8101's own site records it — unchanged), and the calibration: wire recordRuleFired/recordHumanOverride for the remaining isConfiguredGateBlocker codes (excludes linked_issue_scope_mismatch, covered by #8101) #8104 behavior for confidence capture is preserved byte-for-byte.

Scope

  • The PR title follows type(scope): short summary Conventional Commit format, for example fix(api): restore profile access checks.
  • This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
  • This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
  • I linked a currently open issue this PR resolves (e.g. Closes #123) — a linked open issue is required for every contributor PR.

Validation

  • git diff --check
  • npm run actionlint
  • npm run typecheck
  • npm run test:coverage locally; codecov/patch requires ≥99% coverage of the lines AND branches you changed (aim for 100% on your diff so CI variance does not fail near the threshold). Global coverage is a non-blocking trend with a loose 90% backstop, not the gate.
  • npm run test:workers
  • npm run build:mcp
  • npm run test:mcp-pack
  • npm run ui:openapi:check
  • npm run ui:lint
  • npm run ui:typecheck
  • npm run ui:build
  • npm audit --audit-level=moderate
  • New or changed behavior has unit/integration tests for new branches, fallback paths, and sanitizer boundaries

If any required check was skipped, explain why:

Safety

  • No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
  • Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
  • Auth, cookie, CORS, GitHub App, Cloudflare, or session changes include negative-path tests.
  • API/OpenAPI/MCP behavior is updated and tested where needed.
  • UI changes use live API data or real empty/error/loading states, not production mock/demo fallbacks.
  • Visible UI changes include a UI Evidence section below with JPG/JPEG or PNG screenshots arranged as organized, captioned, clickable thumbnails. SVG screenshots are not used as review evidence. Review-only screenshots or recordings are not committed to the repository.
  • Public docs/changelogs are updated where needed; changelogs are only edited for release-prep PRs.

The three unchecked Safety boxes are N/A: no auth/session/UI change. Note the change is itself security-motivated in part: the secret_leak exclusion exists precisely so no leaked credential can ever enter the calibration audit trail, and that property is pinned by a dedicated negative test.

UI Evidence

N/A — no UI change.

Notes

@cleanjunc
cleanjunc requested a review from JSONbored as a code owner July 23, 2026 00:05
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 23, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.37%. Comparing base (c02a277) to head (0ef6ac5).
⚠️ Report is 6 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8135      +/-   ##
==========================================
- Coverage   92.02%   90.37%   -1.66%     
==========================================
  Files         760       99     -661     
  Lines       77394    26010   -51384     
  Branches    23392     5108   -18284     
==========================================
- Hits        71225    23506   -47719     
+ Misses       5061     2233    -2828     
+ Partials     1108      271     -837     
Flag Coverage Δ
shard-1 79.48% <83.33%> (+22.38%) ⬆️
shard-2 34.97% <50.00%> (-19.37%) ⬇️
shard-3 40.82% <100.00%> (-9.51%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/queue/processors.ts 95.76% <100.00%> (ø)
src/rules/advisory.ts 97.95% <100.00%> (+0.07%) ⬆️

... and 661 files with indirect coverage changes

@loopover-orb loopover-orb Bot added the gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. label Jul 23, 2026
@loopover-orb

loopover-orb Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Tip

✅ LoopOver review result - approve/merge recommended

Review updated: 2026-07-23 00:30:52 UTC

3 files · 1 AI reviewer · no blockers · readiness 100/100 · CI green · clean

✅ Suggested Action - Approve/Merge

  • safe to merge

Review summary
This PR extends recordConfiguredGateBlockerSignals to capture bounded raw-context metadata (the AI review's diff for ai_consensus_defect/ai_review_split, or the finding's own detail string for every other non-excluded code) while explicitly hard-excluding secret_leak via a named, exported RAW_CONTEXT_EXCLUDED_CODES set. The exclusion logic in src/rules/advisory.ts:1108-1115 is correctly ordered (confidence is set first, then the excluded-code check gates diff/rawSignal capture), and the new caller wiring in src/queue/processors.ts threads buildAiReviewDiff(await getReviewFiles()) through the already-memoized file fetch rather than re-fetching. Test coverage in configured-gate-blocker-signals.test.ts directly exercises the security-critical secret_leak exclusion path plus both AI-code and non-AI-code capture branches.

Nits — 5 non-blocking
  • src/rules/advisory.ts:1108 reuses RAW_CONTEXT_MAX_DIFF_CHARS (120000) to bound `finding.detail` (rawSignal) even though that constant's stated purpose is mirroring the AI review's diff-slice bound — consider a separate, smaller constant for the detail-string bound since detail text is never diff-sized.
  • src/queue/processors.ts: buildAiReviewDiff(await getReviewFiles()) is now called on every `if (evaluation)` pass regardless of whether any AI-judgment finding is present in configuredBlockers — relies entirely on getReviewFiles() being memoized to stay cheap; worth a one-line comment noting that dependency so a future refactor doesn't silently drop the memoization.
  • The doc comment on recordConfiguredGateBlockerSignals says the AI diff is 'never re-fetched or re-derived,' but the call site actually re-derives it via buildAiReviewDiff(await getReviewFiles()) rather than reusing a stored diff variable from the earlier AI review step — technically correct (same memoized files, same pure function) but the wording overstates it slightly.
  • Consider renaming RAW_CONTEXT_MAX_DIFF_CHARS to something bound-agnostic (e.g. RAW_CONTEXT_MAX_CHARS) since it now bounds both diff and non-diff (detail) content.
  • A short comment at the buildAiReviewDiff(await getReviewFiles()) call site confirming getReviewFiles() is memoized would help future readers avoid assuming this is a fresh fetch per gate evaluation.

Decision drivers

  • ✅ Code review — No blockers (1 reviewer)
  • ✅ Gate result — Passing (No configured blocker found.)
Context & advisory signals — never blocks the verdict
Signal Result Evidence
Linked issue ✅ Linked #8130
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 51 registered-repo PR(s), 16 merged, 33 issue(s).
Contributor context ✅ Confirmed Gittensor contributor cleanjunc; Gittensor profile; 51 PR(s), 33 issue(s).
Improvement ✅ Minor risk: clean · value: minor · LLM: moderate
Linked issue satisfaction

Addressed
The diff adds an exported RAW_CONTEXT_EXCLUDED_CODES constant with secret_leak and the required rationale comment, threads the bounded AI review diff for ai_consensus_defect/ai_review_split from the existing caller without re-fetching, captures a detail-based rawSignal fallback for non-diff codes after an audit noted in the doc comment, and includes tests covering the secret_leak negative case, th

Review context
  • Author: cleanjunc
  • Role context: outside_contributor
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: not available
  • Official Gittensor activity: 51 PR(s), 33 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Keep the PR focused and include validation evidence before maintainer review.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
🧪 Chat with LoopOver

Ask LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @loopover ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @loopover chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @loopover mention with a real question is routed to the closest matching read-only command automatically — no exact syntax required.

Full command reference: https://loopover.ai/docs/loopover-commands

🧪 Experimental — new and may change.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

  • Re-run LoopOver review

@loopover-orb loopover-orb Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LoopOver approves — the gate is satisfied and CI is green.

@loopover-orb
loopover-orb Bot merged commit 028079e into JSONbored:main Jul 23, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

calibration: capture bounded raw context for the remaining isConfiguredGateBlocker codes, excluding secret_leak

1 participant