fix(explain): scope-filtered candidates leak their summary to the viewer - #640
Merged
plind-junior merged 1 commit intoJul 30, 2026
Conversation
kai392
force-pushed
the
fix/critical-issue-explain-ranking-scope-leak
branch
4 times, most recently
from
July 30, 2026 15:21
a5af5d7 to
a270074
Compare
the candidate list is built from every hit the pipeline saw, and summaries were keyed off that same pre-scope fused set. so a viewer scoped to one project got back the text of claims `kb.search` and `kb.context` withhold from it — the module docstring promises the opposite, and the surface is exposed over mcp, jsonl and the cli. listing the candidate is deliberate: naming the gate that hid it is what makes the report useful for tuning, and `test_scope_filtered_candidate_is_attributed_to_scope` pins that. so this keeps the row and its `scope-filtered` gate and withholds only the text, by sourcing summaries from the scoped set rather than the fused one. a candidate the viewer can retrieve is unaffected.
Member
|
Checked this against test and the diagnosis holds. Summaries was keyed off the pre-scope hits while scoped sat ten lines above it, so One followup, not for this PR: The report still exposes a filtered candidate's id and ranks. It's fine while scope is a relevance filter, but #608 makes the viewer no longer caller-declared, and that turns existence disclosure into the next question here. |
3 tasks
plind-junior
pushed a commit
that referenced
this pull request
Jul 30, 2026
a retracted/superseded/redacted claim or archived page correctly reported gate: "status-filtered", but explain_ranking sourced summaries from `scoped` (post scope-filter, pre status-filter), so its full live text still came back regardless of which stage actually dropped it. #640 fixed the equivalent leak for the scope_filter stage, moving summaries off the raw fused set. that fix's chosen source, `scoped`, is itself still upstream of `_filter_live_hits`, so the same withholding was never extended to status_filter. source summaries from `live` instead — the set that survived both the scope and status gates, matching what kb.search/kb.context expose. new tests mirror #640's scope-filtered regression test for a superseded claim and an archived page, asserting an empty summary and that the secret text is absent from the dumped response. Fixes #650
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
explain_rankingsources candidate summaries from the scoped hit setinstead of the pre-scope fused set. one word in the dict comprehension; the
candidate list itself is unchanged.
Why
the candidate loop deliberately walks every hit the pipeline saw, so a dropped
artifact is still explained rather than silently missing. but
summarieswaskeyed off that same pre-scope set, so the summary rode along for candidates
filter_hitshad just removed. the result is that a viewer gets back the textof artifacts
kb.searchandkb.contextwithhold from it — the moduledocstring promises exactly the opposite:
same viewer, same query, on
testbefore this patch:kb.explain_rankingis registered on all four surfaces — the mcp tool, thejsonl handler,
METHODS, and the cli — and on the remote transports theviewer arrives as a request param, so this is the read path that answers with
content the sibling read paths refuse.
worth being precise about the severity:
scoping.pysays retrieval surfacestreat scope as a relevance filter and artifacts stay readable as plaintext
on disk, so this is not a local privilege boundary. it matters for the
multi-project and remote-transport shapes — the same reason
kb.auditgrewviewer scoping and
tests/test_audit_scoping.py— and for the scoped-tokenwork in #608, where the viewer stops being caller-declared.
What might break
nothing on disk, no method signature changes, no
kb.*shape change — thesummarykey is still always present, just""for a candidate the viewercould not retrieve.
the deliberate part of the design is preserved on purpose. i read
test_scope_filtered_candidate_is_attributed_to_scopeas pinning intent:listing the hidden candidate and naming its gate is what makes the report
useful for tuning. so the row, the
scope-filteredgate and the stage listall stay exactly as they were — that test passes untouched. only the text is
withheld. if you'd rather drop such candidates from the report entirely, say
so and i'll switch it, but that would reverse a choice #628 looks to have
made on purpose.
VEP
not a surface change — no vep.
Tests
make checkpasses locally (lint + mypy + pytest)CHANGELOG.mdupdated under## [Unreleased]tests/test_explain_ranking.py::test_scope_filtered_candidate_does_not_carry_its_summary— asserts the gate is still reported, the summary is
"", the secret stringappears nowhere in the serialised result, and a retrievable candidate still
carries its summary. it fails on the previous code with the leaked text in the
assertion diff.
validation run:
no ui files touched, so no screenshots.