feat(server): kb.explain_ranking — why a result ranked where it did

[minion1227]

Closes #432

_retrieve returns (kind, id, summary, score, backend) — one opaque score and a backend label. tuning fusion, the reranker, or the recency / pages-first signals means guessing: how much of a score came from lexical vs semantic rank, what the rrf contribution was, which stage moved a candidate, or which gate cut it. the existing --explain on vouch search and explain=True on build_context_pack emit {kind, id, score, backend}, which is the opaque score restated rather than a breakdown.

the shape

kb.explain_ranking re-runs the pipeline and snapshots every candidate's rank and score after each stage. a candidate present in one snapshot and absent from the next was removed by that stage — that is what the reported gate names.

$ vouch explain-ranking "jwt" --limit 2
backend: hybrid (configured hybrid, semantic unavailable)
stages active: fusion, recency, strategy=vouch.strategies.provenance

claim/c-dead  gate=status-filtered  lexical=1 semantic=None  rrf=0.016393
    hybrid         rank=1 score=0.016393
    scope_filter   rank=1 score=0.016393

claim/c2  gate=kept  lexical=2 semantic=None  rrf=0.016129
    hybrid         rank=2 score=0.016129
    scope_filter   rank=2 score=0.016129
    status_filter  rank=1 score=0.016129 rank+1
    recency        rank=1 score=0.016129
    pages_first    rank=1 score=0.016129 (off)
    rerank         rank=1 score=0.016129 (off)
    strategy       rank=1 score=0.016129
    limit          rank=1 score=0.016129

the retracted claim is explained up to the stage that cut it rather than being silently absent, which is the case the issue is really about. --format json emits the same thing machine-readably.

design notes worth a second pair of eyes

no context.py edits. every stage helper — rrf_fuse, filter_hits, _filter_live_hits, _maybe_recency, _maybe_pages_first, _maybe_rerank, _maybe_strategy — is already importable, so the instrumentation re-runs them instead of threading a trace flag through the hot path. that keeps the retrieval path byte-identical and means this does not conflict with #621, which is editing context.py right now.

the stage chain is a deliberate composition, not a copy of one caller. it chains _retrieve's ranking stages, the lifecycle gate search_kb applies, and the budget gate build_context_pack applies, so one call covers every stage an artifact can die at. scope filtering runs without a limit and truncation is its own limit stage — the same "scope first so status filtering can refill the window" ordering search_kb documents, and it keeps a candidate lost to the window attributable instead of folded into the scope filter. if you would rather this mirror exactly one caller's chain, say which and i will narrow it.

two stage shapes. recency / pages_first are rescoring-only, so their signal is the score delta; rerank / strategy are ordering-only, so theirs is the rank delta. a stage that is off still appears, flagged (off), so the chain reads continuously rather than going quiet.

the uncited gate is defensive, and does not drop. require_citations does not remove an item — build_context_pack keeps it and fails the pack — so uncited renames the gate on the candidate responsible for that failure. it also cannot fire today: Claim rejects evidence=[] on the model ("claim must cite at least one Source or Evidence id"), which closes every write path, so no stored claim is uncited. i kept the branch because it mirrors the check the pack still makes and starts reporting the moment that invariant is relaxed — but it is forced in tests rather than reachable through the public API, and i would rather flag that than have it read as covered behaviour. happy to drop it if you would prefer the gate list not carry an unreachable member.

registration

five sites, not four — test_hot_memory_universal_coverage requires every kb.* method to be declared in HOT_MEMORY_COVERED or HOT_MEMORY_EXCLUDED, which CLAUDE.md's "four registration sites" list does not mention. excluded here: a recency sidebar would perturb the output being inspected. worth adding to CLAUDE.md separately.

• server.py — kb_explain_ranking MCP tool
• jsonl_server.py — _h_explain_ranking + HANDLERS["kb.explain_ranking"]
• capabilities.py — METHODS
• cli.py — vouch explain-ranking
• hot_memory.py — HOT_MEMORY_EXCLUDED

review gate

read-only. no writes, no proposals, no kb.approve, no lifecycle change; storage.py and proposals.py are untouched. viewer scoping goes through the same filter_hits as kb.context, so nothing is exposed that the caller could not already retrieve — test_scope_filtered_candidate_is_attributed_to_scope pins that a viewer-invisible claim dies at scope_filter.

tests

30 cases in tests/test_explain_ranking.py, covering the two the issue names plus the branches around them:

• fused-only query reporting per-retriever ranks and the rrf contribution
• each retracted status (archived / superseded / redacted) and an archived page → status-filtered
• limit-dropped, budget-dropped, scope-filtered
• all four backend branches, the auto → substring fall-through, and a sqlite3.Error from a broken fts5 index degrading to no lexical hits
• disabled stages reported applied: False; recency's score delta; strategy naming the configured plugin
• all three surfaces (MCP / JSONL / CLI text + json) and the METHODS registration

verification

pytest tests/ -q --ignore=tests/embeddings   green
mypy src                                     Success: no issues found in 117 source files
ruff check src tests                         All checks passed!
diff-cover --fail-under 100                  100%, 151/151 changed lines in src/vouch

the diff-coverage gate from #619 is green on every changed line.

[minion1227]

note on the red trust-gate: it is unavoidable for this issue rather than something i can fix in the branch.

CORE_GLOBS in pr_bot.py includes capabilities.py, server.py, jsonl_server.py and cli.py — which are exactly the four registration sites #432 specifies, and the four test_capabilities requires for method-list parity. so any new kb.* method from a non-OWNER author trips the gate by construction; #618 is red for the same reason on a one-line server.py change. everything else is green on all three python versions, including the #619 diff-coverage gate at 100%.

the only way to make it pass would be to not register the method, which fails test_capabilities and the issue's own acceptance criteria. flagging rather than working around it — the owner review the gate asks for is the intended path.

one other thing worth a decision, repeated from the description since it is the part i would push back on if i were reviewing: the uncited gate cannot fire. Claim rejects evidence=[] on the model, so no stored claim is uncited, and the branch is forced in tests rather than reachable through the public API. i kept it to mirror the check build_context_pack still makes, but if you would rather the gate list not advertise an unreachable member, say so and i will drop it and the two tests that pin it.

[minion1227]

status update, and a correction to my earlier note: the trust-gate situation changed under this pr and it is no longer red — it is now absent, which is worse.

#630 merged into test at 10:59:59 today and deleted .github/workflows/trust-gate.yml. the context is still in the test ruleset's required checks, so nothing reports it: the head commit's status rollup returns state: pending with an empty statuses array. that is why this pr reads BLOCKED with every check green — there is no failure to fix, just a required check waiting on a workflow that no longer exists. #630's own changelog entry flagged exactly this and asked for the context to be removed from the ruleset.

so the earlier framing — that the gate was unavoidably red for a non-OWNER touching core paths — no longer applies. that gate is gone, and #630 says CODEOWNERS plus arm-auto-merge.yml hold the bar it was enforcing. this pr is core-path, so it still needs your review; it just no longer has a red check attached to it.

the block is repo-wide rather than specific to this pr: #623, #626 and #632 are all BLOCKED / MERGEABLE the same way, and #618 joins them once it is up to date. i opened #633 to fix the versionable half — scripts/setup_repo_guards.sh still declared trust-gate and coderabbit-approved, so re-running it as-is would have re-applied the bad list. the live ruleset (19022847, auto-merge guard (test)) still needs one owner action; the script matches it by name, so bash scripts/setup_repo_guards.sh test updates it in place.

nothing needed on the branch here — it is up to date with test (0 behind), ci is green on all three python versions, and diff coverage is 100%. the open question from the description still stands: whether to keep the unreachable uncited gate or drop it and its two tests.

[plind-junior]

LGTM!