Skip to content

fix(context): apply rerank on embedding and fts5 backends too - #594

Merged
plind-junior merged 2 commits into
vouchdev:testfrom
atoz96:fix/rerank-parity-across-backends
Jul 30, 2026
Merged

fix(context): apply rerank on embedding and fts5 backends too#594
plind-junior merged 2 commits into
vouchdev:testfrom
atoz96:fix/rerank-parity-across-backends

Conversation

@atoz96

@atoz96 atoz96 commented Jul 29, 2026

Copy link
Copy Markdown

retrieval.rerank is global config, but _retrieve only runs the rerank stage on the auto/hybrid branch. an operator who pins retrieval.backend to embedding or fts5 and sets retrieval.rerank.enabled: true gets no reranking at all — silently. the setting reads as active in config.yaml and does nothing.

why this looks like an oversight rather than a decision

the embedding branch already carries the comment for exactly this class of bug:

# Parity with the hybrid path: an operator who opted into
# recency gets it regardless of which backend serves the query.
filtered = _maybe_recency(store, hits=filtered)

recency was given parity across branches; rerank was missed at the same seam. nothing in _configured_rerank scopes the setting to a backend — it reads retrieval.rerank and returns (enabled, top_k) for any caller.

the change

same stage in the same order — recency, pages-first, then rerank — on both single-retriever branches.

the substring fall-through is deliberately left alone: it is the last-resort scan after both retrievers came back empty, and it applies none of the three stages today. widening that is a separate decision.

tests

test_context_rerank_applies_on_single_retriever_backends, parametrized over embedding and fts5. it fails on the previous code with the unreranked order:

AssertionError: assert ['c1', 'c2'] == ['c2', 'c1']

verification

pytest tests/ -q --ignore=tests/embeddings   green
mypy src                                     Success: no issues found in 115 source files
ruff check src tests                         All checks passed!

retrieval.rerank is global config, but _retrieve only ran the rerank
stage on the auto/hybrid branch. an operator who pinned
retrieval.backend to embedding or fts5 and set
retrieval.rerank.enabled: true got no reranking at all, silently — the
setting reads as active in config.yaml and does nothing.

the embedding branch already carries the comment for exactly this class
of bug ("parity with the hybrid path: an operator who opted into recency
gets it regardless of which backend serves the query"); rerank was
missed when that parity was established. same stage, same order —
recency, pages-first, then rerank — on both single-retriever branches.

the substring fall-through is left alone: it is the last-resort scan
after both retrievers came back empty, and it applies none of the three
stages today.

covered by a parametrized regression over both backends; it fails on the
previous code with the unreranked order.
@coderabbitai

coderabbitai Bot commented Jul 29, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@plind-junior, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 59 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: a32371df-0ea0-42f5-a039-010adac46304

📥 Commits

Reviewing files that changed from the base of the PR and between 647c34d and a2159ef.

📒 Files selected for processing (2)
  • src/vouch/context.py
  • tests/test_retrieval_backend.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added retrieval context, search, synthesis, and evaluation tests tests and fixtures size: XS less than 50 changed non-doc lines labels Jul 29, 2026
@plind-junior
plind-junior merged commit 33fcb1b into vouchdev:test Jul 30, 2026
18 of 19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

retrieval context, search, synthesis, and evaluation size: XS less than 50 changed non-doc lines tests tests and fixtures

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants