Skip to content

fix(rag): scope chunk reads by repository - #4651

Merged
JSONbored merged 1 commit into
mainfrom
codex/fix-issue-rag-path-leakage
Jul 11, 2026
Merged

fix(rag): scope chunk reads by repository#4651
JSONbored merged 1 commit into
mainfrom
codex/fix-issue-rag-path-leakage

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Motivation

  • Prevent a cross-repository RAG metadata leak where vector matches from a truncated shared namespace could expose another repo's retrieved paths and text when only the requested repo was authorized.

Description

  • Scope chunk-text lookups by project and repo so vector matches can only hydrate text when the stored repo_chunks row belongs to the authorized repository by changing retrieveContextWithMetrics to call readChunkTexts(storage, project, repo, ids).
  • Update readChunkTexts to accept project + repo and include WHERE project = ? AND repo = ? AND id IN (...) so D1 reads cannot return victim repo rows for a different requested repo.
  • Add a regression unit test that constructs two repo names with the same truncated RAG namespace and asserts that a vector match whose chunk row belongs to the colliding (victim) repo is dropped (no context, kept chunks, or retrieved paths).
  • Regenerate Cloudflare worker runtime types after the local cf-typegen check reported drift.

Testing

  • Ran npx vitest run test/unit/rag.test.ts, which passed the modified and new tests for retrieveContextWithMetrics and readChunkTexts.
  • Ran git diff --check and npm run cf-typegen, both of which succeeded and produced the updated worker-configuration.d.ts artifact.
  • Ran npm run test:ci, which failed to complete due to existing timeout failures in unrelated test/unit/queue.test.ts sweep tests during the coverage run; the local coverage run was stopped after those timeouts and is not addressed by this change.
  • Ran npm audit --audit-level=moderate, which failed due to the npm registry audit endpoint returning 403 Forbidden (environmental/registry issue, not caused by this change).

Codex Task

@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 10, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.22%. Comparing base (3ae7ece) to head (ae4d002).
⚠️ Report is 7 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4651   +/-   ##
=======================================
  Coverage   94.22%   94.22%           
=======================================
  Files         441      441           
  Lines       38741    38741           
  Branches    14117    14117           
=======================================
  Hits        36504    36504           
  Misses       1577     1577           
  Partials      660      660           
Files with missing lines Coverage Δ
src/review/rag.ts 100.00% <100.00%> (ø)
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 10, 2026
@loopover-orb

loopover-orb Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Warning

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

⏸️ Gittensory review result - manual review recommended

Review updated: 2026-07-11 00:22:42 UTC

2 files · 1 AI reviewer · 2 blockers · readiness 93/100 · CI green · unstable

⏸️ Suggested Action - Manual Review

  • No linked issue detected — If this PR is intended to solve an issue, link it explicitly in the PR body.
  • Maintainer requires a linked issue — Link the relevant issue (for example Closes #123) before opening the PR.

Review summary
The diff correctly scopes `readChunkTexts` by `project` and `repo` via a `WHERE project = ? AND repo = ? AND id IN (...)` clause and updates the sole call site in `retrieveContextWithMetrics` to pass `opts.project`/`opts.repo`, directly closing the cross-repo vector-namespace collision described in the motivation. The new test builds two repo names that collide under `ragNamespace`'s truncation and asserts the victim's chunk row is dropped when the requester's project/repo don't match the bound query params — this exercises the real D1 filtering behavior rather than a fabricated scenario, since the mock only returns rows when the bound values equal the victim's stored key. The bundled `worker-configuration.d.ts` regen is unrelated generated churn but is disclosed as a required `cf-typegen` drift fix.

Nits — 5 non-blocking
  • src/review/rag.ts: the PR bundles a large, unrelated `worker-configuration.d.ts` regen with a security fix — consider splitting generated-artifact updates into their own PR going forward for cleaner history.
  • The PR description doesn't reference a tracked issue for this fix; even for a security-motivated change, linking the originating report/issue would help future readers understand provenance.
  • test/unit/rag.test.ts: consider adding a case where the vector match's chunk row belongs to the *same* repo/project to explicitly assert the positive (non-dropped) path stays intact alongside the new negative-path test.
  • Confirm `readChunkTexts`'s catch-path error logging includes enough context (project/repo) to debug scoped-query failures in production, since the WHERE clause is now more specific.
  • If other call sites of `readChunkTexts` exist beyond `retrieveContextWithMetrics`, verify they were all updated to the new 4-arg signature (not shown in this diff).

Concerns raised — review before merging

  • No linked issue detected — If this PR is intended to solve an issue, link it explicitly in the PR body.
  • Maintainer requires a linked issue — Link the relevant issue (for example Closes #123) before opening the PR.
Signal Result Evidence
Code review ❌ 2 blockers 1 reviewer
Linked issue ⚠️ Missing No linked issue or no-issue rationale found.
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (no linked issue context).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 48 registered-repo PR(s), 40 merged, 275 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 48 PR(s), 275 issue(s).
Gate result ❌ Blocking Repo-configured hard blocker found.
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: not available
  • Official Gittensor activity: 48 PR(s), 275 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • Explain no-issue PR.
  • Link the issue being solved, or explicitly explain why this is a no-issue PR.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@loopover-orb loopover-orb Bot added the manual-review Gittensor contributor context label Jul 10, 2026
@JSONbored
JSONbored force-pushed the codex/fix-issue-rag-path-leakage branch from ab457e5 to ae4d002 Compare July 11, 2026 00:12
@JSONbored
JSONbored merged commit cf19a4e into main Jul 11, 2026
11 checks passed
@JSONbored
JSONbored deleted the codex/fix-issue-rag-path-leakage branch July 11, 2026 00:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. manual-review Gittensor contributor context

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant