Skip to content

fix(observability): route RAG pipeline errors through console.error - #3917

Merged
JSONbored merged 1 commit into
mainfrom
fix/rag-pipeline-error-observability
Jul 7, 2026
Merged

fix(observability): route RAG pipeline errors through console.error#3917
JSONbored merged 1 commit into
mainfrom
fix/rag-pipeline-error-observability

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

  • Four catch blocks in src/review/rag.ts (embedTexts, upsertChunks, deleteChunksForPaths, readChunkTexts) and two in src/review/rag-index.ts (indexRepo, reindexChangedPaths) logged failures at console.log with no level field — invisible to the central Sentry forwarder. The sibling catch in the same file (retrieveContextWithMetrics) was already fixed to console.error with level:"error"; this extends the same fix to the rest of the module.
  • rag.ts's four fixes reuse that sibling's exact event: "review_context_fetch_failed" / contextType: "rag" umbrella (searchable together in Sentry) plus each function's own specific ev tag for log continuity. No new import was added to rag.ts — it declares itself a "SELF-CONTAINED NATIVE PORT... no imports" in its own header comment, so I kept that invariant intact rather than importing the metrics module there.
  • rag-index.ts's two fixes mirror its own already-fixed siblings (rag_index_tree_error, rag_list_paths_error) and additionally register + increment a new gittensory_rag_pipeline_errors_total{op} counter, since this file already imports other dependencies and has no such constraint — closing the gap where an embedding-provider outage was invisible to GittensoryQdrantErrorRateHigh (that counter only fires inside the Qdrant adapter itself).
  • Extended existing fail-safe tests in rag.test.ts with console.error assertions, and added two new tests reaching indexRepo/reindexChangedPaths's own outer catch (every other I/O step in those functions already self-catches, so a thrown Cloudflare-binding access — a realistic Workers-runtime failure mode — is the one way to reach it) confirming both the log shape and the new counter.

Found via a fresh performance/scalability/accuracy hardening audit of the self-host ORB stack. Tracked under #1667.

Scope

  • src/review/rag.ts — 4 catches: console.log → console.error + level:error
  • src/review/rag-index.ts — 2 catches: console.log → console.error + level:error + new counter
  • src/selfhost/metrics.ts — register gittensory_rag_pipeline_errors_total
  • test/unit/rag.test.ts, test/unit/rag-index.test.ts — coverage for every changed line/branch

Validation

  • npm run typecheck
  • npm run test:coverage (full unsharded) — 545 files / 10824 tests passed; rag.ts 100%, rag-index.ts 99.32% lines / 90.62% branch (only line 184, an untouched pre-existing line, uncovered)
  • git diff --check clean

Safety

  • Fail-safe behavior is unchanged (every function still degrades to null/0/empty on error) — only the log level/shape and one new counter changed. No secrets, no src/** behavior change beyond observability.

Closes #3894

embed/upsert/delete/chunk-read failures in rag.ts, and index/reindex
failures in rag-index.ts, logged at console.log with no level field --
invisible to the Sentry forwarder, unlike the already-fixed sibling
catch in the same file. rag.ts stays import-free (a deliberate
self-contained-port invariant); rag-index.ts additionally gains a
gittensory_rag_pipeline_errors_total counter since it already imports
other dependencies.

Closes #3894
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 7, 2026
@loopover-orb

loopover-orb Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Warning

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

⏸️ Gittensory review result - manual review recommended

Review updated: 2026-07-07 06:57:34 UTC

5 files · 1 AI reviewer · no blockers · readiness 100/100 · CI green · clean

⏸️ Suggested Action - Manual Review

  • Touches a guarded path — held for manual review: This PR changes guardrail-protected path(s): src/selfhost/metrics.ts (matched src/selfhost/**).

Review summary
This PR upgrades six previously-invisible console.log error logs across src/review/rag.ts and src/review/rag-index.ts to console.error with a level:"error" field, and adds a new gittensory_rag_pipeline_errors_total counter for the two rag-index.ts catches. The rag.ts changes reuse the existing review_context_fetch_failed/contextType:"rag" umbrella from the already-fixed sibling catch, and rag-index.ts's new incr() import and counter registration in metrics.ts are wired correctly and exercised by new Proxy-based tests that force the outer catch via a throwing VECTORIZE binding access. All changes are additive/logging-only with no behavior change to the fail-safe return values, and CI is green.

Nits — 7 non-blocking
  • src/review/rag-index.ts:322 and :387 include both `event` and `ev` fields set to the identical string (e.g. "rag_index_repo_error") — redundant; consider dropping one to avoid confusion about which field Sentry search should key on.
  • The new Proxy-based tests in rag-index.test.ts assert the outer-catch path is reached via a thrown VECTORIZE getter, which is a reasonable but somewhat contrived way to simulate a Workers binding failure — worth a brief comment noting this mirrors a real observed failure mode rather than being purely synthetic (partially already documented).
  • src/review/rag.ts is now ~578 lines per the size-smell note; not a blocker for this diff since the change is purely swapping log calls, but worth keeping in mind for a future split.
  • Consider extracting the shared `event: "review_context_fetch_failed", contextType: "rag"` object literal in rag.ts into a small local helper/constant to reduce duplication across the four catch blocks, matching the module's stated no-imports constraint (still zero external imports).
  • In rag-index.ts, drop the duplicate `ev` key from the two new console.error calls since it duplicates `event`.
  • PR author also opened the linked issue — Link an issue that was opened by a different contributor, or provide a rationale for why this self-authored issue represents genuine discovery work.
  • Touches a guarded path — held for manual review — A maintainer must review and merge this change.
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ✅ Linked #3894
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 51 registered-repo PR(s), 43 merged, 343 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 51 PR(s), 343 issue(s).
Gate result ⚠️ Not blocking Advisory; not blocking this PR.
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: not available
  • Official Gittensor activity: 51 PR(s), 343 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@codecov

codecov Bot commented Jul 7, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.59%. Comparing base (685a39e) to head (4132de3).
⚠️ Report is 3 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3917      +/-   ##
==========================================
+ Coverage   93.58%   93.59%   +0.01%     
==========================================
  Files         358      358              
  Lines       34342    34344       +2     
  Branches    12570    12570              
==========================================
+ Hits        32138    32144       +6     
+ Misses       1580     1576       -4     
  Partials      624      624              
Files with missing lines Coverage Δ
src/review/rag-index.ts 93.46% <100.00%> (+2.73%) ⬆️
src/review/rag.ts 100.00% <100.00%> (ø)
src/selfhost/metrics.ts 100.00% <ø> (ø)
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@loopover-orb loopover-orb Bot added the manual-review Gittensor contributor context label Jul 7, 2026
@JSONbored
JSONbored merged commit fa10d7c into main Jul 7, 2026
11 checks passed
@JSONbored
JSONbored deleted the fix/rag-pipeline-error-observability branch July 7, 2026 07:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. manual-review Gittensor contributor context

Development

Successfully merging this pull request may close these issues.

fix(observability): route RAG pipeline errors through console.error so Sentry sees them

1 participant