Skip to content

feat(miner): retry a transient 5xx in the discovery fanout - #5425

Merged
JSONbored merged 1 commit into
JSONbored:mainfrom
real-venus:feat/gittensory-miner-fanout-retry-v1
Jul 12, 2026
Merged

feat(miner): retry a transient 5xx in the discovery fanout#5425
JSONbored merged 1 commit into
JSONbored:mainfrom
real-venus:feat/gittensory-miner-fanout-retry-v1

Conversation

@real-venus

Copy link
Copy Markdown
Contributor

Summary

A network blip while fetching a repo's policy docs or target issues was silently swallowed into a warning, dropping that repo's results for the entire discover run — with no retry.

This wraps the discovery fanout's single shared fetch helper (githubGetJson) with the same fetchWithRetry discipline the CI and gate-verdict pollers already use (#4829): a transient 5xx response is retried with bounded exponential backoff before the repo is dropped, so a brief blip recovers instead of losing results. Since every fanout fetch (policy docs, target issues, search issues) goes through that one helper, the retry covers all three paths with a single wiring point.

  • A 4xx/404 is returned immediately (no retry).
  • A thrown network error still propagates to each caller's try/catch, unchanged.
  • sleepFn is threaded through normalizeOptions so tests inject an instant retry; a persistent 5xx still warns after the retries are exhausted (existing behaviour preserved).

Scope

Validation

  • npm run typecheck
  • npm run test:coverage (full unsharded suite)
  • test/unit/miner-opportunity-fanout.test.ts: a transient 5xx is retried and the target's issues are kept (not dropped, no warning) — the Add retry on transient fetch failures in the discovery fanout #4830 acceptance; the existing persistent-5xx tests still warn (after instant retries) unchanged
  • Every added line covered; the persistent-5xx tests inject an instant sleepFn so they stay fast

Safety

  • Read-only: wraps existing GET calls; no writes, no new endpoints; rate-limit telemetry recording is unchanged
  • Bounded attempts + capped backoff; a persistent failure still degrades to a warning as before
  • No secrets, tokens, wallets, trust scores, or reward values in code, tests, or this description

Closes #4830

A network blip while fetching a repo's policy docs or target issues was
silently swallowed into a warning, dropping that repo's results for the
entire discover run with no retry. Wrap the fanout's single shared fetch
helper (githubGetJson) with the same fetchWithRetry discipline the CI and
gate-verdict pollers use (JSONbored#4829): a transient 5xx response is retried with
bounded exponential backoff before the repo is dropped, so a brief blip
recovers instead of losing results.

A 4xx/404 is returned immediately and a thrown network error still
propagates to each caller's try/catch (unchanged). sleepFn is threaded
through normalizeOptions so tests inject an instant retry; a persistent
5xx still warns after the retries are exhausted.

Closes JSONbored#4830
@real-venus
real-venus requested a review from JSONbored as a code owner July 12, 2026 18:51
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.45%. Comparing base (59ba5d1) to head (fd61b79).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #5425   +/-   ##
=======================================
  Coverage   94.45%   94.45%           
=======================================
  Files         554      554           
  Lines       44371    44371           
  Branches    14659    14659           
=======================================
  Hits        41912    41912           
  Misses       1784     1784           
  Partials      675      675           
Flag Coverage Δ
shard-1 43.92% <0.00%> (-0.25%) ⬇️
shard-2 34.78% <75.00%> (+0.04%) ⬆️
shard-3 31.61% <0.00%> (-0.31%) ⬇️
shard-4 31.38% <75.00%> (+0.08%) ⬆️
shard-5 32.93% <100.00%> (-0.40%) ⬇️
shard-6 43.89% <0.00%> (+0.57%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...ackages/gittensory-miner/lib/opportunity-fanout.js 89.93% <100.00%> (ø)
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@loopover-orb loopover-orb Bot added the gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. label Jul 12, 2026
@loopover-orb

loopover-orb Bot commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Tip

🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩

✅ Gittensory review result - approve/merge recommended

Review updated: 2026-07-12 18:58:15 UTC

3 files · 1 AI reviewer · no blockers · readiness 82/100 · CI green · clean

✅ Suggested Action - Approve/Merge

  • safe to merge

Review summary
The change correctly routes all discovery fanout GitHub GETs through the existing retry helper and threads the normalized `sleepFn` so tests can avoid real backoff. The visible production path preserves the existing per-call warning behavior after persistent 5xx responses, and the new transient-503 test exercises the real `fetchCandidateIssuesWithSummary` path rather than fabricating an impossible payload. I do not see a reachable correctness defect in the provided diff.

Nits — 4 non-blocking
  • nit: packages/gittensory-miner/lib/opportunity-fanout.d.ts exposes `sleepFn` as part of the public TypeScript API even though the implementation comment frames it as a test hook, so either document that it is intentionally supported or keep the test-only surface internal if that is not meant for callers.
  • nit: test/unit/miner-opportunity-fanout.test.ts only proves a transient 5xx recovery for the repo issues endpoint, while the PR description explicitly claims the same behavior for policy-doc and search fetches through the shared helper.
  • packages/gittensory-miner/lib/opportunity-fanout.d.ts: add a short JSDoc note for `sleepFn` if you intend consumers to rely on it, or explain why exposing the retry sleeper in the declaration file is the right public contract.
  • test/unit/miner-opportunity-fanout.test.ts: add one narrow transient-5xx assertion for `searchCandidateIssuesWithSummary` or a policy document fetch to pin the shared-helper claim across another caller without relying only on inference.
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ✅ Linked #4830
Related work ⚠️ 3 scoped overlaps Top overlaps are listed below; lower-confidence bulk is hidden.
Change scope ❌ 8/20 High review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 103 registered-repo PR(s), 54 merged, 12 issue(s).
Contributor context ✅ Confirmed Gittensor contributor real-venus; Gittensor profile; 103 PR(s), 12 issue(s).
Gate result ✅ Passing No configured blocker found.
Improvement ✅ Minor risk: clean · value: minor — Code changes are accompanied by test evidence.
Review context
  • Author: real-venus
  • Role context: outside_contributor
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: not available
  • Official Gittensor activity: 103 PR(s), 12 issue(s).
  • Related work: Titles/paths share 3 meaningful terms. (issue #4767, issue #4830)
  • Related work: Titles/paths share 3 meaningful terms. (issue #4801, issue #4830)
  • Related work: Titles/paths share 3 meaningful terms. (issue #4830, issue #4803)
Contributor next steps
  • Review top overlaps.
  • Add a concise scope and risk note.
  • Check active issues and PRs before submitting.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
[BETA] Chat with Gittensory

Ask Gittensory a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @gittensory ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @gittensory chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @gittensory mention with a real question is routed to the closest matching read-only command automatically -- no exact syntax required.

Full command reference: https://gittensory.aethereal.dev/docs/gittensory-commands

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@loopover-orb loopover-orb Bot added the manual-review Gittensor contributor context label Jul 12, 2026
@JSONbored
JSONbored merged commit 6b703a4 into JSONbored:main Jul 12, 2026
16 checks passed
@loopover-orb loopover-orb Bot added gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. and removed gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. labels Jul 12, 2026
@JSONbored JSONbored added gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. and removed gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. labels Jul 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. manual-review Gittensor contributor context

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add retry on transient fetch failures in the discovery fanout

2 participants