Skip to content

contribution-profile-extract.js's getJson has no retry-on-5xx, unlike opportunity-fanout.js's sibling fetcher, and its degraded result gets cached #7090

Description

@JSONbored

Context

packages/loopover-miner/lib/opportunity-fanout.js's githubGetJson (lines 123-138) wraps every GitHub API GET
in fetchWithRetry (http-retry.js) specifically so "a transient SERVER error (a 5xx RESPONSE) or a transient
GitHub RATE-LIMIT response... is retried a bounded number of times" rather than being treated as a permanent
signal on the first blip.

packages/loopover-miner/lib/contribution-profile-extract.js's getJson (lines 75-87) is the same shape of
request — a small, bounded, AbortSignal.timeout-guarded GitHub API GET, used by extractContributionProfile
to fetch a repo's labels and CONTRIBUTING.md/AI-USAGE.md-style docs — but it calls fetchImpl directly with
no retry at all: a single 5xx or transient network hiccup makes getJson return null, which the extraction
logic (per this file's own header comment, "any fetch/parse failure degrades a signal to absent/unknown
rather than erroring") folds into a low-confidence ContributionProfile.

This matters more than a typical "missing retry" gap because the degraded result is not just returned once and
discarded — discover-cli.js's resolveContributionProfilesForDiscover (lines 169-187) immediately
cache.put(profile, ctx.nowMs)s whatever extract() returns into contribution-profile-cache.js's persistent
store, keyed by repoFullName with its own freshness TTL (CONTRIBUTION_PROFILE_CACHE_TTL_MS). A transient
GitHub 5xx during one discover run therefore doesn't just degrade that one run's ranking — it gets PERSISTED as
the repo's cached eligibility signal and continues silently disabling (or weakening) contribution-profile- filter.js's label-based eligibility filtering for that repo until the cache entry naturally expires, even
though the repo's real label taxonomy was reachable moments before or after the blip.

Requirements

  • getJson (packages/loopover-miner/lib/contribution-profile-extract.js) MUST retry a transient GitHub 5xx or
    rate-limit response (429 / secondary-403) via fetchWithRetry from ./http-retry.js, matching
    opportunity-fanout.js's githubGetJson usage: same retryable-status semantics, same bounded attempts, same
    exponential-backoff-or-Retry-After (whichever is longer) delay.
  • The existing per-attempt request timeout (REQUEST_TIMEOUT_MS via AbortSignal.timeout) MUST be preserved —
    fetchWithRetry's own timeoutMs option already gives each retry attempt a FRESH timeout, so this is a
    straightforward pass-through, not a redesign.
  • getJson MUST remain "never throws" (its documented contract): the function's return type/behavior for a
    non-retryable failure (a plain 4xx, or every retry attempt exhausted) MUST be unchanged — still null, never
    a thrown error.
  • The new retry MUST be injectable/testable without real timers, mirroring every other fetchWithRetry call
    site's own sleepFn seam.

Deliverables

  • getJson in packages/loopover-miner/lib/contribution-profile-extract.js retries transient 5xx/rate-limit
    responses via fetchWithRetry instead of returning null on the first such response.
  • A test asserting a label/doc fetch that returns a transient 5xx on its first attempt but succeeds on a
    later attempt (within the retry budget) produces the SAME ContributionProfile confidence/signals as an
    immediate success, not a degraded absent/unknown result.
  • A test asserting a fetch that fails on every retry attempt still degrades to absent/unknown exactly as
    today (never throws), preserving the existing fail-open contract once retries are genuinely exhausted.

Test Coverage Requirements

99%+ Codecov patch coverage on every changed line and branch in contribution-profile-extract.js, including the
retry-succeeds and retry-exhausted branches for both the label-list fetch and the doc-content fetch call sites,
plus the two regression tests above using an injected sleepFn (no real timers).

Expected Outcome

A transient GitHub 5xx/rate-limit blip during discover's contribution-profile extraction no longer gets
permanently cached as a degraded, low-confidence eligibility signal for that repo — contribution-profile- extract.js rides out the same class of transient failure opportunity-fanout.js's sibling fetcher already
does, before falling back to its existing, unchanged absent/unknown degradation once retries are genuinely
exhausted.

Links & Resources

  • packages/loopover-miner/lib/contribution-profile-extract.js:75-87 (getJson) — the code to change.
  • packages/loopover-miner/lib/opportunity-fanout.js:123-138 (githubGetJson) — the sibling fetcher already
    using fetchWithRetry for the identical class of request.
  • packages/loopover-miner/lib/http-retry.js (fetchWithRetry) — the shared retry helper to adopt.
  • packages/loopover-miner/lib/contribution-profile-cache.js, discover-cli.js's
    resolveContributionProfilesForDiscover — where a degraded profile gets persisted and why the caching makes
    this worth fixing beyond a single run.

Metadata

Metadata

Assignees

No one assigned

    Labels

    gittensor:bugGittensor-scored bug fix — scores a 0.05x multiplier.help wantedExtra attention is needed

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions