You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
⚠️ Definition of Done: this issue must be completed in full, in a single PR. Do not split this
work across multiple PRs, and do not defer any Deliverable below to a follow-up issue. A PR that
satisfies only some of the Deliverables, stubs a required test, or leaves a checkbox
partially-done does NOT resolve this issue and will be closed.
Context
generateSignalSnapshots in src/queue/signal-snapshot.ts (~line 65) iterates repos with a plain for (const repo of repositories) loop and no per-repo error isolation. If any repo's DB
reads/writes throw partway through (the Promise.all batch of listIssueSignalSample / listOpenPullRequests / etc., or the later persistSignalSnapshot call, has no .catch), every
subsequent repo in that same invocation's batch silently never gets its snapshot generated, and
the caller just sees one rejected promise for the whole run — there's no way to tell which repo
failed or that the rest were skipped as a side effect.
This is reachable with a multi-repo batch via /v1/internal/jobs/generate-signal-snapshots/run
(src/api/routes.ts, repoFullName is optional there) and via the requestedBy === "test" bypass
in job-dispatch.ts's "generate-signal-snapshots" case. In normal production the sibling fanOutRepoSignalSnapshotJobs fans this out into one isolated queue job per repo, so this
multi-repo path is rarely exercised with more than one repo — but it's a real, reachable gap. test/unit/queue-trends.test.ts only exercises single-repo calls, no multi-repo/partial-failure
scenario.
The precedent for the correct fix already exists in the same file family: src/queue/job-dispatch.ts's "backfill-registered-repos" case (#8355) explicitly switched from Promise.all to Promise.allSettled with per-item failure tracking, for the identical reason —
one repo's failure must never silently block or skip its siblings in the same fan-out.
Requirements
generateSignalSnapshots must isolate per-repo failures: one repo's error (from either the Promise.all data-gathering step or the persistSignalSnapshot write) must not prevent any
other repo in the same invocation from being processed.
Failures must be surfaced, not silently swallowed — collect the failing repo full names and
throw a single aggregate error listing them (matching fix(queue): backfill-registered-repos cron fan-out has no per-repo isolation, risking duplicate dispatch on partial failure #8355's pattern: attempt every repo
exactly once regardless of an earlier one's outcome, then report which ones failed) so the
invocation is still marked failed for observability, and log a structured error per failed repo
(e.g. console.error(JSON.stringify({ level: "error", event: "generate_signal_snapshots_repo_failed", repoFullName, reason }))).
Do not change fanOutRepoSignalSnapshotJobs or the per-repo queue-job fan-out path — this issue
is only about isolating failures within a single generateSignalSnapshots invocation's own
multi-repo loop.
A new test in test/unit/queue-trends.test.ts (or a new adjacent test file) covers a
multi-repo call where one repo's data-gathering or persistence throws, asserting: (a) the
other repo(s) in the same batch still get their snapshot generated/persisted, and (b) the
function still surfaces an error identifying the failed repo.
Both deliverables are required in this single PR.
Test Coverage Requirements
This repo enforces 99%+ Codecov patch coverage, branch-counted, on every changed line/branch in src/**. The new isolation logic and its failure/success branches must be covered by the new test
above.
Expected Outcome
A multi-repo generateSignalSnapshots invocation where one repo's data-gathering or persistence
throws no longer silently skips every subsequent repo in the batch — each repo is attempted
independently, successes persist regardless of a sibling's failure, and failures are identifiable
in the surfaced error.
Links & Resources
src/queue/signal-snapshot.ts (function to fix, ~line 65)
Context
generateSignalSnapshotsinsrc/queue/signal-snapshot.ts(~line 65) iterates repos with a plainfor (const repo of repositories)loop and no per-repo error isolation. If any repo's DBreads/writes throw partway through (the
Promise.allbatch oflistIssueSignalSample/listOpenPullRequests/ etc., or the laterpersistSignalSnapshotcall, has no.catch), everysubsequent repo in that same invocation's batch silently never gets its snapshot generated, and
the caller just sees one rejected promise for the whole run — there's no way to tell which repo
failed or that the rest were skipped as a side effect.
This is reachable with a multi-repo batch via
/v1/internal/jobs/generate-signal-snapshots/run(
src/api/routes.ts,repoFullNameis optional there) and via therequestedBy === "test"bypassin
job-dispatch.ts's"generate-signal-snapshots"case. In normal production the siblingfanOutRepoSignalSnapshotJobsfans this out into one isolated queue job per repo, so thismulti-repo path is rarely exercised with more than one repo — but it's a real, reachable gap.
test/unit/queue-trends.test.tsonly exercises single-repo calls, no multi-repo/partial-failurescenario.
The precedent for the correct fix already exists in the same file family:
src/queue/job-dispatch.ts's"backfill-registered-repos"case (#8355) explicitly switched fromPromise.alltoPromise.allSettledwith per-item failure tracking, for the identical reason —one repo's failure must never silently block or skip its siblings in the same fan-out.
Requirements
generateSignalSnapshotsmust isolate per-repo failures: one repo's error (from either thePromise.alldata-gathering step or thepersistSignalSnapshotwrite) must not prevent anyother repo in the same invocation from being processed.
throw a single aggregate error listing them (matching fix(queue): backfill-registered-repos cron fan-out has no per-repo isolation, risking duplicate dispatch on partial failure #8355's pattern: attempt every repo
exactly once regardless of an earlier one's outcome, then report which ones failed) so the
invocation is still marked failed for observability, and log a structured error per failed repo
(e.g.
console.error(JSON.stringify({ level: "error", event: "generate_signal_snapshots_repo_failed", repoFullName, reason }))).fanOutRepoSignalSnapshotJobsor the per-repo queue-job fan-out path — this issueis only about isolating failures within a single
generateSignalSnapshotsinvocation's ownmulti-repo loop.
Deliverables
generateSignalSnapshotsinsrc/queue/signal-snapshot.tsisolates per-repo failures usingthe same
Promise.allSettled+ collected-failures + aggregate-throw pattern asjob-dispatch.ts's"backfill-registered-repos"case (fix(queue): backfill-registered-repos cron fan-out has no per-repo isolation, risking duplicate dispatch on partial failure #8355).test/unit/queue-trends.test.ts(or a new adjacent test file) covers amulti-repo call where one repo's data-gathering or persistence throws, asserting: (a) the
other repo(s) in the same batch still get their snapshot generated/persisted, and (b) the
function still surfaces an error identifying the failed repo.
Both deliverables are required in this single PR.
Test Coverage Requirements
This repo enforces 99%+ Codecov patch coverage, branch-counted, on every changed line/branch in
src/**. The new isolation logic and its failure/success branches must be covered by the new testabove.
Expected Outcome
A multi-repo
generateSignalSnapshotsinvocation where one repo's data-gathering or persistencethrows no longer silently skips every subsequent repo in the batch — each repo is attempted
independently, successes persist regardless of a sibling's failure, and failures are identifiable
in the surfaced error.
Links & Resources
src/queue/signal-snapshot.ts(function to fix, ~line 65)src/queue/job-dispatch.ts(precedent —"backfill-registered-repos"case, fix(queue): backfill-registered-repos cron fan-out has no per-repo isolation, risking duplicate dispatch on partial failure #8355)test/unit/queue-trends.test.ts(existing single-repo test file to extend)