fix(backfill): hydrate merged PR changed files in segment backfill - #353
Merged
JSONbored merged 2 commits intoJun 4, 2026
Merged
Conversation
JSONbored
requested changes
Jun 3, 2026
JSONbored
left a comment
Owner
There was a problem hiding this comment.
@galuis116 the backfill problem statement is useful, but the PR needs hygiene fixes before merge.
A few notes:
- Hydrating changed files for recent merged PRs is a real data-quality improvement.
- Preserving non-empty existing file lists on re-upsert is the right invariant.
- The current title/body do not meet this repo's PR requirements.
Required changes:
- Retitle to
fix(backfill): hydrate merged PR changed files in segment backfill. - Update the PR body to the current template with Summary, Scope, Validation, Safety, and Notes.
- Include exact validation commands and security/privacy notes for the backfill data surface.
Validation expected:
- Keep the current green CI.
- Include the targeted backfill tests plus
git diff --checkin the PR body.
The recent-merged segment backfill stored changedFiles: [] and the upsert clobbered any existing files. Fetch files in the segment path (like the monolithic path) and preserve a non-empty file list on re-upsert.
galuis116
force-pushed
the
fix/recent-merged-changed-files-backfill
branch
from
June 3, 2026 22:10
1a9fe17 to
5cb82b2
Compare
Contributor
Author
|
Thanks for the review — addressed the hygiene items (no code changes from the reviewed diff):
CI is green and re-running on the amended commit. Ready for another look. |
JSONbored
approved these changes
Jun 4, 2026
Owner
There was a problem hiding this comment.
@galuis116 this is in good shape from this pass.
A few notes:
- The segment backfill now hydrates changed-file paths instead of storing empty arrays.
- The coalesce guard avoids clobbering existing file data with empty incoming lists.
- The validation evidence is specific and covers the affected backfill, queue, data-quality, API, typecheck, and diff-check surfaces.
No code changes are requested from this review pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #352.
Summary
The production segment-pipeline backfill stored
changedFiles: []for every recent-merged PR (backfillRecentMergedSegmenthard-coded[]), while only the legacy monolithic path fetched the real list.upsertRecentMergedPullRequestalso unconditionally overwrotechangedFilesJson, clobbering any populated list, and no detail-sync hydrates merged-PR files. Sorecent_merged_pull_requests.changedFileswas always empty in production, blinding the outcome path/size/test-evidence dimensions and collision file-overlap (both readmergedDetail.changedFiles).Scope
src/github/backfill.ts— fetch each merged PR's changed files in the recent-merged segment viafetchPullRequestFiles(mirrors the monolithic path; reuses the existingmapWithConcurrency, REST -> GraphQL fallback, and rate-limit handling).src/db/repositories.ts—upsertRecentMergedPullRequestkeeps the existingchangedFilesJsonon conflict when the incoming list is empty (no clobber).test/unit/backfill.test.ts— segment-hydration + coalesce tests.No schema, API, or public-surface changes.
Validation
Branch coverage stays >= 97% (both coalesce ternary branches and the segment path are covered by the new tests). CI
validateis green.Safety
/pulls/{number}/filesGitHub endpoint the monolithic backfill already uses — no secrets, no diffs, no file content, and no new external surface or privileges.[]on failure (a warning is recorded, no throw).Notes
mapWithConcurrency(..., 8)).buildRepoOutcomePatternsignores therecent_merged_pull_requeststable — merge-rate analysis runs almost without merged PRs, falsely flagging healthy repos as "high closure risk" #312 / fix(signals): include recent_merged_pull_requests in outcome pattern #316 (which fix the analysis ignoring recent-merged) — this fixes the backfill data those consumers read.