Skip to content

feat(selfhost): scheduled + on-demand refresh for repo-doc generation - #3202

Merged
loopover-orb[bot] merged 2 commits into
mainfrom
feat/repo-doc-scheduled-refresh
Jul 4, 2026
Merged

feat(selfhost): scheduled + on-demand refresh for repo-doc generation#3202
loopover-orb[bot] merged 2 commits into
mainfrom
feat/repo-doc-scheduled-refresh

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

Closes out the #2993 repo-doc generation roadmap by adding a scheduled refresh sweep and an on-demand manual trigger, both routed through one shared runner so eligibility and diffing can never diverge between the two paths.

  • Scheduled sweep: a new daily job (repo-doc-refresh-sweep, enqueued at hour 9 UTC alongside the other self-host cron jobs) fans out to every repo with repoDocGeneration.enabled, but only enqueues a per-repo refresh if that repo's own repoDocGeneration.refreshIntervalDays (default 7 / weekly, operator-configurable per repo via .gittensory.yml) has actually elapsed since the last attempt.
  • Manual trigger: a new MCP tool, gittensory_refresh_repo_docs, lets a maintainer force an immediate refresh for one repo without waiting for the interval, using the same requireRepoManageAccess gate as other repo-management MCP tools.
  • Shared runner (src/github/repo-doc-refresh-runner.ts): both paths call performRepoDocRefresh, which resolves the repo's action mode the same way the executor does, delegates to the existing openRepoDocPullRequest (unchanged -- it already owns enable/scope/eligibility/diffing/delivery), and records a "last attempted" marker afterward regardless of outcome. A manual trigger resets the scheduled clock too, so the sweep doesn't immediately re-check a repo an operator just refreshed by hand.
  • No new DB layer: "last attempted" tracking reuses the existing generic signal-snapshot table (a new repo-doc-refresh-attempt signal type) instead of a migration, matching feat(config): add .gittensory.yml surface to enable/scope repo-doc generation #3002's own manifest-only precedent for this feature area.
  • The no-meaningful-change short-circuit and the enable/scope/exclusion surface are both inherited unchanged from openRepoDocPullRequest -- there is no separate eligibility code path for the scheduled vs. manual trigger.

Closes #3003. Part of #2993.

Scope

  • Change is narrowly scoped to this issue
  • No changes to site/, CNAME, or **/lovable/**
  • No changelog edits
  • No secrets, wallets, hotkeys, trust scores, or reward values added anywhere

Validation

  • npm run typecheck
  • Targeted test run: npx vitest run test/unit/repo-doc-refresh-schedule.test.ts test/unit/repo-doc-refresh-runner.test.ts test/unit/repo-doc-refresh-sweep.test.ts test/unit/mcp-refresh-repo-docs.test.ts test/unit/focus-manifest.test.ts test/unit/gittensory-focus-manifest.test.ts test/unit/repo-doc-pr.test.ts test/unit/no-direct-octokit.test.ts -- 325/325 passing
  • npm run test:changed (diff-aware full sweep against origin/main, incl. test/unit/queue.test.ts) -- 5736/5743 passing (7 pre-existing skips, 0 failures)
  • Verified 100% branch coverage on every new/changed line via coverage/lcov.info BRDA parsing (both the new dedicated test files and the new conditionals added to the large shared src/queue/processors.ts and src/mcp/server.ts)
  • No generated-artifact regeneration needed: new MCP tool has no static tool-name manifest to regenerate (confirmed via grep); no wrangler binding/var or Drizzle schema changes

Safety

  • No secrets/wallets/hotkeys/trust-score/reward terms anywhere
  • MCP tool gated behind requireRepoManageAccess, matching the access-control pattern of other repo-management tools; static "mcp" identity still respects MCP_ACTUATION_REPO_ALLOWLIST
  • Scheduled sweep only ever opens a pull request (never a direct commit), same as the existing on-demand path

Adds a daily fan-out sweep (queued behind each repo's own
repoDocGeneration.refreshIntervalDays, default weekly) plus a manual MCP
trigger, both routed through one shared runner so eligibility and
diff-aware no-change short-circuiting can never diverge between the two
paths. Last-attempt tracking reuses the existing signal-snapshot table
instead of a new migration.

Closes #3003. Part of #2993.
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@JSONbored JSONbored self-assigned this Jul 4, 2026
@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 4, 2026
@loopover-orb

loopover-orb Bot commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Tip

🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩

✅ Gittensory review result - approve/merge recommended

Review updated: 2026-07-04 20:23:18 UTC

13 files · 1 AI reviewer · no blockers · readiness 100/100 · CI green · clean

✅ Suggested Action - Approve/Merge

  • safe to merge

Review summary
The change adds a shared repo-doc refresh runner, a daily self-host sweep, and an MCP manual trigger, with the actual PR creation still delegated to the existing repo-doc PR path. The core wiring is coherent: both scheduled and manual paths call `performRepoDocRefresh`, and the new manifest field is parsed/defaulted and covered by focused unit tests. I do not see a reachable correctness blocker in the provided diff, but the sweep implementation leaves a couple of maintainability and operational edges worth tightening.

Nits — 6 non-blocking
  • nit: `src/queue/processors.ts:1727` performs one `getLastRepoDocRefreshAttemptedAt` lookup per installed repo during fan-out, so a large self-host install turns the daily sweep into an avoidable N+1 DB pattern; consider a bulk snapshot lookup or bounded batching even though the cadence is only daily.
  • nit: `src/github/repo-doc-refresh-runner.ts:21` relies on `listSignalSnapshots` returning the newest snapshot first; add an assertion-style test with two attempts or use an explicit latest helper so the rate limiter cannot silently read an older marker if repository ordering changes.
  • nit: `src/index.ts:189` enqueues the repo-doc sweep outside the existing GitHub maintenance headroom gate used for other self-host background work; confirm this should still run when `sweepThrottledUntil` is set, or gate it the same way to avoid adding background GitHub traffic during low-budget windows.
  • nit: `src/signals/focus-manifest.ts:1007` adds a new `.gittensory.yml` field but the visible diff does not include docs/example schema updates; confirm the config contract is documented wherever operators learn valid `repoDocGeneration` keys.
  • In `src/queue/processors.ts:1727`, replace the per-repo marker read with a bulk load keyed by repo full name, or batch the reads with a small concurrency cap.
  • PR author also opened the linked issue — Link an issue that was opened by a different contributor, or provide a rationale for why this self-authored issue represents genuine discovery work.
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ✅ Linked #3003
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 60 registered-repo PR(s), 50 merged, 438 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 60 PR(s), 438 issue(s).
Gate result ✅ Passing No configured blocker found.
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository registration is not available in the local Gittensory cache.
  • Public profile languages: Python, TypeScript, JavaScript, Ruby, Go, Kotlin, MDX, Shell
  • Official Gittensor activity: 60 PR(s), 438 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • No action.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@codecov

codecov Bot commented Jul 4, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.74%. Comparing base (c283373) to head (280a4f1).
⚠️ Report is 7 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #3202   +/-   ##
=======================================
  Coverage   94.73%   94.74%           
=======================================
  Files         270      272    +2     
  Lines       29718    29768   +50     
  Branches    10838    10850   +12     
=======================================
+ Hits        28154    28204   +50     
  Misses        918      918           
  Partials      646      646           
Files with missing lines Coverage Δ
src/github/repo-doc-refresh-runner.ts 100.00% <100.00%> (ø)
src/index.ts 94.73% <100.00%> (+0.14%) ⬆️
src/mcp/server.ts 95.87% <100.00%> (+0.06%) ⬆️
src/queue/processors.ts 92.92% <100.00%> (+0.05%) ⬆️
src/review/repo-doc-refresh-schedule.ts 100.00% <100.00%> (ø)
src/signals/focus-manifest.ts 98.56% <100.00%> (+<0.01%) ⬆️
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@loopover-orb loopover-orb Bot added gittensor manual-review Gittensor contributor context labels Jul 4, 2026
The new hour===9 branch in enqueueScheduledJobs had no scheduled-worker
test exercising either side, dropping patch coverage below the gate.

@loopover-orb loopover-orb Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gittensory approves — the gate is satisfied and CI is green.

@loopover-orb
loopover-orb Bot merged commit 568d6b3 into main Jul 4, 2026
10 checks passed
@loopover-orb
loopover-orb Bot deleted the feat/repo-doc-scheduled-refresh branch July 4, 2026 20:26
loopover-orb Bot pushed a commit that referenced this pull request Jul 4, 2026
…p fan-out (#3219)

fanOutRepoDocRefreshSweepJobs called getLastRepoDocRefreshAttemptedAt
once per installed repo, an N+1 query pattern flagged in review on
PR #3202 after it merged (the finding was advisory and did not block
the owner-PR auto-merge, so it landed as follow-up work). Adds
listLatestSignalSnapshotsForTargets, a bulk latest-per-target-key
query, and a getLastRepoDocRefreshAttemptedAtBulk wrapper so the
daily sweep does one round trip across every candidate repo instead
of scaling linearly with the installed-repo count.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. manual-review Gittensor contributor context

Development

Successfully merging this pull request may close these issues.

feat(selfhost): scheduled + on-demand refresh for repo-doc generation

1 participant