Skip to content

feat(miner-scale): add fleet run-manifest for multi-repo worktree scheduling - #4402

Merged
JSONbored merged 1 commit into
JSONbored:mainfrom
jeffrey701:feat/fleet-run-manifest
Jul 9, 2026
Merged

feat(miner-scale): add fleet run-manifest for multi-repo worktree scheduling#4402
JSONbored merged 1 commit into
JSONbored:mainfrom
jeffrey701:feat/fleet-run-manifest

Conversation

@jeffrey701

Copy link
Copy Markdown
Contributor

Closes #4299

What

Adds a tolerant parser for the fleet run-manifest — the top-level config a fleet operator authors to run the miner across many repos at once. It declares which repos are in scope for a fleet run and how a finite worktree/concurrency budget is split between them.

This is the operator-side analogue of, and deliberately not the same file as, the maintainer-authored .gittensory-miner.yml goal spec (miner-goal-spec.ts). It follows the exact same tolerant-parser convention already established in this codebase:

  • every field optional, unknown keys ignored;
  • a malformed field degrades to a documented default with a warning instead of throwing;
  • never throws — a fleet run can surface "your run-manifest had problems" without hard-failing.

Schema

  • repos — a list of target repos. Each entry is either a bare "owner/repo" string (uses the default per-repo budget) or a { repoFullName, maxConcurrentWorktrees } mapping. Invalid or duplicate entries are skipped with a warning. repoFullName is a canonical owner/repo, compatible with the opportunity-fanout target normalization. Default: [].
  • repos[].maxConcurrentWorktrees — max concurrent worktrees for that repo. A positive integer (floored; sub-1 falls back to the default). Default: 1.
  • totalConcurrentWorktrees — total concurrent worktrees across the whole fleet. A positive integer. Default: 1.

Byte cap (64 KiB) and repo-count cap (500) match the defensive limits used elsewhere in the engine.

Scope

This module produces only the parsed, typed manifest. Driving the concurrency allocator from it, and reading the same repo list into the cross-repo backlog, are separate concerns tracked by their own issues — both consume this manifest; neither wiring lives here.

Files

  • packages/gittensory-engine/src/fleet-run-manifest.ts — the parser (mirrors miner-goal-spec.ts).
  • packages/gittensory-engine/src/index.ts — barrel re-export of the public API.
  • packages/gittensory-miner/docs/fleet-run-manifest.md — distinction doc vs .gittensory-miner.yml.
  • test/unit/fleet-run-manifest-parser.test.ts — parser tests (100% line and branch coverage of the new file).

Testing

npx vitest run test/unit/fleet-run-manifest-parser.test.ts
npm run typecheck

All 11 tests pass; typecheck clean. The new engine file is at 100% line and branch coverage.

…eduling

Add a tolerant parser for the fleet run-manifest — the operator-side config
that declares which repos a fleet run works across and how a finite
worktree/concurrency budget is split between them. This is the operator
analogue of the maintainer-authored `.gittensory-miner.yml` goal spec and
mirrors its parser convention: every field optional, unknown keys ignored,
malformed input degrades to a documented default with a warning rather than
throwing.

- packages/gittensory-engine/src/fleet-run-manifest.ts: parseFleetRunManifest
  / parseFleetRunManifestContent (JSON or YAML), repo-list normalization
  (bare "owner/repo" string or { repoFullName, maxConcurrentWorktrees }
  mapping), dedupe, per-repo and total concurrency budgets, size/count caps.
- packages/gittensory-engine/src/index.ts: barrel re-export of the API.
- packages/gittensory-miner/docs/fleet-run-manifest.md: distinction doc so the
  fleet run-manifest is not conflated with `.gittensory-miner.yml`.
- test/unit/fleet-run-manifest-parser.test.ts: full coverage of the parser.

Closes JSONbored#4299
@jeffrey701
jeffrey701 requested a review from JSONbored as a code owner July 9, 2026 09:03
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 9, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.98%. Comparing base (64fb1a3) to head (e6ef86c).
⚠️ Report is 27 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4402      +/-   ##
==========================================
+ Coverage   93.96%   93.98%   +0.01%     
==========================================
  Files         401      402       +1     
  Lines       36892    36967      +75     
  Branches    13490    13514      +24     
==========================================
+ Hits        34667    34742      +75     
  Misses       1569     1569              
  Partials      656      656              
Files with missing lines Coverage Δ
...ckages/gittensory-engine/src/fleet-run-manifest.ts 100.00% <100.00%> (ø)
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@loopover-orb loopover-orb Bot added the gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. label Jul 9, 2026
@loopover-orb

loopover-orb Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Tip

🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩

✅ Gittensory review result - approve/merge recommended

Review updated: 2026-07-09 17:23:12 UTC

4 files · 1 AI reviewer · no blockers · readiness 82/100 · CI green · clean

✅ Suggested Action - Approve/Merge

  • safe to merge

Review summary
PR #4402 adds a tolerant YAML/JSON parser for a new 'fleet run-manifest' config, closing linked issue #4299 as required. It closely mirrors the already-established miner-goal-spec.ts convention (typed schema, frozen defaults, per-field tolerant normalizers, byte/count caps, present/warnings result), and the new unit tests exercise the real parsing paths (string vs. object repo entries, dedup, invalid names, non-list/non-numeric fields, the repo cap, byte cap, and JSON/YAML/malformed content) rather than fabricating unreachable states. The change is narrowly scoped to the parser, its barrel export, tests, and docs; wiring into the concurrency allocator and cross-repo backlog is explicitly and correctly deferred to separate issues, consistent with the diff. CI is fully green on this commit.

Nits — 5 non-blocking
  • fleet-run-manifest.ts's normalizeRepoList reuses DEFAULT_FLEET_RUN_MANIFEST.totalConcurrentWorktrees as the fallback for the per-repo maxConcurrentWorktrees default instead of a dedicated constant — works only because both defaults are currently 1, and would silently drift if the fleet-wide default ever changes independently.
  • utf8ByteLength and normalizePositiveInteger in fleet-run-manifest.ts are verbatim duplicates of the same functions in miner-goal-spec.ts; now that a second module needs them, consider extracting a shared internal util per this repo's DRY expectations.
  • parseFleetRunManifestContent checks the byte cap against the untrimmed content but parses the trimmed string — harmless but slightly inconsistent ordering.
  • The external brief's depth-5 nesting flag on normalizeRepoList's loop is inherent to the three-way string/object/invalid branch and mirrors the existing goal-spec parser's shape, not worth restructuring.
  • Introduce a dedicated per-repo default constant (e.g. DEFAULT_PER_REPO_MAX_CONCURRENT_WORKTREES) instead of borrowing totalConcurrentWorktrees's default in normalizeRepoList.
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ✅ Linked #4299
Related work ⚠️ 2 scoped overlaps Top overlaps are listed below; lower-confidence bulk is hidden.
Change scope ❌ 8/20 High review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 96 registered-repo PR(s), 36 merged, 4 issue(s).
Contributor context ✅ Confirmed Gittensor contributor jeffrey701; Gittensor profile; 96 PR(s), 4 issue(s).
Gate result ✅ Passing No configured blocker found.
Linked issue satisfaction

Addressed
The PR adds the fleet-run-manifest schema/parser in gittensory-engine mirroring parseMinerGoalSpecContent's tolerant-default pattern, documents the explicit distinction from .gittensory-miner.yml under packages/gittensory-miner/docs/, and includes unit tests following the referenced goal-spec parser test pattern; wiring to the allocator/portfolio-queue is explicitly and reasonably deferred to sibl

Review context
  • Author: jeffrey701
  • Role context: outside_contributor
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: TypeScript, Clojure, JavaScript
  • Official Gittensor activity: 96 PR(s), 4 issue(s).
  • Related work: Titles/paths share 7 meaningful terms. (issue #4297, issue #4299)
  • Related work: Titles/paths share 6 meaningful terms. (issue #4299, issue #4307)
Contributor next steps
  • Review top overlaps.
  • Add a concise scope and risk note.
  • Check active issues and PRs before submitting.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@JSONbored
JSONbored merged commit 4cf2adb into JSONbored:main Jul 9, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. manual-review Gittensor contributor context

Development

Successfully merging this pull request may close these issues.

feat(miner-scale): add fleet run-manifest for multi-repo worktree scheduling

2 participants