Skip to content

feat(miner-portfolio): PortfolioQueueManager stateful wrapper + SQLite persistence #4285

Description

@JSONbored

This is narrower than it sounds: both halves implied by "PortfolioQueueManager" already exist and are shipped — they're just not composed together yet.

  • The pure primitive (feat(miner-portfolio): define PortfolioQueueManager types + pure bucketing primitives #2326, closed) lives at packages/gittensory-engine/src/portfolio/queue.ts: PortfolioQueueItem/PortfolioQueue/PortfolioCaps types plus enqueueItem, dequeueItem, and nextEligibleItems(queue, caps) (queue.ts:148) — a pure, in-memory selector that respects a globalWipCap/perRepoWipCap (queue.ts:24-27) and diversifies across repos so it never returns two consecutive items from the same bucket while another bucket still has eligible work.
  • The SQLite persistence (feat(miner-foundation): local portfolio/queue store for gittensory-miner #2292, closed) lives at packages/gittensory-miner/lib/portfolio-queue.js: initPortfolioQueueStore (portfolio-queue.js:79) with enqueue/dequeueNext/listQueue/markDone, already wired to the CLI (gittensory-miner queue {list,next,done} via packages/gittensory-miner/lib/portfolio-queue-cli.js, dispatched at packages/gittensory-miner/bin/gittensory-miner.js:41-43).

The gap: portfolio-queue.js's dequeueNext() (portfolio-queue.js:180) claims a single row via a plain priority DESC, enqueued_at ASC, rowid ASC ordered UPDATE ... RETURNING (portfolio-queue.js:122-128) — it never applies WIP caps or cross-repo diversification. The engine's nextEligibleItems isn't consumed anywhere outside its own test file today (packages/gittensory-engine/test/portfolio-queue.test.ts is the only caller in the repo — confirmed by grepping for globalWipCap/perRepoWipCap across packages/ and src/). There is also no PortfolioQueueManager symbol anywhere in the codebase (zero hits searching packages/ and src/). So the real remaining work is the stateful wrapper named in the title: compose the persisted SQLite store with the pure engine selector so claiming "what's next" respects caps and diversification instead of a single naive row-claim.

This is a pure-plus-persistence composition issue, not an enforcement issue — it decides what the miner works on next out of its own backlog, not whether a write action is allowed to proceed. The actual safety/enforcement chokepoint (composing rate-limit, budget caps, and the non-convergence detector into one allow/deny decision) is separate, maintainer-owned work tracked in #2340 ("wire the fail-closed Governor chokepoint before every write action," milestone 13) and is explicitly out of scope here.

Deliverables

  • New packages/gittensory-miner/lib/portfolio-queue-manager.js (+ .d.ts) exposing a manager that composes initPortfolioQueueStore (portfolio-queue.js:79) with the engine's nextEligibleItems/enqueueItem/dequeueItem (@jsonbored/gittensory-engine is already a dependencies entry in packages/gittensory-miner/package.json:37)
  • A caps-aware batch-claim operation: load current queued/in_progress rows from SQLite, project them into the engine's PortfolioQueue/PortfolioQueueItem shape, call nextEligibleItems(queue, caps), then atomically transition the selected rows to in_progress — extend the existing UPDATE ... RETURNING pattern (portfolio-queue.js:122-128) so the batch claim stays race-safe under a concurrent second writer on the same file, exactly like the current single-row dequeueNext
  • Accept PortfolioCaps (globalWipCap/perRepoWipCap) as a plain constructor/function argument — do NOT wire it to .gittensory-miner.yml in this issue; config-parsing is a separate, not-yet-filed concern, so keep scope to the manager itself (mirror feat(miner-foundation): local portfolio/queue store for gittensory-miner #2292's own "priority is a placeholder, not wired to a source here" disclaimer style)
  • Unit tests against an in-memory store (initPortfolioQueueStore(":memory:"), already supported per portfolio-queue.js:81-82): empty queue, a single repo saturated at its per-repo cap, multi-repo diversification, and caps of 0 returning nothing — mirroring the scenarios packages/gittensory-engine/test/portfolio-queue.test.ts already covers for the pure primitive, but exercised through the persisted manager
  • Leave queue next's existing single-row behavior (portfolio-queue-cli.js:159-180) untouched — the caps-aware batch claim is new surface, not a breaking change to the existing CLI command

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    gittensor:featureGittensor-scored feature linked to a feature issue — scores a 0.25x multiplier.gittensor:priorityMaintainer-selected Gittensor priority — scores a 1.5x multiplier.help wantedExtra attention is needed

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions