Skip to content

fix(miner): scope portfolio-queue by forge host, not bare repoFullName - #5583

Merged
JSONbored merged 1 commit into
mainfrom
forge-scope-ledger-keys
Jul 13, 2026
Merged

fix(miner): scope portfolio-queue by forge host, not bare repoFullName#5583
JSONbored merged 1 commit into
mainfrom
forge-scope-ledger-keys

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

  • portfolio-queue.js's PRIMARY KEY (repo_full_name, identifier) let two forge hosts (github.com vs. a GitHub Enterprise host, De-hardcode discovery from gittensory's own conventions #4784) serving a same-named owner/repo collide in the portfolio queue.
  • Rebuilds the constraint to PRIMARY KEY (api_base_url, repo_full_name, identifier) as the store's next schema migration (v2 → v3, chained after the existing leased_at migration), backfilling existing rows with the pre-De-hardcode discovery from gittensory's own conventions #4784 implicit default https://api.github.com.
  • Threads an optional apiBaseUrl through the store's API (enqueue/markDone/markFailed/reclaimStuckItem/requeueItem/batchClaim), the admin CLI (--api-base-url on done/release/requeue), and two real call sites that already had a resolved apiBaseUrl in scope but weren't threading it through: enqueueRankedDiscovery (discover-cli.js's actual discovery pipeline) and the stuck-lease/malformed-identifier sweeps in portfolio-queue-expiry.js and loop-cli.js, which now echo each item's own apiBaseUrl back instead of defaulting — a defaulted echo would act on the wrong host's row whenever two hosts share an owner/repo+identifier.
  • markDone now uses UPDATE ... RETURNING (matching this store's other statements), removing a structurally-unreachable defensive branch the old separate-SELECT pattern left behind.
  • Every existing caller is unaffected: apiBaseUrl defaults to https://api.github.com when omitted.
  • claimNextBatch's engine-driven selection (portfolio-queue-manager.js) has no forge dimension in @jsonbored/gittensory-engine's PortfolioQueueItem shape — documented in-code as a known, safe (no-collision, just no-claim) limitation rather than silently patched over; retrofitting the engine primitive is out of this store-level fix's scope.

Second of 5 stores from #5563 (claim-ledger.js landed in #5576; governor-state.js's reputation table, run-state.js, deny-hook-synthesis.js follow in separate PRs).

Test plan

  • npx vitest run test/unit/miner-portfolio-queue*.test.ts test/unit/miner-loop-cli.test.ts test/unit/miner-discover-cli.test.ts test/unit/miner-portfolio-discovery.test.ts test/unit/miner-migrate-cli.test.ts — all passing, including a migration test (pre-existing on-disk file upgrades through both chained migrations and preserves rows) and cross-host collision regression tests
  • Mutation-tested the two apiBaseUrl-echo fixes (portfolio-queue-expiry.js, loop-cli.js): reverted each fix, confirmed the new regression test fails with the expected wrong-row assertion, restored and reconfirmed green
  • npx tsc --noEmit clean
  • npm run build:miner clean
  • npm run test:coverage (full, unsharded) — 15816/15828 passing (12 pre-existing skips), 100% patch coverage on touched lines/branches
  • npm audit --audit-level=moderate — 0 vulnerabilities

@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 13, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
loopover-ui 9c2ffaf Commit Preview URL

Branch Preview URL
Jul 13 2026, 07:26 AM

@codecov

codecov Bot commented Jul 13, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.00%. Comparing base (d402777) to head (9c2ffaf).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5583      +/-   ##
==========================================
+ Coverage   94.99%   95.00%   +0.01%     
==========================================
  Files         573      573              
  Lines       45490    45512      +22     
  Branches    14680    14680              
==========================================
+ Hits        43212    43240      +28     
+ Misses       1527     1521       -6     
  Partials      751      751              
Flag Coverage Δ
shard-1 44.03% <35.71%> (-0.01%) ⬇️
shard-2 35.70% <75.00%> (+0.03%) ⬆️
shard-3 31.94% <23.21%> (-0.22%) ⬇️
shard-4 31.78% <19.64%> (-1.09%) ⬇️
shard-5 32.93% <28.57%> (+1.07%) ⬆️
shard-6 44.39% <69.64%> (+0.21%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
packages/gittensory-miner/lib/discover-cli.js 100.00% <100.00%> (ø)
packages/gittensory-miner/lib/loop-cli.js 96.83% <100.00%> (+1.80%) ⬆️
...ckages/gittensory-miner/lib/portfolio-discovery.js 97.56% <100.00%> (+0.06%) ⬆️
...ckages/gittensory-miner/lib/portfolio-queue-cli.js 97.78% <100.00%> (+0.19%) ⬆️
...ges/gittensory-miner/lib/portfolio-queue-expiry.js 100.00% <100.00%> (ø)
...es/gittensory-miner/lib/portfolio-queue-manager.js 94.87% <100.00%> (+2.56%) ⬆️
packages/gittensory-miner/lib/portfolio-queue.js 96.90% <100.00%> (+1.16%) ⬆️
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@JSONbored JSONbored self-assigned this Jul 13, 2026
@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 13, 2026
@loopover-orb

loopover-orb Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Warning

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

⏸️ Gittensory review result - manual review recommended

Review updated: 2026-07-13 07:32:19 UTC

20 files · 1 AI reviewer · 2 blockers · readiness 93/100 · CI green · clean

⏸️ Suggested Action - Manual Review

Review summary
This threads an `apiBaseUrl` dimension through the portfolio-queue's primary key, statements, store API, CLI, and the two real call sites (discover-cli/portfolio-discovery, loop-cli, portfolio-queue-expiry) that previously let two forge hosts serving a same-named owner/repo collide. The v2→v3 migration correctly rebuilds the table with the composite PK, preserves rowid-order via `ORDER BY rowid`, and backfills existing rows with the github.com default; the SQL parameter order in every statement (api_base_url, repo_full_name, identifier) matches the call sites I checked line-by-line. The `markDone` switch to `UPDATE ... RETURNING` correctly removes a dead defensive branch, and `claimNextBatch`'s known engine-side limitation (no forge dimension) is honestly documented rather than silently patched.

Blockers

  • packages/gittensory-miner/lib/portfolio-queue-manager.js:99 drops apiBaseUrl from claimNextBatch's selected targets, so when a non-default-host item and a github.com item share repoFullName+identifier the engine can select the non-default row but batchClaim defaults the claim to https:​//api.github.com and marks the wrong row in_progress; change the manager selection path to preserve apiBaseUrl through entriesToPortfolioQueue/queueItemId parsing or do not pass hostless targets into store.batchClaim.
Nits — 5 non-blocking
  • packages/gittensory-miner/lib/portfolio-queue-cli.js:114 reimplements arg parsing as a manual loop (depth 5) instead of reusing the `parseJsonFlag` helper pattern used elsewhere in the file — consider extracting the `--api-base-url` handling to keep nesting down.
  • packages/gittensory-miner/lib/loop-cli.js now sits at ~434 lines (over the repo's 400-line file-size convention) after this change; consider whether the malformed-identifier/markDone branches could move to a helper.
  • The `batchClaim`/`claimNextBatch` path silently defaults to the github.com host whenever the engine's `PortfolioQueueItem` omits `apiBaseUrl` (portfolio-queue-manager.js) — documented in-code as acceptable (fail-to-claim, not miscorrelate), but worth a tracking issue since it's a real functional gap for non-default-host claims via the batch path.
  • Consider a follow-up issue to retrofit `@​jsonbored/gittensory-engine`'s `PortfolioQueueItem` with an `apiBaseUrl` field so `claimNextBatch` isn't permanently limited to the default host.
  • portfolio-queue-cli.js's new inline flag-parsing loop could be factored into a shared option-parsing helper shared with `parseDiscoverArgs`'s equivalent loop in discover-cli.js, since both now hand-roll the same `--flag value` pattern.

Concerns raised — review before merging

  • No linked issue detected — If this PR is intended to solve an issue, link it explicitly in the PR body.
  • Maintainer requires a linked issue — Link the relevant issue (for example Closes #123) before opening the PR.
📋 Copy for AI agents — paste into your coding agent
Fix the following blocker(s) from this PR review:

1. No linked issue detected — If this PR is intended to solve an issue, link it explicitly in the PR body.

2. Maintainer requires a linked issue — Link the relevant issue (for example `Closes #123`) before opening the PR.
Signal Result Evidence
Code review ❌ 2 blockers 1 reviewer
Linked issue ⚠️ Missing No linked issue or no-issue rationale found.
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (no linked issue context).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 44 registered-repo PR(s), 36 merged, 379 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 44 PR(s), 379 issue(s).
Gate result ❌ Blocking Repo-configured hard blocker found.
Improvement ✅ Minor risk: clean · value: minor — Code changes are accompanied by test evidence.
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: not available
  • Official Gittensor activity: 44 PR(s), 379 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • Explain no-issue PR.
  • Link the issue being solved, or explicitly explain why this is a no-issue PR.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
[BETA] Chat with Gittensory

Ask Gittensory a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @gittensory ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @gittensory chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @gittensory mention with a real question is routed to the closest matching read-only command automatically -- no exact syntax required.

Full command reference: https://gittensory.aethereal.dev/docs/gittensory-commands

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

PRIMARY KEY (repo_full_name, identifier) let two different forge hosts
(github.com vs. a GitHub Enterprise host, #4784) serving a same-named
owner/repo collide in the portfolio queue. Rebuild the constraint to
PRIMARY KEY (api_base_url, repo_full_name, identifier) as the store's next
schema migration (v2 -> v3, after the existing leased_at migration),
backfilling existing rows with the pre-#4784 implicit default.

Thread an optional apiBaseUrl through the store's API (enqueue/markDone/
markFailed/reclaimStuckItem/requeueItem/batchClaim), the admin CLI
(--api-base-url on done/release/requeue), and the two places that already
had a resolved apiBaseUrl in scope but weren't passing it through:
enqueueRankedDiscovery (discover-cli.js's real discovery pipeline) and the
stuck-lease/malformed-identifier sweeps in portfolio-queue-expiry.js and
loop-cli.js, which now echo each item's own apiBaseUrl back instead of
defaulting -- a defaulted echo would touch the wrong host's row whenever
two hosts share an owner/repo+identifier.

markDone now uses UPDATE ... RETURNING (matching this store's own existing
statements), removing a structurally-unreachable defensive branch the old
separate-SELECT pattern left behind.

Every existing caller is unaffected: apiBaseUrl defaults to
https://api.github.com when omitted. claimNextBatch's engine-driven
selection (portfolio-queue-manager.js) has no forge dimension in
@jsonbored/gittensory-engine's PortfolioQueueItem shape -- documented as a
known, safe (no-collision) limitation rather than silently patched over.

Advances #5563 (portfolio-queue.js of 5 affected stores; claim-ledger.js
landed in #5576).
@JSONbored
JSONbored force-pushed the forge-scope-ledger-keys branch from 9022816 to 9c2ffaf Compare July 13, 2026 07:25
JSONbored added a commit that referenced this pull request Jul 13, 2026
repo_full_name TEXT PRIMARY KEY let two forge hosts (github.com vs. a
GitHub Enterprise host, #4784) serving a same-named owner/repo share one
"current discover/plan/prepare state" row. Rebuild the constraint to
PRIMARY KEY (api_base_url, repo_full_name), backfilling existing rows with
the pre-#4784 implicit default. The migration uses INSERT OR IGNORE so a
pre-existing row with an already-invalid state (this store's read path
already fails closed on those) can't abort the whole rebuild.

Thread an optional apiBaseUrl through the store's API (getRunState/
setRunState) and the admin CLI (--api-base-url on state get/set). Every
existing caller is unaffected: apiBaseUrl defaults to
https://api.github.com when omitted.

manage-status.js's collectRunPortfolio folds run state into one row per
repo NAME (not per host) for its dashboard view -- documented in-code as a
known, safe (no data loss, read-only) display limitation for the narrow
case of the same repo name existing on two hosts, rather than silently
patched over; broadening that fold to be host-aware is a separate,
larger dashboard-shape change.

Advances #5563 (run-state.js of 5 affected stores; claim-ledger.js landed
in #5576, portfolio-queue.js in #5583).
@JSONbored
JSONbored merged commit 8e42774 into main Jul 13, 2026
19 checks passed
@JSONbored
JSONbored deleted the forge-scope-ledger-keys branch July 13, 2026 07:32
JSONbored added a commit that referenced this pull request Jul 13, 2026
repo_full_name TEXT PRIMARY KEY let two forge hosts (github.com vs. a
GitHub Enterprise host, #4784) serving a same-named owner/repo share one
"current discover/plan/prepare state" row. Rebuild the constraint to
PRIMARY KEY (api_base_url, repo_full_name), backfilling existing rows with
the pre-#4784 implicit default. The migration uses INSERT OR IGNORE so a
pre-existing row with an already-invalid state (this store's read path
already fails closed on those) can't abort the whole rebuild.

Thread an optional apiBaseUrl through the store's API (getRunState/
setRunState) and the admin CLI (--api-base-url on state get/set). Every
existing caller is unaffected: apiBaseUrl defaults to
https://api.github.com when omitted.

manage-status.js's collectRunPortfolio folds run state into one row per
repo NAME (not per host) for its dashboard view -- documented in-code as a
known, safe (no data loss, read-only) display limitation for the narrow
case of the same repo name existing on two hosts, rather than silently
patched over; broadening that fold to be host-aware is a separate,
larger dashboard-shape change.

Advances #5563 (run-state.js of 5 affected stores; claim-ledger.js landed
in #5576, portfolio-queue.js in #5583).
JSONbored added a commit that referenced this pull request Jul 13, 2026
… repoFullName (#5591)

repo_full_name TEXT PRIMARY KEY let two forge hosts (github.com vs. a
GitHub Enterprise host, #4784) serving a same-named owner/repo share one
reputation-history row. Rebuild the constraint to
PRIMARY KEY (api_base_url, repo_full_name), following governor-state.js's
existing idempotent-ALTER convention (ensurePauseColumns) rather than
introducing the applySchemaMigrations framework into this file for the
first time -- a full PRIMARY KEY rebuild still needs the create-new/copy/
drop/rename dance, just gated by a column-presence check instead of a
version stamp.

Uses INSERT OR IGNORE for the copy step (a corrupted legacy row violating
the rebuilt table's NOT NULL columns is dropped, not a migration-aborting
crash), regression-tested the same way as run-state.js's #5563 fix.

Threads an optional apiBaseUrl through loadReputationHistory/
saveReputationHistory. loadReputationHistory/saveReputationHistory have no
real callers yet (attempt-runner.js's own comments note this governor
input isn't wired into the attempt pipeline yet), so there is no CLI
surface or call site to thread through -- this is purely the storage-layer
fix, mirroring openClaimLedgerReadOnly's scaffold-only precedent.

Advances #5563 (governor_reputation_history of 5 affected stores;
claim-ledger.js landed in #5576, portfolio-queue.js in #5583, run-state.js
in #5585; deny-hook-synthesis.js remains).
JSONbored added a commit that referenced this pull request Jul 13, 2026
…re repoFullName (#5595)

deny_rule_proposals' PRIMARY KEY (repo_full_name, id) let two forge hosts
(github.com vs. a GitHub Enterprise host, #4784) serving a same-named
owner/repo share one proposal row (and its maintainer approval decision).
Rebuild the constraint to PRIMARY KEY (api_base_url, repo_full_name, id).

This file has no schema-version framework of its own (unlike the
package's other local stores) -- it uses a raw DatabaseSync connection, no
applySchemaMigrations. Follows governor-state.js's idempotent
column-presence-gated rebuild convention instead of introducing a new
framework dependency here for the first time. Uses INSERT OR IGNORE for
the copy step, matching the fix already applied to every other #5563
migration in this epic: a legacy row with an already-invalid status value
(this store's own CHECK-constrained schema already rejects those) is
dropped, not a migration-aborting crash.

Threads an optional apiBaseUrl through refreshProposals/listProposals/
setProposalStatus/resolveEffectiveRules. initDenyHookSynthesisStore has no
real callers yet (feeds the consumption surface #2343 will eventually wire
into evaluateDenyHooks; this store owns derivation + audit, not live hook
interception), so there is no CLI surface or call site to thread through --
purely the storage-layer fix, mirroring governor_reputation_history's
scaffold-only precedent.

Closes #5563 (5th and final store: claim-ledger.js in #5576,
portfolio-queue.js in #5583, run-state.js in #5585, governor-state.js in
#5591, plus the claimNextBatch/migration hardening follow-up in #5594).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. manual-review Gittensor contributor context

Development

Successfully merging this pull request may close these issues.

Scope local ledger keys by forge host, not bare repoFullName

1 participant