Skip to content

purge-cli.js's right-to-be-forgotten sweep misses portfolio-queue.js and run-state.js #6599

Description

@JSONbored

Context

packages/loopover-miner/lib/purge-cli.js (#5564) implements loopover-miner purge --repo <owner/repo>, the operator-invoked right-to-be-forgotten path. Its own header comment states it "deletes every row for one repo from the four stores that have a real repoColumn" — claim-ledger.js, event-ledger.js, governor-ledger.js, and prediction-ledger.js — each via a purgeByRepo method built on store-maintenance.js's shared purgeStoreByRepo, driven by a purge spec (CLAIM_LEDGER_PURGE_SPEC, EVENT_LEDGER_PURGE_SPEC, GOVERNOR_LEDGER_PURGE_SPEC, PREDICTION_LEDGER_PURGE_SPEC in store-maintenance.js).

That "four stores" claim is incomplete. Two more local stores persist rows keyed by repo_full_name and are not covered by purge-cli.js at all:

  • packages/loopover-miner/lib/portfolio-queue.js's miner_portfolio_queue table has a repo_full_name column that is part of its own primary key (PRIMARY KEY (api_base_url, repo_full_name, identifier)).
  • packages/loopover-miner/lib/run-state.js's miner_run_state table has a repo_full_name column that is likewise part of its primary key (PRIMARY KEY (api_base_url, repo_full_name)).

Neither store has a purgeByRepo method, neither has a purge spec in store-maintenance.js, and neither appears in purge-cli.js's REAL_PURGE_TARGETS. An operator running loopover-miner purge --repo owner/repo today leaves that repo's rows in both stores untouched, with no warning (unlike attempt-log.js, which is deliberately reported as not-purgeable with an explicit ATTEMPT_LOG_NOT_PURGEABLE_NOTE because it genuinely has no repo column to purge by — portfolio-queue.js and run-state.js have no such structural excuse).

Requirements

  • packages/loopover-miner/lib/store-maintenance.js MUST export two new purge specs, PORTFOLIO_QUEUE_PURGE_SPEC = { table: "miner_portfolio_queue", repoColumn: "repo_full_name" } and RUN_STATE_PURGE_SPEC = { table: "miner_run_state", repoColumn: "repo_full_name" }, following the exact shape of the four existing purge specs.
  • packages/loopover-miner/lib/portfolio-queue.js's store object MUST gain a purgeByRepo(repoFullName) method that calls purgeStoreByRepo(db, PORTFOLIO_QUEUE_PURGE_SPEC, normalizeRepoFullName(repoFullName)), mirroring claim-ledger.js's purgeByRepo implementation (throwing on a missing/malformed repoFullName rather than silently no-opping).
  • packages/loopover-miner/lib/run-state.js's store object MUST gain a purgeByRepo(repoFullName) method with the same contract, using RUN_STATE_PURGE_SPEC.
  • packages/loopover-miner/lib/purge-cli.js's REAL_PURGE_TARGETS array MUST include entries for portfolio-queue and run-state, wired the same way as the four existing entries (name, optionKey, opener, resolveDbPath, spec), so both runPurgeDryRun and runPurge cover them automatically.
  • purge-cli.js's header comment claiming "the four stores that have a real repoColumn" MUST be updated to name all six stores now covered.
  • runPurgeDryRun's --dry-run output and runPurge's real-purge output MUST report per-store row counts for portfolio-queue and run-state exactly as they already do for the four existing stores (no special-casing).

Deliverables

  • PORTFOLIO_QUEUE_PURGE_SPEC and RUN_STATE_PURGE_SPEC added to packages/loopover-miner/lib/store-maintenance.js.
  • purgeByRepo added to packages/loopover-miner/lib/portfolio-queue.js and packages/loopover-miner/lib/run-state.js.
  • packages/loopover-miner/lib/purge-cli.js's REAL_PURGE_TARGETS and header comment updated to include both stores.
  • Tests in each store's existing unit test file covering purgeByRepo (deletes only the targeted repo's rows, leaves other repos untouched, returns 0 when nothing matches, throws on an invalid repoFullName) mirroring test/unit/miner-prediction-ledger.test.ts's existing purgeByRepo test block.
  • A test in packages/loopover-miner's purge-cli test file asserting both --dry-run and a real purge now report portfolio-queue and run-state in their per-store output.

Test Coverage Requirements

All touched paths are under packages/loopover-miner/lib/**, covered by this repo's 99%+ Codecov patch gate. Every new line — both purgeByRepo methods, both new purge specs, and the REAL_PURGE_TARGETS additions — must be exercised by the tests above.

Expected Outcome

loopover-miner purge --repo <owner/repo> removes that repo's rows from every local store that persists repo-identifying data, not just four of the six that structurally support it, and both --dry-run and the real purge report exactly what was (or would be) removed from portfolio-queue and run-state alongside the four stores already covered.

Links & Resources

  • packages/loopover-miner/lib/store-maintenance.js (shared purgeStoreByRepo/countStoreByRepo)
  • packages/loopover-miner/lib/claim-ledger.js (reference purgeByRepo implementation)
  • packages/loopover-miner/lib/purge-cli.js (Add a purge / right-to-be-forgotten path across the local ledgers #5564)
  • test/unit/miner-prediction-ledger.test.ts's existing purgeByRepo describe block (reference test shape)

Metadata

Metadata

Assignees

No one assigned

    Labels

    gittensor:bugGittensor-scored bug fix — scores a 0.05x multiplier.help wantedExtra attention is needed

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions