Skip to content

miner(discover): --dry-run still creates and writes the contribution-profile cache #10000

Description

@JSONbored

⚠️ Definition of Done: this issue must be completed in full, in a single PR. Do not split this
work across multiple PRs, and do not defer any Deliverable below to a follow-up issue. A PR that
satisfies only some of the Deliverables, stubs a required test, or leaves a checkbox
partially-done does NOT resolve this issue and will be closed.

Context

loopover-miner discover --dry-run documents a hard zero-write contract in two places.

packages/loopover-miner/lib/discover-cli.ts:557-561:

  // #4847: fetch + rank are read-only GitHub GETs and pure local computation, so a dry run still does them for
  // real (that's the useful "what would this discover?" output) -- but it never opens any local store (portfolio
  // queue, policy-doc cache, policy-verdict cache), since opening a not-yet-existing SQLite store file is itself
  // a write.

packages/loopover-miner/lib/discover-cli.ts:530-536:

      // #9679: --dry-run must make ZERO filesystem writes, but initEventLedger creates + migrates + prunes the
      // ledger file. On the dry-run path only read the override when the ledger file ALREADY exists (opening a
      // not-yet-existing SQLite file is itself a write, and retention pruning can delete rows)

The dry-run branch nevertheless calls the default contribution-profile resolver, at
packages/loopover-miner/lib/discover-cli.ts:579-583:

      const profilesByRepo = await resolveProfiles(repoFullNames, {
        githubToken,
        ...(apiBaseUrl !== undefined ? { apiBaseUrl } : {}),
        ...(options.nowMs !== undefined ? { nowMs: options.nowMs } : {}),
      });

resolveProfiles defaults to resolveContributionProfilesForDiscover
(packages/loopover-miner/lib/discover-cli.ts:464-498), which opens and writes a real SQLite store as soon as a
GitHub token is present:

  const profiles = new Map();
  if (!ctx.githubToken) return profiles;
  const initCache = (ctx.initCache as typeof initContributionProfileCache | undefined) ?? initContributionProfileCache;
  ...
  const cache = initCache();

initContributionProfileCache (packages/loopover-miner/lib/contribution-profile-cache.ts:73-86) goes through
openLocalStoreAdapteropenLocalStoreDb
(packages/loopover-miner/lib/local-store.ts:66-88), which does mkdirSync(dirname(resolvedPath), …), constructs
new DatabaseSync(resolvedPath) (creating the file), chmodSync(resolvedPath, 0o600), then runs
CREATE TABLE IF NOT EXISTS … and applySchemaMigrations(db, []) (which stamps PRAGMA user_version). On a cache
miss the dry run then also writes rows via cache.put(profile, ctx.nowMs)
(packages/loopover-miner/lib/discover-cli.ts:491).

A real --dry-run invocation always carries a token — githubToken is resolved at
packages/loopover-miner/lib/discover-cli.ts:511 from process.env[tokenEnv], and without one the fan-out this
dry run is previewing could not authenticate at all. So on every realistic operator dry run,
contribution-profile-cache.sqlite3 is created (or mutated) and the printed
"DRY RUN: no portfolio-queue write was made." line at packages/loopover-miner/lib/discover-cli.ts:623 is
misleading about what the command actually did to local state.

A second, independent defect in the same resolver: resolveContributionProfilesForDiscover calls initCache()
with no argument, so resolveContributionProfileCacheDbPath() falls back to process.env
(packages/loopover-miner/lib/contribution-profile-cache.ts:40-46, local-store.ts:20-46). Every other store path
in runDiscover is resolved from options.env — see packages/loopover-miner/lib/discover-cli.ts:529-530:

      const ledgerEnv = options.env ?? process.env;
      const ledgerDbPath = resolveEventLedgerDbPath(ledgerEnv);

So a caller that supplies options.env with LOOPOVER_MINER_CONFIG_DIR (or
LOOPOVER_MINER_CONTRIBUTION_PROFILE_CACHE_DB) gets every store redirected except the contribution-profile
cache, which silently writes to the ambient process.env location instead.

The existing regression coverage misses both: test/unit/miner-discover-cli.test.ts:581 ("#4847: --dry-run …
never opens any local store") and the #9679 block at test/unit/miner-discover-cli.test.ts:2271 both run without
a githubToken, so resolveContributionProfilesForDiscover returns early at its if (!ctx.githubToken) guard and
the cache is never reached.

Requirements

  • On the --dry-run path in runDiscover, the default contribution-profile resolver must never open or create
    the contribution-profile cache SQLite file and must never write a row to it. Apply the same "skip a file that
    doesn't exist yet" discipline the #9679 event-ledger block at
    packages/loopover-miner/lib/discover-cli.ts:537 already uses: when the cache file does not exist, no store is
    opened; when it already exists, reads are permitted but put must not be called during a dry run.
  • The eligibility filter must still run on the dry-run path with whatever profiles were resolved, so the previewed
    excluded set continues to match what a real run would produce. The behaviour asserted by
    test/unit/miner-discover-cli.test.ts:1821 ("applies the same eligibility filter on a --dry-run") must not
    change.
  • resolveContributionProfilesForDiscover must accept the caller's env object and resolve the cache DB path from
    it via resolveContributionProfileCacheDbPath(env), passing the result to initContributionProfileCache(dbPath).
    runDiscover must thread options.env ?? process.env through on both the dry-run and real-run call sites
    (packages/loopover-miner/lib/discover-cli.ts:579 and :701).
  • When no env is supplied, path resolution must remain byte-identical to today (process.env), so every existing
    caller is unaffected.
  • The RunDiscoverOptions.resolveContributionProfiles injection seam must keep working: an injected resolver is
    still called on both paths and is still the only thing invoked (no second, parallel default resolver call).
  • Do NOT change filterCandidatesByProfiles, the ranker, or the enqueue path.

⚠️ Required pattern: mirror packages/loopover-miner/lib/discover-cli.ts:526-546 — the #9679 event-ledger
block: resolve the DB path from the caller's env, guard the open with existsSync on the dry-run path, and keep
the real-run path opening unconditionally. What does NOT satisfy this issue: (a) deleting the
resolveProfiles(...) call from the dry-run branch entirely, which would silently change the previewed
excluded set and break test/unit/miner-discover-cli.test.ts:1821; (b) adding a new dryRun-aware wrapper
resolver alongside resolveContributionProfilesForDiscover instead of fixing that function and its call sites;
(c) a test-only PR that adds a token to the existing #4847/#9679 tests without changing discover-cli.ts.

Deliverables

  • resolveContributionProfilesForDiscover in packages/loopover-miner/lib/discover-cli.ts accepts env and
    dryRun in its ctx argument, resolves the cache path via resolveContributionProfileCacheDbPath(env),
    and passes that path to initContributionProfileCache(...).
  • With dryRun: true and a cache file that does not exist, resolveContributionProfilesForDiscover opens no
    store at all and returns a Map built from freshly-extracted profiles (or an empty Map when there is no
    token) — asserted in test/unit/miner-discover-cli.test.ts against a temp LOOPOVER_MINER_CONFIG_DIR, with
    existsSync(resolveContributionProfileCacheDbPath(env)) still false after the run.
  • With dryRun: true and a cache file that already exists and holds a fresh row for the candidate repo, the
    run reads it and makes no put — asserted by comparing the row's fetched_at before and after.
  • runDiscover(["acme/widgets", "--dry-run", "--json"], { env: { LOOPOVER_MINER_CONFIG_DIR: tmp }, githubToken: "t", … }) creates no contribution-profile-cache.sqlite3 anywhere — asserted in
    test/unit/miner-discover-cli.test.ts.
  • A non-dry runDiscover with options.env = { LOOPOVER_MINER_CONFIG_DIR: tmp } and a token creates the cache
    file under tmp (not under the ambient process.env location) — asserted in
    test/unit/miner-discover-cli.test.ts.
  • A regression test at test/unit/miner-discover-cli.test.ts named for this bug (e.g. REGRESSION: --dry-run creates no contribution-profile cache file when a GitHub token is present) that fails against the current
    code.

All Deliverables above are required in a single PR. A PR that satisfies only some of them — for example one that
adds the existsSync dry-run guard but leaves initCache() resolving its path from process.env — does not
resolve this issue.

Test Coverage Requirements

This repo enforces 99%+ Codecov patch coverage, branch-counted. vitest.config.ts's coverage.include lists
packages/loopover-miner/lib/**/*.ts, so discover-cli.ts is measured and gated. The change introduces or touches
these branches, and both arms of each need a test: the dryRun true/false split inside
resolveContributionProfilesForDiscover; the existsSync(cacheDbPath) true/false split on the dry-run path; the
ctx.env supplied / omitted fallback to process.env; the existing if (!ctx.githubToken) return profiles;
early-return (both arms); and the cache-hit cached && !cached.stale vs miss arms, which must be exercised on both
the dry-run and real-run paths.

Expected Outcome

loopover-miner discover --dry-run becomes a genuinely zero-write command in the presence of a GitHub token — the
state it advertises it will not touch is the state it actually does not touch — and every local store path
runDiscover resolves, including the contribution-profile cache, honours the caller-supplied env instead of
silently falling back to the ambient process environment.

Links & Resources

  • packages/loopover-miner/lib/discover-cli.ts:464-498resolveContributionProfilesForDiscover, opens and
    writes the cache with no dry-run or env awareness
  • packages/loopover-miner/lib/discover-cli.ts:526-546 — the #9679 event-ledger guard to mirror
  • packages/loopover-miner/lib/discover-cli.ts:557-583 — the dry-run branch and its documented zero-write contract
  • packages/loopover-miner/lib/contribution-profile-cache.ts:73-130initContributionProfileCache / put
  • packages/loopover-miner/lib/local-store.ts:66-88openLocalStoreDb's mkdir/create/chmod
  • test/unit/miner-discover-cli.test.ts:581, :1821, :2271 — existing dry-run coverage that misses this path

Metadata

Metadata

Assignees

No one assigned

    Labels

    gittensor:bugGittensor-scored bug fix — scores a 0.05x multiplier.help wantedExtra attention is needed

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions