⚠️ Definition of Done: this issue must be completed in full, in a single PR. Do not split this
work across multiple PRs, and do not defer any Deliverable below to a follow-up issue. A PR that
satisfies only some of the Deliverables, stubs a required test, or leaves a checkbox
partially-done does NOT resolve this issue and will be closed.
Context
runLoop primes its ledger cursor by reading the whole ledger and taking the last element's seq —
packages/loopover-miner/lib/loop-cli.ts:342:
let sinceSeq = eventLedger.readEvents({}).at(-1)?.seq ?? 0;
readEvents with no filter runs SELECT * FROM miner_event_ledger ORDER BY seq ASC
(packages/loopover-miner/lib/event-ledger.ts:186, :231-233) and maps every row through rowToEntry
(packages/loopover-miner/lib/event-ledger.ts:126-135), which does a JSON.parse(row.payload_json) per row:
function rowToEntry(row: EventDbRow): LedgerEntry {
return {
id: row.id,
seq: row.seq,
type: row.event_type,
repoFullName: row.repo_full_name,
payload: JSON.parse(row.payload_json),
createdAt: row.created_at,
};
}
So every loopover-miner loop invocation materializes and JSON-parses the miner's entire append-only audit trail
in order to read a single integer, then discards all of it. The ledger is unbounded by default: retention is
opt-in and off unless an operator sets LOOPOVER_MINER_LEDGER_RETENTION_DAYS or
LOOPOVER_MINER_LEDGER_RETENTION_MAX_ROWS
(packages/loopover-miner/lib/store-maintenance.ts:8, :19-20, :146-156), and the module header describes the
ledger as "an immutable audit trail of every significant miner-loop event"
(packages/loopover-miner/lib/event-ledger.ts:14-17). A long-lived AMS box accumulates one row per discovered
issue, per plan step, per manage-poll snapshot, per PR outcome — and runLoop, the daemon meant to run
continuously, pays for all of them at startup.
The exact statement needed already exists inside the store, one line away from the fix —
packages/loopover-miner/lib/event-ledger.ts:180:
const nextSeqStatement = db.prepare("SELECT COALESCE(MAX(seq), 0) + 1 AS nextSeq FROM miner_event_ledger");
seq is INTEGER NOT NULL UNIQUE (packages/loopover-miner/lib/event-ledger.ts:168), so SQLite has an index on
it and MAX(seq) is an index lookup, not a scan.
Every other consumer of the cursor already works on bounded, filtered reads:
buildLoopClosureSummary is called with { sinceSeq, repoFullName }
(packages/loopover-miner/lib/loop-cli.ts:600) and its result reassigns the cursor at :602. Only the priming
read at :342 is unbounded.
Requirements
EventLedger gains a latestSeq(): number method that returns the current MAX(seq), or 0 for an empty
ledger, without materializing any row. Implement it with a SELECT COALESCE(MAX(seq), 0) AS latestSeq FROM miner_event_ledger statement prepared once at open time, alongside the existing statements at
packages/loopover-miner/lib/event-ledger.ts:180-195.
packages/loopover-miner/lib/loop-cli.ts:342 uses eventLedger.latestSeq() and no longer calls
readEvents({}).
latestSeq() must be exposed on the EventLedger type
(packages/loopover-miner/lib/event-ledger.ts:45-51) and mirrored by a module-level convenience export next to
appendEvent / readEvents (packages/loopover-miner/lib/event-ledger.ts:256-262), matching this module's
existing default-ledger convention.
RunLoopOptions.initEventLedger (packages/loopover-miner/lib/loop-cli.ts:94) is an injection seam — every
test double supplying an EventLedger must be updated so the loop still works with an injected ledger, and the
new method must be part of the seam, not read off the concrete store type.
latestSeq() must return 0 for a ledger with no rows, so runLoop's existing ?? 0 semantics are preserved
exactly and the first cycle's sinceSeq is unchanged.
- After an
appendEvent, latestSeq() must return the newly-appended entry's seq — the two must never
disagree.
- Do NOT change
readEvents, appendEvent, rowToEntry, the retention path, or the ledger's immutability
invariant (packages/loopover-miner/lib/event-ledger.ts:14-23).
- Do NOT change the other
readEvents() callers
(packages/loopover-miner/lib/manage-status.ts:141, calibration-cli.ts, metrics-cli.ts,
ams-calibration.ts, signal-tracking-store.ts) — they genuinely need the rows.
⚠️ Required pattern: add the prepared statement next to nextSeqStatement
(packages/loopover-miner/lib/event-ledger.ts:180) and return it from the same object literal the other methods
are defined on (packages/loopover-miner/lib/event-ledger.ts:197-248). What does NOT satisfy this issue:
(a) leaving readEvents({}) in place and slicing it (.slice(-1)), which still loads and parses every row;
(b) adding a limit/order option to readEvents and threading it through every caller, a change to the
ledger's whole read surface; (c) caching the seq in a module-level variable, which goes stale the moment a
sibling process appends.
Deliverables
All Deliverables above are required in a single PR. A PR that satisfies only some of them — for example one that
adds latestSeq() to the ledger but leaves packages/loopover-miner/lib/loop-cli.ts:342 calling
readEvents({}) — does not resolve this issue.
Test Coverage Requirements
This repo enforces 99%+ Codecov patch coverage, branch-counted. vitest.config.ts's coverage.include lists
packages/loopover-miner/lib/**/*.ts, so event-ledger.ts and loop-cli.ts are measured and gated. Every branch
the change introduces needs both arms tested: latestSeq() on an empty ledger (the COALESCE zero path) and on a
populated one; the module-level convenience export's lazily-opened default-ledger path
(packages/loopover-miner/lib/event-ledger.ts:251-254); and, in runLoop, both the initial-halt path (which
must still prime the cursor) and the normal path that reaches buildLoopClosureSummary.
Expected Outcome
Starting loopover-miner loop costs one indexed MAX(seq) lookup instead of loading and JSON-parsing the miner's
entire event ledger, so loop start-up time and memory stop growing with the length of the audit trail on
long-lived AMS boxes.
Links & Resources
packages/loopover-miner/lib/loop-cli.ts:342 — the unbounded priming read
packages/loopover-miner/lib/loop-cli.ts:600-602 — the only other cursor consumer, already bounded
packages/loopover-miner/lib/event-ledger.ts:126-135 — rowToEntry's per-row JSON.parse
packages/loopover-miner/lib/event-ledger.ts:180 — the existing MAX(seq) statement to mirror
packages/loopover-miner/lib/event-ledger.ts:186, :231-233 — the unfiltered SELECT *
packages/loopover-miner/lib/store-maintenance.ts:8, :146-156 — retention is opt-in, so the ledger is
unbounded by default
Context
runLoopprimes its ledger cursor by reading the whole ledger and taking the last element'sseq—packages/loopover-miner/lib/loop-cli.ts:342:readEventswith no filter runsSELECT * FROM miner_event_ledger ORDER BY seq ASC(
packages/loopover-miner/lib/event-ledger.ts:186,:231-233) and maps every row throughrowToEntry(
packages/loopover-miner/lib/event-ledger.ts:126-135), which does aJSON.parse(row.payload_json)per row:So every
loopover-miner loopinvocation materializes and JSON-parses the miner's entire append-only audit trailin order to read a single integer, then discards all of it. The ledger is unbounded by default: retention is
opt-in and off unless an operator sets
LOOPOVER_MINER_LEDGER_RETENTION_DAYSorLOOPOVER_MINER_LEDGER_RETENTION_MAX_ROWS(
packages/loopover-miner/lib/store-maintenance.ts:8,:19-20,:146-156), and the module header describes theledger as "an immutable audit trail of every significant miner-loop event"
(
packages/loopover-miner/lib/event-ledger.ts:14-17). A long-lived AMS box accumulates one row per discoveredissue, per plan step, per manage-poll snapshot, per PR outcome — and
runLoop, the daemon meant to runcontinuously, pays for all of them at startup.
The exact statement needed already exists inside the store, one line away from the fix —
packages/loopover-miner/lib/event-ledger.ts:180:seqisINTEGER NOT NULL UNIQUE(packages/loopover-miner/lib/event-ledger.ts:168), so SQLite has an index onit and
MAX(seq)is an index lookup, not a scan.Every other consumer of the cursor already works on bounded, filtered reads:
buildLoopClosureSummaryis called with{ sinceSeq, repoFullName }(
packages/loopover-miner/lib/loop-cli.ts:600) and its result reassigns the cursor at:602. Only the primingread at
:342is unbounded.Requirements
EventLedgergains alatestSeq(): numbermethod that returns the currentMAX(seq), or0for an emptyledger, without materializing any row. Implement it with a
SELECT COALESCE(MAX(seq), 0) AS latestSeq FROM miner_event_ledgerstatement prepared once at open time, alongside the existing statements atpackages/loopover-miner/lib/event-ledger.ts:180-195.packages/loopover-miner/lib/loop-cli.ts:342useseventLedger.latestSeq()and no longer callsreadEvents({}).latestSeq()must be exposed on theEventLedgertype(
packages/loopover-miner/lib/event-ledger.ts:45-51) and mirrored by a module-level convenience export next toappendEvent/readEvents(packages/loopover-miner/lib/event-ledger.ts:256-262), matching this module'sexisting default-ledger convention.
RunLoopOptions.initEventLedger(packages/loopover-miner/lib/loop-cli.ts:94) is an injection seam — everytest double supplying an
EventLedgermust be updated so the loop still works with an injected ledger, and thenew method must be part of the seam, not read off the concrete store type.
latestSeq()must return0for a ledger with no rows, sorunLoop's existing?? 0semantics are preservedexactly and the first cycle's
sinceSeqis unchanged.appendEvent,latestSeq()must return the newly-appended entry'sseq— the two must neverdisagree.
readEvents,appendEvent,rowToEntry, the retention path, or the ledger's immutabilityinvariant (
packages/loopover-miner/lib/event-ledger.ts:14-23).readEvents()callers(
packages/loopover-miner/lib/manage-status.ts:141,calibration-cli.ts,metrics-cli.ts,ams-calibration.ts,signal-tracking-store.ts) — they genuinely need the rows.Deliverables
EventLedger.latestSeq()exists inpackages/loopover-miner/lib/event-ledger.ts, is declared on theEventLedgertype, and is backed by aCOALESCE(MAX(seq), 0)statement prepared at open time.initEventLedger(":memory:").latestSeq()returns0; after threeappendEventcalls it returns3andequals the third entry's
seq— asserted intest/unit/miner-event-ledger.test.ts.runLoopno longer callsreadEventsto primesinceSeq: with an injected event ledger whosereadEventsis a spy, arunLoopinvocation that halts on the initial kill switch callslatestSeq()onceand
readEventszero times — asserted intest/unit/miner-loop-cli.test.ts.runLoopprimessinceSeqfromlatestSeq(): with an injected ledger reportinglatestSeq() === 7, thefirst
buildLoopClosureSummarycall receivessinceSeq: 7— asserted intest/unit/miner-loop-cli.test.ts.latestSeq() === 0), the firstbuildLoopClosureSummarycall receivessinceSeq: 0— asserted intest/unit/miner-loop-cli.test.ts.REGRESSION: runLoop primes its ledger cursor without reading every event) that fails against the current code.All Deliverables above are required in a single PR. A PR that satisfies only some of them — for example one that
adds
latestSeq()to the ledger but leavespackages/loopover-miner/lib/loop-cli.ts:342callingreadEvents({})— does not resolve this issue.Test Coverage Requirements
This repo enforces 99%+ Codecov patch coverage, branch-counted.
vitest.config.ts'scoverage.includelistspackages/loopover-miner/lib/**/*.ts, soevent-ledger.tsandloop-cli.tsare measured and gated. Every branchthe change introduces needs both arms tested:
latestSeq()on an empty ledger (theCOALESCEzero path) and on apopulated one; the module-level convenience export's lazily-opened default-ledger path
(
packages/loopover-miner/lib/event-ledger.ts:251-254); and, inrunLoop, both the initial-halt path (whichmust still prime the cursor) and the normal path that reaches
buildLoopClosureSummary.Expected Outcome
Starting
loopover-miner loopcosts one indexedMAX(seq)lookup instead of loading and JSON-parsing the miner'sentire event ledger, so loop start-up time and memory stop growing with the length of the audit trail on
long-lived AMS boxes.
Links & Resources
packages/loopover-miner/lib/loop-cli.ts:342— the unbounded priming readpackages/loopover-miner/lib/loop-cli.ts:600-602— the only other cursor consumer, already boundedpackages/loopover-miner/lib/event-ledger.ts:126-135—rowToEntry's per-rowJSON.parsepackages/loopover-miner/lib/event-ledger.ts:180— the existingMAX(seq)statement to mirrorpackages/loopover-miner/lib/event-ledger.ts:186,:231-233— the unfilteredSELECT *packages/loopover-miner/lib/store-maintenance.ts:8,:146-156— retention is opt-in, so the ledger isunbounded by default