Repository navigation
feat(agent-finops): persist model usage ledger and expose cost queries #266
Description
Parent roadmap
mctlhq/.github#48
Depends on
mctlhq/.github#49— canonicalModelUsageRecord/ pricing contract.mctlhq/.github#50— first DevLoop producer instrumentation for the pilot.
Goal
Provide an mctl-owned durable usage/cost ledger and query surface so token/cost history survives trace-backend retention changes and can answer operator questions such as:
- how much did this DevLoop cost?
- how much did investigator vs implementer vs shepherd spend?
- what did issue/PR/repository X spend in a time window?
- which provider/model accounted for the spend?
- what fraction of usage came from cache/reasoning tokens?
The ledger is a financial/usage read model, not a replacement for traces or provider invoices.
Scope
Durable append-oriented store
Persist normalized records conforming to #49 with deterministic idempotency/dedupe.
Requirements:
- append-oriented/auditable history;
- unique invocation/dedupe key;
- trace/execution/workflow correlation;
- no prompt/completion body required;
- retention longer than or independent from trace-backend retention;
- schema/version migrations preserve historical pricing semantics.
Versioned pricing catalog
Persist or load a versioned mctl pricing catalog with effective dates and provider/model matching rules.
Calculated cost must record the exact pricing version/rates used so historical values remain reproducible after price changes.
Do not overwrite provider-reported cost with calculated cost.
Ingestion contract
Define an authenticated/idempotent ingestion path for usage events emitted by mctl-agents / other trusted producers.
The first implementation may be REST/internal API, queue, or another existing mctl pattern selected during implementation. Do not couple ingestion to a specific tracing backend.
Query surface
Expose read operations sufficient for:
by execution/workflow/work_item
by repository / issue / PR
by agent / DevLoop stage
by provider / model
by time window
by terminal outcome
Return both token breakdown and cost classification (provider_reported, calculated, future invoice_reconciled) without implying estimates are invoice truth.
Reconciliation boundary
Define a future-safe hook for provider billing/export reconciliation, but do not implement full accounting integration in this issue unless trivial.
Acceptance criteria
- Usage records can be ingested idempotently from the DevLoop pilot.
- Duplicate delivery/retry cannot double-count a paid model invocation.
- Historical usage remains queryable after trace data is deleted/replaced.
- Pricing versions/effective rates make calculated historical cost reproducible.
- Provider-reported and calculated cost are returned as separate fields/types.
- Queries can aggregate by issue/PR/repo/execution/agent-stage/model/provider/time window.
- A query for one pilot DevLoop returns the same stage/total usage as its producer records.
- No prompt/completion payload is required or stored for FinOps accounting.
- Unit/integration tests cover idempotency, price changes, unknown model pricing and partial/missing provider usage fields.
Non-goals
- Final UI/dashboard design.
- Hard execution-budget enforcement.
- Provider invoice replacement.
- Storing arbitrary tracing payloads.
- Choosing Traceway/Langfuse/Tempo/Phoenix as a dependency of the ledger.
Activity
mashkovd commentedon Sep 22, 2026
Dependency note: the
#50line here is invertedThis issue's Depends on section lists
mctlhq/.github#50. The canonicalgraph says the opposite, and the graph is right.
roadmap/epics/agent-finops.yaml:So
#50depends on this issue, not the reverse. Taking the prose literallywould make the two mutually blocking and neither could ever start.
It also reads correctly as an ordering:
#50instruments one DevLoop end-to-endto produce records, and there is nowhere to put them until this ledger exists.
A producer before its store is the wrong way round.
This issue's only real dependency is
usage-contract(.github#49), which isnow complete — delivered as ADR-012
(
docs/adr/012-model-usage-cost-attribution-contract.mdinmctl-agents,merged
ce50386).usage-ledgeris therefore ready in the live ready-set.What ADR-012 already settles, so it is not re-litigated here
ResultMessage.model_usageis keyedby model and one run legitimately spans a main loop plus a
cheap-profilesub-agent, so a per-run row merges them irrecoverably.
sha256(session_id | result_uuid | model_key)with anidempotent upsert, with the explicit carve-out that a retry which re-runs the
model is a new invocation with a real new charge and must not be deduped.
provider_reported_costis always null today under the subscriptiondecision; the SDK's
costUSDis a price-table calculation and belongs incalculated_costwith apricing_version.distinguishable from "nothing measured".
agent_executions,joined on
temporal_workflow_id— the spinerecord_executionalready writes.One thing the implementer should know before starting
There is no instrumentation to read from yet:
Six paths spend model quota and none captures usage; three of them never touch
ResultMessageat all. This issue can build the store, the pricing catalog andthe query surface against the contract, but its end-to-end proof needs
#50.That is the correct order — store first, producer second — and is why the
inverted prose above matters.
mashkovd commentedon Sep 22, 2026
Closed by
c963963(PR #344). Acceptance mapping against the merged code, since a closing keyword is not evidence.mainTestIngestIsIdempotent,TestUsageHandlers_IngestIsIdempotentOverHTTPPRIMARY KEY (id)+ON CONFLICT DO NOTHING;TestSuppliedIDCannotBypassTheDedupeKey,TestRetryThatRerunsTheModelIsNotDedupedTestRecordsRemainQueryableByCorrelationWithoutTraceDataTestStoredCostSurvivesAPriceChange,TestCatalogResolvesByInstantNotLatestTestCalculateDoesNotOverwriteProviderReportedCostGroupByset +Filter;TestSummaryAggregatesOverDimensions,TestSummaryByModelUsesTheModelKeyFallbackThe one criterion not fully met
"Ingested idempotently from the DevLoop pilot" is satisfied as the ingestion path is proven idempotent against pilot-shaped records — not as a live pilot emitting them. No producer exists yet:
grep -rn "total_cost_usd\|model_usage" orchestrator/inmctl-agentsstill returns nothing.That is not a gap I left; it is a consequence of the dependency direction. The canonical graph has
.github#50(devloop-instrumentation) depending on this item, not the reverse, so a live pilot could not exist before the ledger it writes to. The issue prose states the dependency backwards — noted in an earlier comment on this issue. Closing on that reading rather than silently treating the criterion as met.What a reader should know before building on this
idis derived server-side and never trusted from the request. A supplied id that disagrees is a400. A producer that could pick its own id would opt out of the guarantee entirely.*int64, and summary buckets stay nullable.SUM()ignores "not measured" and counts "reported zero". Do not make theseNOT NULL DEFAULT 0.USAGE_PRICING_CATALOG), not in the binary. With no catalog the ledger records every token count and derives no cost — deliberately, since inventing rates produces plausible-looking wrong money.503, never an empty200. Zero spend read from an unconfigured ledger is the opposite of the truth.Review: five rounds — 5 P2 + 9 P3 from claude and agy, all verified against the code before fixing. Final two rounds were 0 P1 / 0 P2 with agy
PASS. Residual P3s tracked separately.