Repository navigation
architecture(agent-finops): define canonical model-usage and cost attribution contract #49
Description
Parent roadmap
mctlhq/.github#48
Implementation owner
Primary implementation is expected in mctlhq/mctl-agents, but the current GitHub integration cannot create issues in that repository, so this cross-repo architecture child lives here until the contract is accepted/decomposed further.
Goal
Define the canonical, backend-neutral model usage and cost attribution contract for mctl agent executions before dashboards or budget enforcement are added.
The contract must make one model invocation attributable to the exact execution/work item/repository/issue/PR/agent stage that caused it, without requiring prompt/completion storage or depending on a specific observability backend.
Scope
Invocation-path inventory
Inventory all currently relevant model invocation paths in mctl-agents, including at minimum:
- investigator;
- implementer;
- shepherd/review-feedback;
- reviewer agents where mctl can observe/pay usage;
- any other Temporal/Argo agent worker paths that call a model.
For each path record provider/model, SDK/API entry point, usage fields exposed, cache/reasoning usage, provider-reported cost availability, retry/replay behavior, and current execution/trace correlation.
Canonical ModelUsageRecord
Define a versioned record with fields equivalent to:
id
timestamp
execution_id
workflow_id / run_id
work_item_id?
repository / issue / pr?
agent / devloop_stage
provider / model
usage:
input_tokens
output_tokens
cache_read_tokens?
cache_write_tokens?
reasoning_tokens?
provider_reported_cost?
calculated_cost?
pricing_version?
trace_id / span_id?
outcome
retry_attempt / provider_request_id?
Cost semantics
Keep these concepts distinct:
provider_reported_cost— exact value only when the provider/API reports it;calculated_cost— usage multiplied by a versioned mctl price catalog;invoice_reconciled_cost— optional future reconciliation with provider billing exports.
A calculated/list-price value must never be presented as exact billing.
OpenTelemetry mapping
Use current OTel GenAI semantic conventions where useful, but define an mctl-owned compatibility mapping/version so changes in external semantic conventions cannot silently corrupt historical analytics.
Idempotency / retries
Define deterministic dedupe semantics for provider retry, Temporal retry/replay, process restart, and telemetry/export reprocessing. The same paid model invocation must not be counted twice.
Deliverables
- Invocation-path inventory.
- Versioned
ModelUsageRecordcontract/schema. - Correlation contract covering execution/workflow/work-item/repo/issue/PR/agent/stage.
- Provider → normalized usage mapping table.
- Cost semantics and pricing-version rules.
- Idempotency/dedupe design.
- OTel compatibility mapping/versioning decision.
- Repository boundary recommendation for durable storage/query API.
Acceptance criteria
- Every in-scope model-call path can map to the same usage contract.
- Input/output/cache/reasoning usage remains separately representable when providers expose it.
- Provider-reported, calculated and reconciled costs cannot be confused by consumers.
- Historical calculated cost remains reproducible after price changes.
- Retry/replay semantics prevent double counting.
- Prompt/completion payloads are not required for attribution.
- The contract remains valid if Traceway/Langfuse/Tempo/Phoenix are replaced.
Non-goals
- Dashboards.
- Hard budget enforcement.
- Replacing provider billing/invoices.
- Selecting the final observability backend.
- Persisting prompt/completion content for FinOps.
Activity
mashkovd commentedon Sep 11, 2026
Decision 2026-09-11 (billing model): DevLoop stays on the Claude subscription OAuth token for now. The contract therefore ships with a subscription-quota mode: SDK-reported cost is recorded as
calculated(source = Claude Code price table, pricing version pinned),provider_reported_coststays null, and quota-window consumption is the billable dimension. Estimates are acceptable for the pilot; a move to API-key billing is not a precondition.invoice_reconciled_coststays defined but unused until that changes.mashkovd commentedon Sep 22, 2026
Closure evidence
Delivered as ADR-012 —
docs/adr/012-model-usage-cost-attribution-contract.mdin
mctlhq/mctl-agents, merged asce50386(PR mctlhq/mctl-agents#447).It lives there rather than here because this issue says primary implementation
is expected in
mctl-agents, and that repo already has the ADR convention(
docs/adr/, 005–011) thatusage-ledgerimplementers read. This issue wasthe cross-repo placeholder "until the contract is accepted".
Deliverables
ResultMessage, and model selectionModelUsageRecordschema_versionon the envelopeagent_executionsspineotel_mapping_versionmctl-api, sibling toagent_executionsAcceptance criteria
inventory also records the real cost: three of the six (
run_shepherd,run_incident_responder,run_mentor) never touchResultMessagetoday.schema, with
reasoning_tokensnullable and absent because SDK 0.2.136does not expose it. Absent-vs-zero is a contract rule (invariant 9), so
"nothing spent" and "nothing measured" stay distinguishable.
5 and 6. Per your 2026-09-11 subscription decision,
provider_reported_costis always null:
costUSDis a price table applied to tokens, and puttingit in the provider field would launder an estimate into a billing claim.
calculated_costis never recomputed; a price change makes new records undera new
pricing_version.sha256(session_id|result_uuid|model_key)+ idempotent upsert, invariants 3and 4. The contract keeps the distinction explicit: a retry that re-runs the
model is a new invocation with a real new charge and must not be deduped.
that shape.
versioned, and cost is deliberately not a span attribute: spans are
sampled, a ledger must not be, and a figure under-counted by exactly the
sampling rate would still look plausible.
Two findings worth carrying forward
grep -rn "total_cost_usd\|model_usage" --include="*.py" orchestrator/returns no matches.usage-ledgerstartsfrom zero instrumentation, not from data in the wrong shape.
model_usageis keyed bymodel, and a run legitimately spans a main loop plus a
cheap-profilesub-agent — a per-run record would merge them irrecoverably.
The SDK surface was read from the pinned wheel (sdist from
uv.lock, SHA-256checked against the lock) rather than the copy installed locally, which is
0.1.76 and has a different
ResultMessage. Reading the installed one wouldhave documented a contract for a version the repo does not use.
usage-ledgeris now unblocked.