Skip to content

epic: standardize AMS coding-agent credentials + close the token-usage observability gap (ORB parity) #6844

Description

@JSONbored

Problem

AMS's coding-agent credential model and observability don't match ORB's, in ways that surfaced directly during #6205's live end-to-end verification:

  1. Credential model mismatch. ORB (a Cloudflare Worker) authenticates its AI calls with a plain ANTHROPIC_API_KEY Worker secret — no interactive step, no personal-account tie-in. AMS's claude-cli driver already accepts ANTHROPIC_API_KEY as a valid credential (packages/loopover-miner/lib/status.js:379), but nothing surfaces this as the recommended path for an unattended/server deployment — init --interactive's wizard offers only "Authorize with GitHub"-style device-flow (claude setup-token, tied to a personal Claude.ai/Claude Code subscription) as its first option, with a pasted-PAT flow as the fallback. Setting up AMS on a real server during Verify + document AMS's standalone usage end-to-end against a real, non-gittensor GitHub repo #6205's verification meant going through a full interactive device-flow dance (visit a URL, paste back a code) for something that could have been a single ANTHROPIC_API_KEY env var, matching how ORB is already configured on the same class of infrastructure.
  2. Token-usage observability gap. ORB's ai_usage_events table (src/db/schema.ts:1289) records real, granular input_tokens/output_tokens per AI call, feeding the "AI usage" Grafana dashboard. AMS has the equivalent dashboard infrastructure already built (grafana/dashboards/miner-usage.json's "Total tokens" panel, attempt-log.sqlite3's tokens_used column) but the column is always NULL — no coding-agent driver currently reports real token counts back into it, so the panel silently shows 0 regardless of real usage. This is misleading, not just incomplete: it looks like a working metric.

Notably, Claude Code's own --output-format json response already includes a full usage object (input_tokens, output_tokens, cache_creation_input_tokens, cache_read_input_tokens, per-model breakdowns) — confirmed directly during #6205's manual driver reproduction. The data is available at the source; it just isn't being read and persisted.

Proposal

Bring AMS's coding-agent credential handling and usage observability to parity with ORB's:

  • Make ANTHROPIC_API_KEY a first-class, explicitly recommended path for unattended/fleet-mode deployments (mirroring ORB's own credential model), while keeping the Claude.ai-subscription OAuth path available and clearly scoped to interactive/laptop use where an operator already has Claude Code usage included in a subscription plan. Cover the equivalent question for codex-cli.
  • Parse and persist real per-attempt token usage from each driver's own reported data into attempt_log_events.tokens_used, closing the gap that currently makes the "Total tokens" panel always read zero.

Deliverables

Tracked via the sub-issues below.

Boundaries

  • This is about AMS's own credential/observability plumbing, not about changing ORB or its Worker secrets in any way.
  • Not scoped to unify AMS's local SQLite ledgers with ORB's D1-backed ai_usage_events table into one shared store — they're different deployment models (local CLI vs. Worker+D1); parity here means comparable behavior and dashboard fidelity, not a shared table.

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.

Projects

Status
Done

Relationships

None yet

Development

No branches or pull requests

Issue actions