Skip to content

feat: Add token usage + model reporting for cloud-agent-next (v2) code reviews #499

@alex-alecu

Description

@alex-alecu

Problem

Code review summaries on PRs should display token usage and model info as a footer (e.g. Reviewed by claude-sonnet-4.6 · 13,579 tokens). This was implemented in PR #407 but only works for v1 (cloud-agent) reviews. When the code-review-cloud-agent-next feature flag is enabled, usage data is never reported and the footer never appears.

Root Cause

The v1 and v2 flows have fundamentally different architectures:

Aspect v1 (cloud-agent) v2 (cloud-agent-next)
SSE stream processing Yes — processEventStream() parses api_req_started events None — fire-and-forget HTTP call
Usage accumulation Yes — orchestrator accumulates model, tokens, cost from events None
reportUsage() called Yes — in runWithCloudAgent() finally block No — missing from runWithCloudAgentNext()
Callback includes usage No (just status) No (just status)
Footer on PR Works Always skipped

The orchestrator's runWithCloudAgentNext() (cloudflare-code-review-infra/src/code-review-orchestrator.ts) does not receive any streaming events, so it cannot accumulate usage data. The ExecutionCallbackPayload from cloud-agent-next (cloud-agent-next/src/callbacks/types.ts) also does not include usage fields.

Possible Approaches

Option A: Include usage in callback payload (recommended)

Cloud-agent-next already tracks per-message cost and tokens via AssistantMessage.info.cost and AssistantMessage.info.tokens. Add accumulated model, totalTokensIn, totalTokensOut, totalCost to the ExecutionCallbackPayload so it's delivered alongside the completion status.

Changes:

  • cloud-agent-next/src/callbacks/types.ts — add usage fields to ExecutionCallbackPayload
  • cloud-agent-next/src/persistence/CloudAgentSession.ts — accumulate usage from messages when building callback payload
  • src/app/api/internal/code-review-status/[reviewId]/route.ts — extract usage from v2 callback and write to DB (bypass the polling in getReviewUsageData)

Option B: Have cloud-agent-next call usage endpoint directly

Cloud-agent-next independently calls POST /api/internal/code-review-usage/{reviewId} with accumulated usage data before firing the completion callback.

Changes:

  • Cloud-agent-next session completion logic — add HTTP call to usage endpoint
  • Need to pass reviewId and INTERNAL_API_SECRET through to the session

Option C: Have orchestrator query cloud-agent-next for usage

After receiving the completion callback, the orchestrator queries cloud-agent-next for session usage stats, then calls reportUsage().

Changes:

  • Cloud-agent-next — add a tRPC endpoint to return session usage stats
  • Orchestrator — add usage query + reportUsage() to v2 callback handling

Context

  • Usage data fields: model (string), total_tokens_in (int), total_tokens_out (int), total_cost_musd (int, microdollars)
  • DB columns already exist on cloud_agent_code_reviews table (migration 0023)
  • updateCodeReviewUsage() in src/lib/code-reviews/db/code-reviews.ts handles the DB write
  • appendUsageFooter() in src/lib/code-reviews/summary/usage-footer.ts handles formatting
  • The status endpoint at src/app/api/internal/code-review-status/[reviewId]/route.ts already has the footer update logic — it just needs usage data to be available

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestkilo-auto-fixAuto-generated label by Kilokilo-triagedAuto-generated label by Kilo

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions