-
Notifications
You must be signed in to change notification settings - Fork 24
Description
Problem
Code review summaries on PRs should display token usage and model info as a footer (e.g. Reviewed by claude-sonnet-4.6 · 13,579 tokens). This was implemented in PR #407 but only works for v1 (cloud-agent) reviews. When the code-review-cloud-agent-next feature flag is enabled, usage data is never reported and the footer never appears.
Root Cause
The v1 and v2 flows have fundamentally different architectures:
| Aspect | v1 (cloud-agent) | v2 (cloud-agent-next) |
|---|---|---|
| SSE stream processing | Yes — processEventStream() parses api_req_started events |
None — fire-and-forget HTTP call |
| Usage accumulation | Yes — orchestrator accumulates model, tokens, cost from events | None |
reportUsage() called |
Yes — in runWithCloudAgent() finally block |
No — missing from runWithCloudAgentNext() |
| Callback includes usage | No (just status) | No (just status) |
| Footer on PR | Works | Always skipped |
The orchestrator's runWithCloudAgentNext() (cloudflare-code-review-infra/src/code-review-orchestrator.ts) does not receive any streaming events, so it cannot accumulate usage data. The ExecutionCallbackPayload from cloud-agent-next (cloud-agent-next/src/callbacks/types.ts) also does not include usage fields.
Possible Approaches
Option A: Include usage in callback payload (recommended)
Cloud-agent-next already tracks per-message cost and tokens via AssistantMessage.info.cost and AssistantMessage.info.tokens. Add accumulated model, totalTokensIn, totalTokensOut, totalCost to the ExecutionCallbackPayload so it's delivered alongside the completion status.
Changes:
cloud-agent-next/src/callbacks/types.ts— add usage fields toExecutionCallbackPayloadcloud-agent-next/src/persistence/CloudAgentSession.ts— accumulate usage from messages when building callback payloadsrc/app/api/internal/code-review-status/[reviewId]/route.ts— extract usage from v2 callback and write to DB (bypass the polling ingetReviewUsageData)
Option B: Have cloud-agent-next call usage endpoint directly
Cloud-agent-next independently calls POST /api/internal/code-review-usage/{reviewId} with accumulated usage data before firing the completion callback.
Changes:
- Cloud-agent-next session completion logic — add HTTP call to usage endpoint
- Need to pass
reviewIdandINTERNAL_API_SECRETthrough to the session
Option C: Have orchestrator query cloud-agent-next for usage
After receiving the completion callback, the orchestrator queries cloud-agent-next for session usage stats, then calls reportUsage().
Changes:
- Cloud-agent-next — add a tRPC endpoint to return session usage stats
- Orchestrator — add usage query +
reportUsage()to v2 callback handling
Context
- Usage data fields:
model(string),total_tokens_in(int),total_tokens_out(int),total_cost_musd(int, microdollars) - DB columns already exist on
cloud_agent_code_reviewstable (migration 0023) updateCodeReviewUsage()insrc/lib/code-reviews/db/code-reviews.tshandles the DB writeappendUsageFooter()insrc/lib/code-reviews/summary/usage-footer.tshandles formatting- The status endpoint at
src/app/api/internal/code-review-status/[reviewId]/route.tsalready has the footer update logic — it just needs usage data to be available