Skip to content

feat: Display model + tokens in PR code reviews summary#407

Merged
alex-alecu merged 16 commits intomainfrom
feat/add-model-tokens-to-pr-review-summary
Feb 23, 2026
Merged

feat: Display model + tokens in PR code reviews summary#407
alex-alecu merged 16 commits intomainfrom
feat/add-model-tokens-to-pr-review-summary

Conversation

@alex-alecu
Copy link
Copy Markdown
Contributor

@alex-alecu alex-alecu commented Feb 20, 2026

We cannot include the total tokens in the first summary because we would exclude the tokens used to generate the summary* itself. Instead we manually add the footer after the summary is posted.

The costs for Code Reviewer are already visible in /usage-details:
Screenshot 2026-02-23 at 12 38 42

@alex-alecu alex-alecu requested a review from RSO February 20, 2026 15:24
@alex-alecu alex-alecu self-assigned this Feb 20, 2026
@kilo-code-bot
Copy link
Copy Markdown
Contributor

kilo-code-bot bot commented Feb 20, 2026

Code Review Summary

Status: 1 Issue Found | Recommendation: Merge with minor improvement

Overview

Severity Count
CRITICAL 0
WARNING 0
SUGGESTION 1
Issue Details (click to expand)

SUGGESTION

File Line Issue
src/app/api/internal/code-review-status/[reviewId]/route.ts 387 Missing captureException in catch block that now covers both reactions and usage footer updates
What Looks Good
  • Clean separation of concerns: dedicated /api/internal/code-review-usage endpoint for persisting usage data, separate from the status callback
  • Proper || 0?? 0 fix in the orchestrator for truthiness-safe zero handling
  • Usage accumulators correctly synced to DO persistent state (survives eviction)
  • appendUsageFooter handles idempotent replacement of existing footers with good edge-case coverage
  • Solid test coverage for usage-footer.ts (7 test cases covering provider prefix stripping, replacement, empty body, etc.)
  • DB migration adds nullable columns — safe for rolling deploys
  • Cost stored as microdollars (total_cost_musd) for consistency with existing microdollar_usage table
  • Retry/polling in getReviewUsageData handles the race between orchestrator usage report and cloud agent callback
  • resetCodeReviewForRetry correctly clears the new usage columns
Files Reviewed (14 files)
  • cloudflare-code-review-infra/src/code-review-orchestrator.ts - Usage accumulation, DO state persistence, reportUsage()
  • cloudflare-code-review-infra/src/types.ts - Type additions for usage fields
  • src/app/api/internal/code-review-status/[reviewId]/route.ts - Usage footer update logic (1 suggestion)
  • src/app/api/internal/code-review-usage/[reviewId]/route.ts - New usage persistence endpoint
  • src/db/migrations/0023_slippery_korg.sql - Migration adding usage columns
  • src/db/migrations/meta/0022_snapshot.json - Snapshot update
  • src/db/migrations/meta/0023_snapshot.json - New snapshot
  • src/db/migrations/meta/_journal.json - Journal update
  • src/db/schema.ts - Schema additions for usage columns
  • src/lib/code-reviews/db/code-reviews.ts - DB functions for usage updates
  • src/lib/code-reviews/summary/usage-footer.test.ts - New tests
  • src/lib/code-reviews/summary/usage-footer.ts - New usage footer builder
  • src/lib/integrations/platforms/github/adapter.ts - New updateKiloReviewComment function
  • src/lib/integrations/platforms/gitlab/adapter.ts - New updateKiloReviewNote function

Fix these issues in Kilo Cloud

…usage data

Use 3 retries with 200/400/800ms delays instead of a single 2-second
sleep. Reduces average latency when data is already present and provides
more attempts when it isn't.
Cover buildUsageFooter and appendUsageFooter including first-time
append, footer replacement with exact and variant whitespace, empty
body, unrelated horizontal rules, and multi-slash model slugs.
Sync model/token/cost accumulators to this.state on every LLM API
event so they are written to durable storage by the existing event
batch save. Restore them in loadState() so accumulation resumes
correctly after a DO restart.
@alex-alecu alex-alecu enabled auto-merge February 23, 2026 13:17
}
}
} catch (reactionError) {
} catch (postCompletionError) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SUGGESTION]: Missing captureException — this catch block now covers usage footer updates (API calls to find + update PR comments) in addition to reactions, but errors are only logged, not reported to Sentry.

Consider adding captureException(postCompletionError, { tags: { source: 'code-review-status-post-completion' } }) so failures in the expanded scope are visible in monitoring.

@alex-alecu alex-alecu merged commit 23430cf into main Feb 23, 2026
12 checks passed
@alex-alecu alex-alecu deleted the feat/add-model-tokens-to-pr-review-summary branch February 23, 2026 13:24
alex-alecu added a commit that referenced this pull request Mar 4, 2026
Even if this PR #407 was
implemented, it doesn't work because cloudflare code review worker is
not setup. It was probably deployed once locally.

I have tried a manual pipeline deploy and it failed
https://github.com/Kilo-Org/cloud/actions/workflows/deploy-code-review-worker.yml

I'll first fix the manual pipeline and then merge this automated CD.
alex-alecu added a commit that referenced this pull request Mar 10, 2026
…eviews

PR #407 added model/token tracking to the review summary, but it only
works for v1 (SSE) reviews. All reviews now use v2 (cloud-agent-next),
which skips SSE stream processing entirely, so usage is never reported.

Fix: when the code_reviews record has no usage data, query the billing
tables (microdollar_usage + metadata) by cli_session_id to get the
model, tokens, and cost. Back-fill the code_reviews record so future
reads skip the aggregation.
alex-alecu added a commit that referenced this pull request Mar 10, 2026
…iews (#978)

## Summary

[PR #407](#407) added model +
token info to the PR review summary comment (the "Reviewed by
claude-sonnet-4.6 · 12,345 tokens" footer). But it only works for v1
(SSE-based) reviews. All reviews now run on v2 (cloud-agent-next), which
is callback-based — it never streams SSE events, so usage data is never
collected and the `model`, `total_tokens_in`, `total_tokens_out` columns
stay null.

**The fix:** when the `code_reviews` record has no usage data, we query
the billing tables (`microdollar_usage` + `microdollar_usage_metadata`)
by `cli_session_id`. The billing system already tracks every LLM call
with model, tokens, and cost — we just aggregate it per session. We also
back-fill the `code_reviews` record so future reads don't repeat the
aggregation.

## Verification

- [x] `pnpm typecheck` — no new errors (pre-existing errors in kiloclaw
only)
- [x] `pnpm test usage-footer` — 10/10 pass
- [x] Read through the billing schema to confirm
`microdollar_usage_metadata.session_id` matches
`code_reviews.cli_session_id`

## Visual Changes

N/A

## Reviewer Notes

- The billing query groups by model and picks the one with the most
tokens (the primary review model). This handles sessions that use
multiple models (e.g. a cheaper model for sub-tasks).
- The back-fill write to `code_reviews` is fire-and-forget (`.catch()`)
— if it fails, the footer still shows correctly; we just won't cache the
result.
- Long-term, cloud-agent-next could include usage data in its
`ExecutionCallbackPayload`, but that's a bigger change. This fix works
today with no changes outside the Next.js app.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants