Skip to content

Dashboard context-window bar reports cumulative input tokens, showing false red at >100% of the cap #412

Description

Summary

The dashboard's context-window bar reports cumulative input tokens against a point-in-time cap, so it can exceed 100% and turn red (#ef4444 + pulse) on agents that are nowhere near their context limit.

I hit this on a long-running workflow where an agent showed a solid red context bar at a reported 1,121,132 / 936,000 (120%). Its actual peak context was 561,285 tokens — 60% of the cap.

Root cause

AgentOutput.input_tokens is a billing figure: it sums the prompt tokens of every API call made during an agent execution. context_window_used reuses it as a context figure, which is a point-in-time measurement of a single call.

# src/conductor/engine/workflow.py:4812  (also :5885 for parallel agents)
"context_window_used": output.input_tokens,
"context_window_max": await self._get_context_window_for_agent(resolved_agent, output),

The two values are only equal when an agent execution makes exactly one API call. Two independent paths break that:

1. Copilot provider — parse recovery. On a JSON parse/schema failure the recovery prompt is sent to the same session and its usage is added to the running total:

# src/conductor/providers/copilot.py:1355-1358
# Accumulate usage from recovery calls
if recovery_response.input_tokens is not None:
    total_input_tokens = (total_input_tokens or 0) + recovery_response.input_tokens

The recovery call replays the whole conversation, so its prompt is ~the same size as the original. One recovery therefore roughly doubles the reported context usage. This summing is correct for cost — you are billed for both calls — it is just not a context measurement.

2. Claude / pydantic-ai provider — always. RunUsage aggregates across every request in the run, as usage.py itself notes ("Pydantic AI's RunUsage already aggregates per-request usage"). So input_tokens there is a run total, and context_window_used over-reports on any multi-request agent, no recovery needed.

Evidence

Copilot SDK, claude-sonnet-5, context_tier: long_context, max_prompt_tokens = 936000.

Conductor emitted for this agent:

agent_completed  input_tokens=1121132  context_window_used=1121132  context_window_max=936000

The SDK's own per-API-call ledger (~/.copilot/session-store.db, table assistant_usage_events) for that same session:

turn input_tokens cache_read_tokens finish_reason
22 (original final call) 559,847 556,984 stop
23 (parse-recovery call) 561,285 559,845 stop
sum 1,121,132

559,847 + 561,285 = 1,121,132 — exactly the value Conductor reported. True context usage was 561,285 / 936,000 = 60%.

Note also that a single API call can never exceed max_prompt_tokens, so any reported context_window_used > context_window_max is by definition an accounting artifact rather than a real context state.

Across 15 executions of this agent in one run, the correlation was exact:

executions parse recoveries reported context
9 0 95K–229K (10–24%, green)
6 1 each 593K–1.12M (63–120%, amber/red)

Impact

  • False red/amber context alarms that are indistinguishable from genuine context pressure, which is precisely the signal the bar exists to provide (feat: Add context window % visualization to web dashboard #56).
  • Systematically wrong for the Claude provider on any multi-request agent.
  • min(pct, 100) in the dashboard hides the overflow, so the value silently saturates rather than looking obviously broken.
  • Would produce false positives for any future enforcement built on this field (the get_max_prompt_tokens docstring anticipates such enforcement).

Suggested fix

Track the last observed prompt size separately from the billing total, rather than deriving one from the other.

  • Add a distinct field (e.g. AgentOutput.context_tokens / last_call_input_tokens) that holds the most recent API call's prompt size, and emit that as context_window_used. Leave input_tokens alone so cost accounting stays correct.
  • Copilot: usage_ref[0] in _send_and_wait is already the last assistant.usage value — capture it before the recovery loop sums it. The SDK also exposes UsageGetMetricsResult.last_call_input_tokens ("Input tokens from the most recent main-agent API call"), which is exactly this quantity.
  • Claude/pydantic-ai: take the last request's usage rather than the aggregated RunUsage.
  • Defensively, clamp or drop the bar when used > max, since that state is never physically real.

If a separate field is unattractive, the alternative is to omit context_window_used whenever more than one API call contributed, so the bar hides rather than misleads.

Environment

  • conductor-cli 0.1.27 (also verified present on main)
  • github-copilot-sdk 1.0.9
  • Provider: copilot, model claude-sonnet-5, context_tier: long_context

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions