Skip to content

Count tool-call argument streaming as generation time - #4

Merged
charlesnutter merged 3 commits into
mainfrom
tool-input-marks
Sep 25, 2026
Merged

charlesnutter merged 3 commits into
mainfrom
tool-input-marks

Conversation

@charlesnutter

Copy link
Copy Markdown
Owner

Count the time a model spends writing a tool call's arguments as generation time. Ships as 0.3.3.

Problem

Tool-call arguments (e.g. a file's contents for write) are generated tokens and OpenCode counts them in the step's output, but the stream marks only watched text and reasoning deltas. Tokens were divided by too short a window: a write step showed 3,672 tokens "at 162.9 tok/s" against the engine's 36.4 for the turn, and the Session average read 55.1. Affects OpenCode-derived figures (universal line, per-step rates, Session speed), not engine figures.

Fix

Mark session.tool.input.started and .ended (and .delta, kept in case it is forwarded later).

Measured (MTPLX, OpenCode 2.0.12)

  • session.tool.input.delta reaches the plugin 0 times per step, so deltas alone did not fix it (37.7 vs 32.4).
  • With start/end marks, each step's last mark lands 0–1 ms before step.streamed.
  • 8-step turn that wrote a file: Session 36.4 tok/s against the engine's 36.1.

Debug-only: a per-step stream window line compares the window's end with step.streamed.

@charlesnutter
charlesnutter merged commit cfbd8f8 into main Sep 25, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant