bug(usage): Output Speed divides by total Elapsed time instead of generation time (Elapsed - TTFT) #478

Description

In the Usage -> Request Details table, the Output Speed metric is calculated as Output Tokens / Elapsed instead of Output Tokens / (Elapsed - TTFT).

Because Elapsed includes upstream queueing, prompt prefill (TTFT), and any transparent retry wait time, this severely underreports actual token generation throughput on large-context requests.

Observed Examples

• Request 1: Output: 102, TTFT: 10.4 s, Elapsed: 10.5 s
  • Displayed Output Speed: 9.7 t/s (102 / 10.5)
  • Actual Streaming Speed: ~1,020 t/s (102 / 0.1)
• Request 2: Output: 414, TTFT: 35.1 s, Elapsed: 38.9 s
  • Displayed Output Speed: 10.6 t/s (414 / 38.9)
  • Actual Streaming Speed: 108.9 t/s (414 / 3.8)
• Request 3: Output: 790, TTFT: 9.89 s, Elapsed: 13.7 s
  • Displayed Output Speed: 57.6 t/s (790 / 13.7)
  • Actual Streaming Speed: 207.3 t/s (790 / 3.81)

Expected Behavior

Calculate Output Speed as output_tokens / max(elapsed - ttft, epsilon) for streaming requests where TTFT > 0 and Elapsed > TTFT.