QwenLM#5231) * feat(core,cli): workflow tool token budget + per-run UI surfacing (P5) P5 of the Dynamic Workflows port (QwenLM#4721): per-run output-token budget for the Workflow tool, wired through the orchestrator dispatch gate, WorkflowRunRegistry, BackgroundTasksDialog phase tree, and the /workflows slash command. Also introduces a one-time usage banner the first time a workflow runs in a session, gated by the skipWorkflowUsageWarning setting. Knobs: QWEN_CODE_MAX_TOKENS_PER_WORKFLOW=<int> env, per-run cap skipWorkflowUsageWarning: true setting, suppress banner Budget gate semantics: SOFT cap, not pre-commit reservation. Gate is checked at dispatch entry, so concurrent fan-out (parallel / pipeline) can overshoot by up to (concurrency_window - 1) x per_dispatch_tokens before the first overshoot dispatch throws WorkflowBudgetExceededError. Matches upstream Claude Code 2.1.168 semantics. Operators sizing the cap should subtract the overshoot margin. Implementation: - WorkflowBudgetImpl (workflow-budget.ts) + env resolver with HARD_MAX_TOKENS_CEILING=100M ceiling on the env override. - WorkflowBudgetExceededError carries runId / budgetTotal / spent. - countedDispatch budget gate + onTokens callback feeding budget.recordSpent from getExecutionSummary().outputTokens. - WorkflowOrchestratorEmitter.budgetUpdated event; fires after each successful dispatch, skipped on rejection and when budget is null. - WorkflowTask gains tokensSpent / tokenBudgetTotal / perPhaseTokens fields; WorkflowRunRegistry.onBudgetUpdated attributes deltas to currentPhase at fire time and re-emits statusChange. - WorkflowRunRegistry.shouldShowUsageWarning latch fires once per registry instance; survives reset(). - WorkflowTool wires WorkflowBudgetImpl.fromEnv, threads onTokens into createProductionDispatch, mirrors budget into the registry via the emitter, and prepends the usage banner on the SUCCESS path only. - WorkflowDetailBody + /workflows listing + live phase-tree render budget chip (tokens / cap) and per-phase token totals. Verification (270 + 4 + 4 = 272 core + 42 cli): - workflow-budget.test.ts (18) + workflow-orchestrator.test.ts (+8 P5 + budget-gate + budgetUpdated emitter) - workflow-run-registry.test.ts (+10 P5: budget fields, latch, per-phase attribution, no-op on terminal entries) - workflow.test.ts (+4 P5: banner appears once, suppressed by setting, failure-path latch unchanged, fail-then-success re-emits banner) - workflowsCommand.test.ts (+4 P5: row chip capped/uncapped, detail tokens/cap/per-phase chips) - BackgroundTasksDialog.test.tsx unchanged (32 still pass) - Real-LLM E2E (DashScope qwen3.7-plus): tmux session driving Workflow tool, banner verified in returnDisplay, /workflows shows tokens 0 / cap (no cap) on uncapped run, banner suppression on 2nd run confirmed (latch consumed exactly once). Self-review round 1 fixes: - "hard ceiling" docstring softened to "soft cap" with per_dispatch x concurrency_window overshoot bound documented; banner copy aligned ("soft cap" instead of "hard ceiling"). - Attempted failure-path banner reverted after coreToolScheduler inspection: createErrorResponse hard-codes resultDisplay = error.message whenever result.error is set, so a failure-path banner would have been invisible AND would have silently flipped the registry latch, causing the next successful run to skip the banner too. Failure path now does not touch the latch; failure-path test asserts the fail-then-success run still gets the banner. - skipWorkflowUsageWarning setting placement aligned with skipNextSpeakerCheck sibling under settings.model.*. - QWEN_CODE_MAX_TOKENS_PER_WORKFLOW=0 documented as "treated as unset" with explicit pointer to QWEN_CODE_DISABLE_WORKFLOWS=1 for the "no workflows at all" intent. Refs QwenLM#4721. * fix(core,cli): close P5 review round 1 — token tracking gaps + UI polish (PR QwenLM#5231) Addresses 4 Critical + 7 Suggestions from qwen-code-ci-bot's multi-agent review: Critical fixes (orchestrator core): - #1 (workflow-orchestrator.ts): schema-mode success path was missing the onTokens call entirely, so structured-output agents never recorded against the budget. Lifted the token report to a single `reportTokens` helper invoked once after `subagent.execute()` returns, BEFORE the schema/non-schema branch. Both fast-path and override-path dispatch now hit the same reporting site regardless of terminate mode. - #2 (workflow-orchestrator.ts): the entry budget gate in countedDispatch was bypassed by `parallel()` batches — all N thunks fire-check-queue in a single microtask burst with spent=0, so every queued dispatch passed the gate before any could record tokens. Added a SECOND gate inside the limiter.run callback so queued thunks observe budget mutations from already-completed in-flight dispatches at slot-acquire time, restoring the documented overshoot bound of (concurrency_window - 1) × per_dispatch_tokens (previously up to N × per_dispatch_tokens for a single `parallel()` of N items). - #3 (workflow-orchestrator.ts): CANCELLED / TIMEOUT / MAX_TURNS / ERROR terminations threw without recording tokens, so failed dispatches burned budget silently. Same `reportTokens` lift fixes this — tokens are now read before the terminate-mode check on both paths. - #4 (workflow-orchestrator.ts): added debugLogger.warn at both gate sites (entry + intra-limiter) for budget-rejected dispatches. Suggestion fixes: - #5 (workflow.ts): `resolveUsageBanner` JSDoc still said "Called from BOTH the success and failure paths" after the earlier failure-path revert. Corrected to "SUCCESS path only" with the scheduler-override rationale moved into the docstring. - #6 (workflowsCommand.ts, BackgroundTasksDialog.tsx): null-sentinel perPhaseTokens (tokens spent before the first phase() call) was attributed by the registry but never rendered. Detail view + phase tree now surface a "(no phase)" row when the null-key bucket has spend. - #7 (workflowsCommand.ts, BackgroundTasksDialog.tsx): use the existing `formatTokenCount` helper from `cli/ui/utils/formatters.ts` (the same surface statusLinePresets and TurnCard use) so token counts render as `1.5k / 10k` instead of raw integers. - #8 (workflow-run-registry.ts): `onBudgetUpdated` no longer fires `emitStatusChange` when neither tokensSpent nor tokenBudgetTotal changed. Production code fires `budgetUpdated` after every successful dispatch including zero-output-token ones; gating the emit avoids a no-op UI re-render burst on those. - #11 (workflow.ts): final returnDisplay JSON now includes the `tokens` block whenever any usage is reported OR a cap is set, aligned with `buildLivePhaseTreeDisplay` (was only included when spend > 0, inconsistent with the live render). Test additions: - workflow-budget.test.ts: unchanged (18). - workflow-orchestrator.test.ts: +6 R1 tests (parallel-batch overshoot regression for #2, GOAL+CANCELLED/MAX_TURNS/TIMEOUT/ERROR token recording for #3 via createProductionDispatch, schema-mode success token recording for #1, no-onTokens crash safety). Mock subagent extended with getExecutionSummary + nextOutputTokens to drive these. - workflow-run-registry.test.ts: +1 R1 test for #8 emit gating; rewrote the backwards/zero-delta test to the new monotonic-spent contract. - workflow.test.ts: +1 R1 test for #10 (capped banner shape — was untested; only the uncapped shape had coverage). - workflowsCommand.test.ts: +1 R1 test for #6 null-sentinel surfacing. Updated assertions for #7 formatTokenCount output (`1.5k/10kt`). Total: 282 core tests passing (+10 R1), 43 CLI tests passing (+1 R1), 0 lint, 0 typecheck for workflow-touching files. Real-LLM tmux + JSON E2E reconfirmed end-to-end (banner now says "soft cap", display payload shape unchanged, run registers + completes cleanly). #9 fold: the parallel-batch overshoot test serves as the regression guard for the intra-limiter gate fix in #2. #7 partial: workflowsCommand.ts and BackgroundTasksDialog.tsx are the only two `tokens` render sites in P5; both updated. Other token-bearing surfaces (statusLinePresets, TurnCard) already use the helper. PR: QwenLM#5231 * fix(core,cli): close P5 review round 2 — UI emit dedup + error tail + dialog coverage (PR QwenLM#5231) 3 real findings from qwen-code-ci-bot's round 2 review (the other 11 findings on the same review were already addressed by R1 commit 6c5de81 — the bot used a stale snapshot that did not include R1). Fixes: - #12 (workflow.ts): every dispatch completion produced TWO `safeEmitUpdate` calls — once in the `agentCompleted` handler, once in the `budgetUpdated` handler that fires right after. Over a 1000-agent workflow that's 2000 TUI redraws when 1000 suffices. Dropped the `safeEmitUpdate` call from the `agentCompleted` handler and kept it in `budgetUpdated`; the orchestrator fires the two events back-to-back, so the deferred render shows both updates atomically. Production `WorkflowTool.execute()` always wires `WorkflowBudgetImpl.fromEnv()`, so `budgetUpdated` always fires — test paths that omit budget use the injected dispatch shape and don't exercise this emitter wiring. - #14 (workflow-budget.ts): the WorkflowBudgetExceededError message carried an advisory tail — "Increase QWEN_CODE_MAX_TOKENS_PER_WORKFLOW or unset it to remove the cap" — that reaches the LLM via `tool_result`. The model could surface this to the user and effectively coach them to remove the operator-set budget policy. Trimmed to the factual portion only. Operators can still find the env knob via the `debugLogger.warn` at both gate sites that names `MAX_TOKENS_PER_WORKFLOW_ENV` verbatim. - #15 (BackgroundTasksDialog.test.tsx): WorkflowDetailBody had no rendering test coverage. Added 4 cases under a new R2 #15 describe: capped M/N chip with per-phase tally, uncapped plain-spent + zero- chip suppression, hidden chip when both spend and cap are zero/null, and null-sentinel `(no phase)` row. Declined / declined-with-counter-evidence: - #13 (workflow-budget.ts threat-model docstring): bot claimed the overshoot bound is off-by-one — `concurrency_window × per_dispatch` rather than `(concurrency_window - 1) × per_dispatch`. The latter is the correct tighter upper bound: when the gate first tips, the tipping dispatch's own tokens are already counted in `spent`, and only `concurrency_window - 1` other in-flight dispatches remain to add overshoot. The looser bound the bot suggests would mislead operators into oversized safety margins. Test count: 282 → 283 core (+1 R2 #14 negative assertion), 42 → 47 CLI (+5 R2 #15 + null-sentinel coverage). 0 lint, 0 typecheck for workflow-touching files. PR: QwenLM#5231 * fix(core): close P5 review round 3 — finally-bracket execute(), error-arm budgetUpdated, gate-before-count (PR QwenLM#5231) 3 fixes for round 3 review. wenshao (human maintainer) caught a real production-path token leak that R1's `R1 #3` test missed; bot also found that R2 #12 (UI emit dedup) left the error arm with zero re-renders. Critical fixes (orchestrator core): - #6 (wenshao): `reportTokens` was on the line AFTER `await subagent.execute(...)` at both dispatch sites (fast path :353 + override path :642) — NOT in a `finally`. `AgentHeadless.execute()` re-throws on real reasoning-loop failure (`agent-headless.ts:287-294`), so the production ERROR path skipped `reportTokens` entirely and the dispatch's burned tokens leaked. Wrapped `await subagent.execute()` in `try { ... } finally { reportTokens(...) }` at both sites. `getExecutionSummary()` is safe to read inside the throw path because `AgentHeadless.execute()`'s own outer `finally` finalizes stats before the throw propagates. R1's `R1 #3` test passed only because the mock execute() RETURNED with ERROR mode (the rare `createChat` early-return); the production reasoning-loop throw was untested. Test pattern: R3 #6 tests now mock `execute()` to THROW directly, asserting `onTokens` still fires for both fast path and override path (sibling-drift coverage). - #1 (bot): with R2 #12's UI-emit dedup, the error arm of `countedDispatch` fired `agentCompleted` (no `safeEmitUpdate`) and NEVER fired `budgetUpdated` — producing ZERO UI re-renders per failed dispatch. The registry's `tokensSpent` / `perPhaseTokens` also diverged from `budget.spent()` because the host counter advanced (via the reportTokens-in-finally above) while the registry never saw it. Error arm now also fires `emitter?.budgetUpdated?.()` with the post-throw spent + total. Updated R1's "does NOT fire on dispatch rejection" test to assert the new contract (DOES fire, with the cumulative spent) — that test only passed before because the mock dispatch threw without ever calling `budget.recordSpent`, masking the production behavior. - #7 (wenshao, suggestion → accepted): `agentCount += 1` ran BEFORE the budget gate. After budget exhaustion, every subsequent `agent()` call still incremented `agentCount`, eventually tripping the agent-cap and surfacing the WRONG terminal error (`Workflow exceeded the maximum of N agent() calls per run`) when the real cause was budget exhaustion. Moved the budget gate above the `agentCount += 1`. Also keeps `agentCount` and `agentsDispatched` (registry counter) counting the same set of calls. New test loops 1100 budget-rejected dispatches and asserts the script completes with budget errors only, never agent-cap errors. Declined (round 5 Suggestion bar): - bot #2 (rename `shouldShowUsageWarning` → `tryConsumeUsageWarning`): naming style, R5 → overthinking. - bot #3 (debugLogger on NaN drop in recordSpent): hostile-provider defensive hardening, R5 → overthinking. - bot #4 (debugLogger on negative delta in onBudgetUpdated): same. - bot #5 (triple emitStatusChange per dispatch): efficiency, R5 → overthinking; #1 fix kept the dispatched / completed / budget callback shape, and TUI emits are still 2 per dispatch (the middle one no longer fires safeEmitUpdate per R2 #12). Declined with counter-evidence: - (None this round.) Test count: 283 → 287 core (+4 R3 tests: throw-path fast/override, budgetUpdated on error, agentCount/gate ordering), 47 CLI unchanged. 0 lint, 0 typecheck for workflow-touching files. CI lint failure on this PR is pre-existing main breakage (shellcheck SC2295 in `.github/workflows/qwen-autofix.yml:598` introduced by commit a335f9c, unrelated to this PR's diff) — leaving alone. PR: QwenLM#5231

Release Failed for v0.21.2-nightly.20260501.fb518b72c on 2026-05-01 #9

Description

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions