[Bug] In-flight prune is a no-op (postTokens == prePruneTokens) at 229k tokens → request exceeds server context → empty-response resume loop #346

Activity

1. ranxianglei commented on Aug 28, 2026

   

   ranxianglei

   on Aug 28, 2026

   OwnerAuthor

   More actions

   🤖 ework agent · vllm-qwen/qwen3.8-27b

   [bot] Acknowledged — investigating. The log signature (postTokens == prePruneTokens at 229k, nudged=false, lastCompaction: 0, content parked in prune.messages) points at one of: (a) the model context limit is unresolved on this path so all %-based thresholds (nudge 45/55%, compaction, GC) never fire, or (b) the prune map is populated but the replacement step is a no-op (e.g. everything is protected by preserveRecentMessages/protected tools). Pulling the state file analysis next; will report root cause + fix plan in a follow-up comment.
2. ranxianglei commented on Aug 28, 2026

   

   ranxianglei

   on Aug 28, 2026

   OwnerAuthor

   More actions

   🤖 ework agent · vllm-qwen/qwen3.8-27b

   [bot] Root cause identified (code-level, v1.14.25). Two structural defects compound in your per-message spawn+resume mode:

   1. The model context limit never reaches the messages-transform pipeline in this mode

   state.modelContextLimit is written only by the system.transform hook (lib/hooks.ts:100), which fires after messages.transform within each request. Every %-based safety net in ACP reads that value during messages.transform:

   • nudge thresholds (min 45% / max 55%) → resolve to undefined when the limit is unknown (lib/messages/inject/utils.ts:121)
   • emergency threshold (98%), batch cleanup (100%, lib/gc/merge.ts:201), emergency tool-output truncation (100%, lib/messages/truncate-tools.ts:34)

   In a long-lived process the limit is available from the 2nd LLM request onward (in memory). But with per-message spawn+resume each process handles exactly one user message: messages.transform runs with the limit unknown, then system.transform sets it in memory — and no save ever happens after that, so the value is never persisted (all saveSessionState calls occur during messages.transform, i.e. before the system hook writes it). The only rescue is the fire-and-forget init-time catalog hydration (index.ts:69, client.config.providers()), which degrades silently on failure (server not ready at init, or the model absent from the catalog — e.g. a custom sglang model with no limit.context declared in opencode.json). Net effect: limit unknown forever → overMinLimit/overMaxLimit always false → no anchor nudges, no emergency path, no in-flight reduction. That is exactly your nudged=false + postTokens == prePruneTokens at 229k.

   2. Even with the limit known, the 100% emergency threshold sits above the point where the request actually fails

   The request is rejected when context + system prompt + tool schemas + max_tokens > 262144 — for you that's ~229k context (~87% of the window). But emergency tool-output truncation only arms at 100% of the limit (majorGcThresholdPercent: "100%", lib/messages/truncate-tools.ts:39). The in-flight safety net is armed at a point the request never reaches.

   3. (Contributing) the growth nudge deadlocks at the wall

   getCurrentTokenUsage (lib/token-utils.ts:9) reports the last successful assistant message's prompt size. Once every request is rejected, no new token data lands, the reported size stalls at ~229k, growth ≈ 0, and the growth-based nudge — the only mechanism that doesn't depend on the limit — can never fire again.

   The compressions that did happen (totalPruneTokens 37k/230k) occurred while the session was still growing — consistent with the growth nudge working until the wall.

   Fix plan (PR incoming):

   1. Persist modelContextLimit (+ model identity) from the system.transform hook so the next spawned process loads it.
   2. Lazy catalog hydration on miss in messages.transform (the server is guaranteed up by then — unlike at init).
   3. Configurable fallback limit (compress.contextLimitFallback, default 128k) when the model declares none, plus a WARN log — the safety net stays armed.
   4. Overhead-aware emergency threshold: min(configured, limit − system prompt − output reserve) so truncation fires before the rejection wall, not at 100%.
   5. Loud ERROR log when an over-budget request can't be reduced in-flight. (The exit-0 empty response itself is opencode core behavior — the plugin can't set the exit code; worth reporting upstream to sst/opencode separately.)

   Will post the PR link here once it's up. If you can share one of the state files (~/.local/share/opencode/storage/plugin/acp/ses_7fb*.json) I can confirm the block/limit fields match this analysis.
3. ranxianglei commented on Aug 28, 2026

   

   ranxianglei

   on Aug 28, 2026

   OwnerAuthor

   More actions

   🤖 ework agent · vllm-qwen/qwen3.8-27b

   [bot] Root cause confirmed and fixed — PR: #349

   Why the safety net never engaged (all three from code, matching your logs):

   1. The system hook is the only writer of state.modelContextLimit, and it runs after messages.transform within a request — so on the single request a spawned process handles, every percentage threshold (80% max/min, 98% emergency, 100% GC) resolved against undefined → all consumers no-op.
   2. The system hook never persisted the limit it learned (saves only happen inside the messages-transform pipeline, before the same request's system hook) — so each fresh spawn started from undefined again. Learned and lost, every message.
   3. The init-time catalog seed is fire-and-forget and races server readiness in spawned processes; nothing retried.

   With the limit unknown, postTokens == prePruneTokens and nudged=false at every size are exactly what the code does — the prune itself is fine, it just never had a threshold to act on.

   The fix (5 parts):

   1. System hook persists modelContextLimit + model identity on change → spawned processes resume with the limit known.
   2. One-shot lazy catalog hydration during the request on a catalog miss (server is guaranteed up then).
   3. New compress.contextLimitFallback (default 128000, 0 disables) — bounds the conversation when the limit is genuinely unknown; drives nudges, emergency override, GC, and in-flight truncation. Real model limit always takes precedence.
   4. In-flight truncation is now overhead-aware: fires at min(gc threshold, limit − systemPromptTokens − 16384) instead of 100% of the window. Your production numbers (262144 window, ~17K system, 229479 tokens) now trigger truncation at 228760 — there's a test using the exact 229479 figure.
   5. If the post-transform context still exceeds the budget, ACP logs a loud ERROR ("ACP hard guard: …") with the budget breakdown.

   On "surface an error instead of exit 0": the empty response + exit 0 is opencode-core behavior (the plugin can't set the exit code or reject the request) — the hard-guard ERROR gives the orchestrator a greppable signal in the ACP daily log; I'd file the exit-0 thing upstream to opencode separately.

   Tests: 1049/1049 pass (+20 new); all 11 behavioral tests verified to fail with the fix reverted. Pre-existing #312 invalidation tests unchanged. Rollback: "contextLimitFallback": 0.

   Two minor findings while in here (not blocking):

   • CONFIGURATION.md documents maxContextLimit/minContextLimit defaults as 55%/45%, but the code defaults are 80%/80% — stale docs.
   • The repo is not prettier-clean at HEAD under its own .prettierrc (printWidth: 100 vs the ~80-col style; prettier --write . would touch 412 files) — I left formatting untouched to keep the diff reviewable.

   Still open: your state files would let me check one remaining question — with totalPruneTokens at 37k/230k, compress blocks did exist, so why didn't prune.messages entries actually remove covered messages in-flight? Happy to dig once I have a sample state file + the matching daily log.
4. ranxianglei

   added a commit that references this issue on Aug 28, 2026

   fix: address review findings for #346

   ...

   24626ae
5. ranxianglei commented on Aug 28, 2026

   

   ranxianglei

   on Aug 28, 2026

   OwnerAuthor

   More actions

   🤖 ework agent · vllm-qwen/qwen3.8-27b

   [bot] PR #349 is ready — dual-agent review complete, all findings addressed, full CI green (pr-validation, build, test on Node 22 + 24, e2e, build-artifact).

   • Branch: 2026-08-28_spawn-resume-context-limit (2 commits: 019616f fix + 24626ae review follow-ups)
   • Suite: 1050/1050 locally and in CI; all 11 behavioral tests verified to fail with the fix reverted
   • Review summary: fix: context-limit safety net for spawn+resume mode (#346) #349 (comment)

   Per project rules the merge itself is human-only — please merge #349 when you're ready (CI auto-publishes on a release branch; this one lands on master as a normal fix PR).

   Still open from my earlier comment: the state file + daily log would let me check why prune.messages entries (37k/230k totalPruneTokens) didn't actually remove covered messages in-flight in your production sessions — that's a separate question from the limit-unknown root cause, and I'd like to rule it out before calling this fully closed.
6. ranxianglei

   added 3 commits that reference this issue on Sep 10, 2026

   fix: context-limit safety net for spawn+resume mode (#346)

   ...

   efdaeca

   fix: address review findings for #346

   ...

   e5f01d6

   Merge pull request #349 from ranxianglei/2026-08-28_spawn-resume-cont…

   ...

   255502f
7. ranxianglei

   closed this as completedon Sep 10, 2026