Skip to content

[Bug] Silent context-overflow death on resume when model reports limit.context=0 (custom providers): uncalibrated thresholds → 400 → empty exit-0 loop #347

Closed
#433
#350
Closed
[Bug] Silent context-overflow death on resume when model reports limit.context=0 (custom providers): uncalibrated thresholds → 400 → empty exit-0 loop#347
#433
#350
Labels
bugSomething isn't working

Description

@ranxianglei
ranxianglei
opened on Aug 28, 2026

Summary

Sessions replayed against a custom OpenAI-compatible model that reports no context limit (limit.context = 0) can grow past the backend's actual window and then die on every resume with a 400 Bad Request that opencode swallows (exit 0, no output). The failure is deterministic and unrecoverable for that session id.

Reported from production in ranxianglei/billion-context#317 (2 sessions in 24h; full analysis there). This issue tracks the opencode-acp-side fixes.

Repro (observed in production)

  • opencode 1.14.46, plugin opencode-acp@latest (stable 1.14.19), model vllm-qwen/qwen3.8-27b defined in opencode.json without a limit, backend sglang with real window 262,144.
  • opencode's /config/providers reports "limit": {"context": 0, "output": 0} for the model (verified live).
  • Long-lived headless session, per-message resume (opencode run --session <ses>), compaction.auto: false.

Failure sequence (from ~/.local/share/opencode/log/2026-08-28T142830.log, session ses_7fb2e8b92c3eZ6yUJKdhkjVe1Z):

  1. state.modelContextLimit is undefined — lib/state/model-limits.ts record() drops limit <= 0, so the catalog never gets an entry, and hooks.ts:100 (if (state && input.model?.limit?.context)) never sets it. The persisted state file has no modelContextLimit key at all.
  2. All percentage thresholds resolve to undefined (parseLimitValue in lib/messages/inject/utils.ts: percent + undefined window → undefined): minContextLimit "45%", maxContextLimit "55%", emergencyThresholdPercent 98% → overMaxLimit/overMinLimit permanently false.
  3. Only the absolute growth nudge (50K from baseline) fires — advisory text only. Observed: nudge at 206,326 tokens; model voluntarily compressed just 37,178 tokens total (3 blocks); live replay grew to ~230K.
  4. Request: input 230,527 + max_tokens 32,000 (opencode's fallback for limit.output = 0; user's options.maxTokens: 16384 is not honored — it leaks into the body as a raw maxTokens key sglang ignores) = 262,527 > 262,144 → sglang: Requested token count exceeds the model's maximum context length of 262144 tokens (400, isRetryable: false).
  5. opencode logs session.processor error=Bad Request, publishes session.error, goes idle, exits 0 with no output. Daemon retries 3× (inputs 230,275 / 230,433 / 230,527 — grow ~150 tokens per retry as the nudge message is appended). Fresh session id works immediately.

Note: with max_tokens = 16,384 the same request would have fit (246,911 < 262,144) — the 32,000 fallback pushed it over by 383 tokens.

Impact

  • Any custom-provider deployment (local vLLM/sglang/LM Studio, etc.) where the model has no catalog entry: the percentage-based protection system is silently disabled. Sessions accumulate until replay + completion reserve exceeds the real window, then the session is permanently stuck (every resume 400s; only a new session id heals it).
  • No error surface: exit 0 + empty output. Operators see "agent stopped replying" with nothing in the logs except a buried AI_APICallError.
  • lastCompaction: 0 in state files is a red herring (it only tracks opencode native compaction, which is disabled via compaction.auto: false) — do not chase it.

Immediate workaround (no code)

  1. Declare the real window in opencode.json (schema supports per-model limit; both keys required):
"qwen3.8-27b": { "limit": { "context": 262144, "output": 16384 }, ... }
  1. Optional: absolute thresholds in acp.jsonc (numbers pass through parseLimitValue untouched, no model window needed):
{ "compress": { "maxContextLimit": 150000, "minContextLimit": 100000 } }

Both are partial: nudges stay advisory — a small local model can still under-compress.

Proposed fixes (opencode-acp)

  1. Loud failure instead of silent blindness (small, do first): when limit.context <= 0 / state.modelContextLimit === undefined persists across N requests, log a prominent one-time WARN per session: "model reports no context window; percentage thresholds (min/max/emergency) are disabled — set limit in opencode.json or absolute compress.maxContextLimit/minContextLimit". Today this state is invisible, which is why the blindness went unnoticed for a long-lived session.
  2. Request-side hard guard (the real fix; the "auto-fallback to a compacted rebuild" asked for in billion-context#317): in messages.transform, estimate the wire size (tokenization already exists for nudges) and if it exceeds safeBudget = knownWindow − completionReserve (knownWindow = state.modelContextLimit or absolute maxContextLimit; completionReserve = configured limit.output or a conservative default), deterministically prune-to-fit: drop/summarize the oldest compressible (non-protected) tool outputs until the estimate fits — independent of model cooperation. This converts a hard 400 into a degraded-but-working turn.
  3. Learn the real window from 400s (design note / blocked): sglang's error text states the exact limit (maximum context length of 262144); parsing it and writing state.modelContextLimit would self-heal calibration. Blocked today because opencode exposes no response-error hook to plugins (only experimental.chat.system.transform, experimental.chat.messages.transform, experimental.text.complete). Track upstream; if a hook lands, implement auto-recalibration.

Related upstream issue (opencode)

Filed separately against anomalyco/opencode: opencode run exits 0 with no output on provider 400 (should exit non-zero + print the error), and options.maxTokens is not honored for custom models (32,000 fallback used instead; the configured value leaks as a raw maxTokens body key).

Activity

  1. ranxianglei commented on Aug 28, 2026

    @ranxianglei
    ranxianglei
    on Aug 28, 2026
    OwnerAuthor

    🤖 ework agent · vllm-qwen/qwen3.8-27b

    [bot] On it. Plan for the opencode-acp side:

    1. WARN on uncalibrated window — one-time prominent log per session when state.modelContextLimit stays undefined after N message-transform runs (percentage thresholds silently disabled). Small, do first.
    2. Request-side hard guard (prune-to-fit) — in messages.transform, estimate wire tokens; if estimate > knownWindow − completionReserve (knownWindow = state.modelContextLimit or absolute maxContextLimit), deterministically drop oldest compressible (non-protected) tool outputs until it fits — independent of model cooperation. This is the real fix converting the 400 into a degraded-but-working turn.
    3. Learn window from 400s — blocked upstream (no response-error hook for plugins); will track and note in devlog.

    Working on 1 + 2 now; will reply with a PR link.

  2. ranxianglei commented on Aug 28, 2026

    @ranxianglei
    ranxianglei
    on Aug 28, 2026
    OwnerAuthor

    🤖 ework agent · vllm-qwen/qwen3.8-27b

    [bot] Fixed in PR #348 → #348

    Your root-cause analysis was exactly right — modelContextLimit stays undefined (catalog drops limit<=0 and the system hook guards on limit.context), so every percentage threshold silently no-ops and only the absolute growth nudge fires. I implemented your Fix 1 + Fix 2; Fix 3 stays blocked (no response-error hook for plugins).

    Fix 1 — make the blindness visible (lib/messages/uncalibrated-window.ts, new)
    trackUncalibratedWindow logs a one-time per-session WARN after 3 consecutive transforms with an unresolved window: "Model reports no context window — ACP percentage thresholds are disabled", with a hint to set limit in opencode.json or absolute compress.maxContextLimit/minContextLimit. Counter resets once a window resolves. Threshold of 3 rules out the first-request race (system.transform runs after messages.transform).

    Fix 2 — request-side hard guard (lib/messages/prune-to-fit.ts, new)
    pruneToFit deterministically clears the oldest compressible (non-protected) tool outputs when the estimated wire size exceeds safeBudget = knownWindow − overflowGuardReserve — independent of model cooperation. Converts the hard 400 into a degraded-but-working turn.

    • knownWindow = modelContextLimit (real window) → else absolute modelMaxLimits[provider/model] → else absolute maxContextLimit; undefined (guard off) when only a percent is set. A percent is a nudge threshold, not a window — using it as the window would massively over-prune, so it's deliberately excluded.
    • Estimate = O(1) provider-reported usage + 8192-token margin (covers the new user msg + nudges); precise tokenization only when there's no provider token data.
    • Skips the current turn, user messages, the recent-message protection zone, protected tools, and protected file paths (Bug 39 parity). Idempotent. Logs WARN on fit, ERROR if it clears everything but still exceeds the window.

    New config (validated + schema'd): compress.overflowGuard (bool, default true), compress.overflowGuardReserve (number, default 32768 — covers opencode's 32000 fallback for limit.output=0).

    One honest caveat: in your exact repro the model reported limit.context=0 and you had no absolute maxContextLimit, so knownWindow is still undefined and the guard can't fire yet — but now the WARN tells you precisely what to configure, and the moment you set limit in opencode.json (or an absolute maxContextLimit), the guard actively protects you. The guard does fully protect any custom provider that does declare its window.

    Verification: typecheck clean · build clean · 1054/1054 tests pass (25 new in tests/prune-to-fit.test.ts) · scripts/ci/check-pr.sh green. Devlog: devlog/2026-08-28_overflow-guard/ (REQ + DESIGN + WORKLOG).

    Waiting on dual-agent review + your merge (PR merges are human-only per project rules).

  3. ranxianglei
    added 2 commits that reference this issue on Aug 28, 2026

    fix: address review feedback — B1 estimator + test hardening (#347)

    fb0acdc

    fix: round-2 review — estimator N1/N2, ERROR dedup N3, test hardening (…

    addc530

  4. ranxianglei commented on Aug 28, 2026

    @ranxianglei
    ranxianglei
    on Aug 28, 2026
    OwnerAuthor

    🤖 ework agent · vllm-qwen/qwen3.8-27b

    [bot] Dual-agent review complete — all findings addressed. PR #348 updated (commit addc530).

    Review history (2 independent agents, per AGENTS.md §5.3):

    • Round 1 — REQUEST CHANGES. One blocking finding (B1) + test-hardening items. All fixed.
    • Round 2 — APPROVE, no blocking. Three non-blocking (N1–N3) + six NITs (T1–T6). All fixed.

    What round 2 found and I fixed:

    • N1 — after a compress, the estimator read the last assistant's provider usage, which still includes the range prune() is about to replace with a summary → the first post-compress turn could over-clear. Now uses a precise content count in that case.
    • N2 — the current user message (after the last assistant) wasn't counted, so a large paste into a near-full context could still 400. Now counted.
    • N3 — the "over budget, nothing clearable" ERROR logged on every transform while stuck. Now deduped (once per stuck episode, re-logs after the estimate drops back under budget).
    • T1–T6 — O(n) array copy → backward scan; corrected a misleading docstring; added a production-shape clearing test (tool output on an old assistant message); mocks now use the real time: { created } shape; added a non-completed-trailing-run test; tightened a loose >= 2 assertion to the exact count.

    Verification: typecheck clean · build clean · 1062/1062 tests pass (33 in tests/prune-to-fit.test.ts, +5 new) · N1/N2 mutation-verified (each test fails when its fix is reverted).

    The fix (recap): for models reporting limit.context = 0, ACP now (1) logs a one-time WARN that percentage thresholds are disabled, and (2) runs a request-side guard (pruneToFit) that deterministically clears the oldest non-protected tool outputs to keep the outgoing request within the known window — converting the hard 400 / exit-0 death loop into a degraded-but-working turn.

    No-code workaround (until this ships): declare the real window in opencode.json ("limit": {"context": 262144, "output": 16384}) or set absolute compress.maxContextLimit / minContextLimit in acp.jsonc.

    PR: #348

    Note: per project rules I can't merge PRs — that's a human-only step. It's ready for your merge whenever you're satisfied.

  5. ranxianglei
    added
    bugSomething isn't working
    on Sep 10, 2026
  6. ranxianglei
    added 2 commits that reference this issue on Sep 10, 2026

    fix: context budget guard + no-window warning (#347)

    ce0aca3

    Merge pull request #350 from ranxianglei/2026-08-28_context-budget-guard

    7c39c2b
  7. ranxianglei
    closed this as completedin #350on Sep 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions