fix(opencode): honest context arithmetic for small and unreported model limits - #44898
fix(opencode): honest context arithmetic for small and unreported model limits#44898yanglinfang wants to merge 12 commits into
Conversation
…el limits Three fixes to the budget arithmetic that breaks small local models: - unset/zero limit.context disabled proactive compaction entirely; now a conservative 32k usable window applies, shrunk by a per-session cap learned from provider overflow rejections, with a one-time warning - the compaction reserve was min(20k, maxOutputTokens) — 36%% of a 56k local window; now proportional min(20k, max(2048, 15%% of the window)), with compaction.reserved config keeping absolute priority - unset limit.output was assumed to be 32k, so usable = context - 32000 reached 0 on any window under 32k and compaction re-triggered after every step; now the fallback is window-proportional (25%%, floor 1024) and each request additionally clamps its output budget to the headroom the estimated input actually leaves Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Thanks for your contribution! This PR doesn't have a linked issue. All PRs must reference an existing issue. Please:
See CONTRIBUTING.md for details. |
|
The following comment was made by an LLM, it may be inaccurate: Based on the search results, I found one related PR (excluding the current PR #44898 which appears in results): Related PR:
However, this is not a duplicate but rather a parent/predecessor PR as part of the same feature split. The current PR is a focused bug fix for context arithmetic that's part of a larger effort to support small/local models. No duplicate PRs found |
# Conflicts: # packages/opencode/test/provider/transform.test.ts
# Conflicts: # packages/opencode/test/provider/transform.test.ts
|
Closing this — recent opencode releases handle our small/local-model setup well enough in practice that carrying this isn't worth the review burden. Thanks for the time spent looking at it. |
Issue for this PR
Closes #45368
Part of the #41372 series (tracking PR #44242), but stands alone.
Type of change
What does this PR do?
Fixes the context arithmetic that makes opencode unusable with small local models:
limit.contextdisabled compaction entirely. Routers and local providers frequently report 0; sessions then ran straight into provider overflow errors. Now a conservative 32k usable window applies, shrunk further by a per-session cap learned from actual provider overflow rejections (logged once per session).limit.outputwas assumed to be 32k.usable = context − 32000reaches 0 on any window under 32k, so every completed step counts as overflow and compaction re-triggers after every single step. The fallback is now window-proportional (25%, floor 1024), and each request additionally clamps its output budget to the headroom the estimated input actually leaves, so a long history can't push a request over the window.min(20k, max(2048, 15% of window)).compaction.reservedconfig keeps absolute priority and windows ≥133k keep today's 20k unchanged.How did you verify your code works?
New unit suites for the reserve formula, usable-window derivation, learned caps, and the per-request clamp; the changed upstream expectation (
context: 0→ conservative window instead of never-overflow) is updated with both directions asserted. Fullpackages/opencodesuite green on currentdev.End-to-end A/B on the same machine, model (qwen3:4b-instruct via ollama), config (
context: 16384,output: 0), and coding task:Screenshots / recordings
Evidence panels for the A/B run to be attached below (no UI change in the code itself).
before and after fix
Checklist