Skip to content

fix(compaction): reserve full output headroom #32896

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
dannyward630 wants to merge 1 commit into from
Closed

fix(compaction): reserve full output headroom #32896

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 1 addition & 4 deletions packages/opencode/src/session/overflow.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,12 @@ import type { Provider } from "@/provider/provider"
import { ProviderTransform } from "@/provider/transform"
import type { MessageV2 } from "./message-v2"

const COMPACTION_BUFFER = 20_000

export function usable(input: { cfg: ConfigV1.Info; model: Provider.Model; outputTokenMax?: number }) {
const context = input.model.limit.context
if (context === 0) return 0

const reserved =
input.cfg.compaction?.reserved ??
Math.min(COMPACTION_BUFFER, ProviderTransform.maxOutputTokens(input.model, input.outputTokenMax))
input.cfg.compaction?.reserved ?? ProviderTransform.maxOutputTokens(input.model, input.outputTokenMax)
return input.model.limit.input
? Math.max(0, input.model.limit.input - reserved)
: Math.max(0, context - ProviderTransform.maxOutputTokens(input.model, input.outputTokenMax))
Expand Down
15 changes: 7 additions & 8 deletions packages/opencode/test/session/compaction.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -454,11 +454,10 @@ describe("session.compaction.isOverflow", () => {
),
)

// ─── Bug reproduction tests ───────────────────────────────────────────
// These tests demonstrate that when limit.input is set, isOverflow()
// does not subtract any headroom for the next model response. This means
// compaction only triggers AFTER we've already consumed the full input
// budget, leaving zero room for the next API call's output tokens.
// ─── Output headroom regression tests ─────────────────────────────────
// When limit.input is set, isOverflow() still reserves headroom for the next
// model response. This keeps compaction from waiting until the full input
// budget is consumed, which would leave no room for the next API call's output.
//
// Compare: without limit.input, usable = context - output (reserves space).
// With limit.input, usable = limit.input (reserves nothing).
Expand All @@ -467,7 +466,7 @@ describe("session.compaction.isOverflow", () => {
// Open PRs: #6875, #12924

it.live(
"BUG: no headroom when limit.input is set — compaction should trigger near boundary but does not",
"reserves output headroom when limit.input is set",
provideTmpdirInstance(() =>
Effect.gen(function* () {
const compact = yield* SessionCompaction.Service
Expand All @@ -493,7 +492,7 @@ describe("session.compaction.isOverflow", () => {
)

it.live(
"BUG: without limit.input, same token count correctly triggers compaction",
"without limit.input, same token count correctly triggers compaction",
provideTmpdirInstance(() =>
Effect.gen(function* () {
const compact = yield* SessionCompaction.Service
Expand All @@ -513,7 +512,7 @@ describe("session.compaction.isOverflow", () => {
)

it.live(
"BUG: asymmetry — limit.input model allows 30K more usage before compaction than equivalent model without it",
"limit.input model agrees with equivalent context-limited model",
provideTmpdirInstance(() =>
Effect.gen(function* () {
const compact = yield* SessionCompaction.Service
Expand Down
Loading