[ranxianglei]

What

When an Anthropic-format request carries an anthropic-beta header with a larger-context beta (e.g. context-1m-2025-08-07), the client has negotiated a 1M-token upstream window. The proxy now sizes its own context window to match (1,000,000), overriding the model-table default (200K for claude).

Fixes #302 (reported in #292).

Why

The anthropic-beta header is forwarded upstream (so the provider correctly serves 1M), but bili's own window resolution missed it. The client-facing instance sized its window at the model-table 200K, so a 261K-token conversation read as 131% → EMERGENCY T1, when it was really only 26% of the 1M window the client negotiated. The inner instance (which saw the header) correctly sized 1M — the two instances disagreed.

How

• New anthropicBetaContextWindow(headers) (src/server.ts): parses the anthropic-beta header for context-Nm betas → N × 1,000,000 (largest wins). Generalizes to future larger-context betas; rejects malformed / non-context tokens.
• The beta is resolved at the top of the native-window cascade (above plugin/launcher/registry/table) because it is the most direct per-request evidence of the window the upstream will serve — the table/registry only list the standard window. It also clears the nativeFromFallback flag (authoritative, not a low-confidence fallback, so the effective-window floor must not apply).
• Resolved per-request (re-read every request): the header may appear/disappear between requests of the same session.

Because the window flows into reqConfig.modelContextLimit, this single change fixes all four surfaces the issue lists: compression thresholds, the EMERGENCY trigger, usage-percentage logs, and the window reported to clients/plugins (effectiveContextLimit).

Tests

• Unit (tests/model-windows.test.ts, +7): context-1m-2025-08-07 → 1,000,000; no header / no context beta → undefined (model default); mixed beta list; array header value; case/whitespace; context-Nm generalization + largest-wins; malformed rejection.
• e2e (tests/anthropic-beta-window.test.ts, +3):
  • With the beta → 261K input is 26% of 1M → no EMERGENCY nudge.
  • Without the beta → the same 261K input is 131% of 200K → EMERGENCY nudge fires (control, proves the scenario can fire).
  • Plugin effectiveContextLimit is 1M with the beta and drops to 200K when the header disappears on the next request of the same session (per-request resolution).
• Verified the e2e tests are meaningful: with the fix reverted, tests 1 and 3 fail and the control (test 2) still passes.

Pre-flight

• npm run typecheck — clean
• npm test — 701 pass, 0 fail
• npm run build — success

[github-actions]

📦 Built Package Artifact

Branch: 2026-08-28_anthropic-beta-1m-window (e3bddd4)

Option A — Install from npm PR tag (recommended)

npm install -g billion-context@pr-305

Each push to this PR publishes a new version under the pr-305 npm tag.

Option B — Download artifact

1. Download the artifact from the Actions run
2. Extract the tarball and install:

tar xzf billion-context-pr305.tgz
npm install -g package

---

This comment is automatically updated on each push.

[ranxianglei]

🤖 ework agent · vllm-qwen/qwen3.8-27b

[bot] Reviewed PR #305 (branch 2026-08-28_anthropic-beta-1m-window) against #302. Verdict: LGTM — ready to merge (merge is human-only).

Code review (src/server.ts)

• anthropicBetaContextWindow(headers): the regex /^context-(\d+)m\b/ is anchored + word-bounded, so it matches context-1m-… / context-2m-… and correctly rejects context-m-…, context-1x-…, and my-context-1m-…. Largest-wins, array-header join, and case/whitespace handling are all correct.
• Cascade placement is right: the beta sits at the top of the native-window cascade, but operator compress.modelContextLimit is still applied on top in resolveRequestConfig (src/compress-settings.ts:170), so the documented invariant "operator tuning outranks everything" holds.
• nativeFromFallback is cleared when the beta is present — correct, since the beta is authoritative (the client's own runtime negotiation), so FALLBACK_EFFECTIVE_WINDOW_FLOOR must not raise the window.
• Resolved per-request (reads req.headers every request), matching the requirement that the header may appear/disappear between requests of one session.
• The window cascade is the single resolution site in handle (src/server.ts:749-769); all four surfaces (compression thresholds, EMERGENCY trigger, usage logs, effectiveContextLimit) flow from reqConfig.modelContextLimit. The only other modelContextLimit writes are the downstream learned-limit (only lowers) and output-headroom reservation — both pre-existing. So "one change fixes all four surfaces" checks out.

Pre-flight (ran on the branch):

• npm run typecheck — clean
• npm test — 701 pass / 0 fail
• npm run build — success

Test meaningfulness (verified, not just claimed): I reverted the cascade change (kept the tests) → e2e test 1 (beta → no EMERGENCY) and test 3 (plugin effectiveContextLimit = 1M) both fail, while the control test 2 (no beta → EMERGENCY fires) still passes. Restored the fix → all 3 pass. So the tests genuinely exercise the fix, and the control proves the scenario can fire.

No changes needed. Please merge when ready: #305

[ranxianglei]

Model: glm-5.3

Reviewed (supervisor pass): anthropicBetaContextWindow parses context-Nm from the anthropic-beta header per request, takes the max, and slots ahead of plugin/launcher/registry/configured/table. nativeFromFallback correctly guarded. Tests cover the 200K-table vs 1M-beta case including the 261206-token scenario from #292. Good to merge.