Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions CONFIGURATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -449,6 +449,20 @@ Run `codex login` as usual; the OAuth token travels in the `Authorization` heade
"baseUrl": "http://localhost:8787/bili/https://api.anthropic.com"
```

**Claude Code** — set the `ANTHROPIC_BASE_URL` env var to the `/bili/` URL. (claude's undici fetch ignores `HTTPS_PROXY`, so the `/bili/` URL form is the only manual option — cert MITM cannot intercept it.)

```bash
export ANTHROPIC_BASE_URL="http://localhost:8787/bili/https://api.anthropic.com"
```

> **Auto-compact alignment (manual mode only).** The `bili claude` launcher automatically sets `CLAUDE_CODE_AUTO_COMPACT_WINDOW` to bili's effective window for your model, so claude's own auto-compact threshold lines up with bili's compression budget. In manual `/bili/` mode you must do this yourself — otherwise claude may run its own local auto-compact (a "summarize the conversation" turn) on a threshold that doesn't match bili's window. That is usually harmless (same session-id, so bili re-derives state from the truncation) but noisier than needed. Set it to bili's effective window for your model:
>
> ```bash
> export CLAUDE_CODE_AUTO_COMPACT_WINDOW=<bili effective window in tokens>
> ```
>
> claude clamps this value **down** to the window it perceives for the model (never up), so over-setting is safe. You can also set it persistently via claude's settings (`autoCompactWindow`).

**Other API-key clients** (Cursor / Aider / Continue …) — wherever the upstream URL is configured, prepend `http://localhost:8787/bili/`. Nothing else changes.

The `/bili/` prefix doubles as a **self-detection signal**: billion-context client extensions (billion-context-pi / opencode-acp) recognize it in their own baseUrl and self-disable, so you never get double compression.
Expand Down
14 changes: 14 additions & 0 deletions CONFIGURATION.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -449,6 +449,20 @@ openai_base_url = "http://localhost:8787/bili/https://chatgpt.com/backend-api/co
"baseUrl": "http://localhost:8787/bili/https://api.anthropic.com"
```

**Claude Code** —— 把 `ANTHROPIC_BASE_URL` 环境变量设成 `/bili/` URL。(claude 的 undici fetch 忽略 `HTTPS_PROXY`,所以 `/bili/` URL 形式是唯一的手动方式 —— 证书 MITM 拦不到它。)

```bash
export ANTHROPIC_BASE_URL="http://localhost:8787/bili/https://api.anthropic.com"
```

> **自动压缩对齐(仅手动模式)。** `bili claude` launcher 会自动把 `CLAUDE_CODE_AUTO_COMPACT_WINDOW` 设成 bili 对你模型的有效窗口,让 claude 自己的自动压缩阈值与 bili 的压缩预算对齐。手动 `/bili/` 模式下需要你自己设 —— 否则 claude 可能在与 bili 窗口不一致的阈值上跑它自己的本地自动压缩(一次“总结对话”轮次)。这通常无害(同一 session-id,bili 会从截断中重新推导状态),但比必要的更吵。把它设成 bili 对你模型的有效窗口:
>
> ```bash
> export CLAUDE_CODE_AUTO_COMPACT_WINDOW=<bili 有效窗口 token 数>
> ```
>
> claude 会把这个值**向下**钳制到它自己感知的模型窗口(不会向上),所以设大了是安全的。也可以持久化到 claude 的 settings(`autoCompactWindow`)里。

**其他 API-key 客户端(Cursor / Aider / Continue ……)** —— 只要配置了上游 URL,前面加 `http://localhost:8787/bili/` 就行,其他都不用改。

`/bili/` 前缀还是个**自检测信号**:billion-context 的客户端扩展(billion-context-pi / opencode-acp)能在自己的 baseUrl 里认出它并自禁用,避免双层压缩。
Expand Down
42 changes: 38 additions & 4 deletions src/client-config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,12 @@ import path from "node:path";

export interface ClaudeSettings {
anthropicBaseUrl?: string;
/** Model claude runs: settings `env.ANTHROPIC_MODEL` ?? top-level `model`. */
model?: string;
/** The user's explicit auto-compact window (settings `autoCompactWindow`
* or `env.CLAUDE_CODE_AUTO_COMPACT_WINDOW`) — when set, the launcher
* must NOT override it with its own budget injection (#321). */
autoCompactWindow?: number;
}

export interface ModelWindow {
Expand All @@ -28,6 +34,12 @@ export interface CodexProvider {
export interface CodexConfig {
modelProvider?: string;
openaiBaseUrl?: string;
/** Top-level `model` — the model codex runs (budget alignment, #321). */
model?: string;
/** Top-level `model_context_window` override (if set). */
contextWindow?: number;
/** Top-level `model_auto_compact_token_limit` override (if set). */
autoCompactLimit?: number;
/** Top-level `model` + `model_context_window` override pair (if set). */
modelWindows?: ModelWindow[];
providers: Record<string, CodexProvider>;
Expand Down Expand Up @@ -173,18 +185,35 @@ export function readClaudeSettings(homeDir: string, cwd: string, env: NodeJS.Pro
path.join(cwd, ".claude", "settings.json"),
];
let anthropicBaseUrl: string | undefined;
let model: string | undefined;
let autoCompactWindow: number | undefined;
for (const f of files) {
const obj = readJsonObject(f);
const settingsEnv = obj?.env;
if (settingsEnv && typeof settingsEnv === "object" && !Array.isArray(settingsEnv)) {
const v = (settingsEnv as Record<string, unknown>).ANTHROPIC_BASE_URL;
const e = settingsEnv as Record<string, unknown>;
const v = e.ANTHROPIC_BASE_URL;
if (nonEmpty(v)) anthropicBaseUrl = v;
// env-block values beat same-file top-level settings (claude applies
// the env block as real environment, which outranks settings).
const m = e.ANTHROPIC_MODEL;
if (nonEmpty(m)) model = String(m);
const acw = Number(e.CLAUDE_CODE_AUTO_COMPACT_WINDOW);
if (Number.isFinite(acw) && acw > 0) autoCompactWindow = acw;
}
const tm = obj?.model;
if (nonEmpty(tm) && model === undefined) model = String(tm);
const tacw = Number(obj?.autoCompactWindow);
if (Number.isFinite(tacw) && tacw > 0 && autoCompactWindow === undefined) autoCompactWindow = tacw;
}
// Honor a shell-exported ANTHROPIC_BASE_URL (claude's native override) so
// the launcher wraps the relay the user actually uses, not the default.
if (!anthropicBaseUrl && nonEmpty(env.ANTHROPIC_BASE_URL)) anthropicBaseUrl = env.ANTHROPIC_BASE_URL;
return anthropicBaseUrl ? { anthropicBaseUrl } : {};
return {
...(anthropicBaseUrl ? { anthropicBaseUrl } : {}),
...(model ? { model } : {}),
...(autoCompactWindow ? { autoCompactWindow } : {}),
};
}

/**
Expand All @@ -198,6 +227,7 @@ export function parseCodexToml(text: string): CodexConfig {
let curProvider: string | null = null;
let codexModel: string | undefined;
let codexContextWindow: number | undefined;
let codexAutoCompactLimit: number | undefined;
for (const rawLine of text.split(/\r?\n/)) {
const line = rawLine.trim();
if (!line || line.startsWith("#")) continue;
Expand All @@ -224,10 +254,14 @@ export function parseCodexToml(text: string): CodexConfig {
} else if (curProvider && key === "base_url") {
result.providers[curProvider].baseUrl = val;
}
} else if (numMatch && table === "" && numMatch[1] === "model_context_window") {
codexContextWindow = Number(numMatch[2]);
} else if (numMatch && table === "") {
if (numMatch[1] === "model_context_window") codexContextWindow = Number(numMatch[2]);
else if (numMatch[1] === "model_auto_compact_token_limit") codexAutoCompactLimit = Number(numMatch[2]);
}
}
if (codexModel) result.model = codexModel;
if (codexContextWindow) result.contextWindow = codexContextWindow;
if (codexAutoCompactLimit) result.autoCompactLimit = codexAutoCompactLimit;
const win = toModelWindow(codexModel, codexContextWindow);
if (win) result.modelWindows = [win];
return result;
Expand Down
131 changes: 130 additions & 1 deletion src/launcher.ts
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,9 @@ import { selfPackageRoot, isBiliPiEntry, ompPluginLoadedFrom } from "./plugin-in
function selfDistFile(name: string): string {
return path.join(selfPackageRoot(), "dist", name);
}
import { nonEmpty, resolvePiHome, resolveOmpHome, resolveHermesHome, resolveDshHome, loadClientConfig, collectModelWindows, type ClientConfig, resolveOpencodeConfigFile, type OpencodeConfig, type OpencodeProvider, type HermesConfig, type HermesProvider } from "./client-config.js";
import { nonEmpty, resolvePiHome, resolveOmpHome, resolveHermesHome, resolveDshHome, loadClientConfig, collectModelWindows, type ClientConfig, type CodexConfig, resolveOpencodeConfigFile, type OpencodeConfig, type OpencodeProvider, type HermesConfig, type HermesProvider } from "./client-config.js";
import { loadRoutes, resolveConfiguredContextLimit, lookupContextLimit, type ProviderRoutes } from "./config.js";
import { contextFromRegistry } from "./registry.js";

export {
type ClaudeSettings,
Expand Down Expand Up @@ -349,6 +351,108 @@ export function buildCodexArgs(
return args;
}

/**
* PR-D (#321): budget alignment for launcher-spawned agents. Resolves the
* context window bili will use as its compression denominator for `model` —
* the same chain the proxy applies, minus the per-request-only sources
* (anthropic-beta header, plugin report, launcher window):
* per-route per-model config declaration → built-in CONTEXT_LIMIT_TABLE
* → models.dev registry (snapshot-first, offline-safe).
*/
export async function resolveLauncherWindow(
model: string | undefined,
routes: ProviderRoutes,
upstreamUrl: string | undefined,
): Promise<number | undefined> {
if (!model) return undefined;
let host: string | undefined;
if (upstreamUrl) {
try {
host = new URL(upstreamUrl).hostname;
} catch {
host = undefined;
}
}
return (
resolveConfiguredContextLimit(routes, upstreamUrl, model) ??
lookupContextLimit(model) ??
(await contextFromRegistry(model, host))
);
}

/**
* PR-D (#321): codex's auto-compact budget is keyed off the window CODEX
* believes the model has (its bundled model table — bili has no say in it),
* while bili's ACP compression is keyed off bili's own window resolution.
* Two uncoordinated budgets (#292): when bili's window exceeds codex's
* perception, codex's ledger (server-reported usage, which bili sees) crosses
* codex's ~90% threshold first and fires its native compaction ahead of ACP.
*
* Injecting `-c model_context_window=<W> -c model_auto_compact_token_limit=<W>`
* (W = bili's effective window) makes codex's auto-compact threshold 90%×W —
* ACP (≈55%×W) always fires first, and codex's LOCAL compaction (benign for
* bili: same-session truncation the kernel deactivates by message id) only
* backstops when ACP fails, before codex's 95% hard cap. codex clamps both
* values to its own max_context_window, so an over-generous W degrades to
* codex's own perception instead of overshooting.
*
* Returns [] (no injection) when:
* - no model is configured (nothing to resolve a window for),
* - the user already set `model_context_window` in codex's config.toml
* (bili's proxy uses exactly that value as its launcher window — the
* budget is already aligned by the user's own declaration),
* - bili resolves no window for the model (no authoritative value to inject).
* A user-set `model_auto_compact_token_limit` is honored (not overridden).
*/
/** The base URL codex will actually call: the selected provider's
* `base_url`, else top-level `openai_base_url`, else codex's built-in
* OpenAI default (model-provider-info: `https://api.openai.com/v1`). */
export function codexUpstreamUrl(codex: CodexConfig | undefined): string {
const provider = codex?.modelProvider ? codex?.providers?.[codex.modelProvider]?.baseUrl : undefined;
return provider ?? codex?.openaiBaseUrl ?? "https://api.openai.com/v1";
}

export async function resolveCodexBudgetArgs(opts: {
model: string | undefined;
clientWindow: number | undefined;
clientAutoCompactLimit: number | undefined;
routes: ProviderRoutes;
upstreamUrl: string | undefined;
}): Promise<string[]> {
const { model, clientWindow, clientAutoCompactLimit, routes, upstreamUrl } = opts;
if (!model || clientWindow) return [];
const window = await resolveLauncherWindow(model, routes, upstreamUrl);
if (!window) return [];
const limit = clientAutoCompactLimit ?? window;
return ["-c", `model_context_window=${window}`, "-c", `model_auto_compact_token_limit=${limit}`];
}

/**
* PR-D (#321): claude-code's auto-compact window is a single env knob —
* `CLAUDE_CODE_AUTO_COMPACT_WINDOW` outranks settings and is clamped DOWN to
* the model window claude itself perceives (never up), so injecting bili's
* window is always safe: it tightens claude's threshold to bili's budget when
* bili's window is smaller, and is a no-op when it is larger.
*
* Returns {} (no injection) when: no model resolvable, the user already set
* an explicit auto-compact window (settings `autoCompactWindow` or
* `env.CLAUDE_CODE_AUTO_COMPACT_WINDOW`, or a shell-exported env var), or
* bili resolves no window for the model.
*/
export async function resolveClaudeBudgetEnv(opts: {
model: string | undefined;
userAutoCompactWindow: number | undefined;
shellAutoCompactWindow: string | undefined;
routes: ProviderRoutes;
upstreamUrl: string | undefined;
}): Promise<NodeJS.ProcessEnv> {
const { model, userAutoCompactWindow, shellAutoCompactWindow, routes, upstreamUrl } = opts;
if (!model || userAutoCompactWindow !== undefined || nonEmpty(shellAutoCompactWindow)) return {};
const window = await resolveLauncherWindow(model, routes, upstreamUrl);
if (!window) return {};
return { CLAUDE_CODE_AUTO_COMPACT_WINDOW: String(window) };
}

export function buildClaudeEnv(
origin: string,
caPath: string,
Expand Down Expand Up @@ -1278,6 +1382,9 @@ export async function runLaunch(params: RunLaunchParams, deps: LauncherDeps = {}
const config = loadClientConfig(process.env, process.cwd());
const base = baseClientName(params.client);
const routes = discoverRoutes(base, config);
// bili's own route graph (same sources the spawned proxy child reads —
// used to resolve the budget-alignment window, #321).
const biliRoutes = loadRoutes(process.env);
const domains = dedupeInOrder([...routes.httpsDomains, ...(params.mitmDomains ?? [])]);
const handle = await ensureProxyRunning({ host, port, passthrough, debug, mitmDomains: domains, modelWindows: collectModelWindows(config) }, deps);
console.error(
Expand Down Expand Up @@ -1436,13 +1543,35 @@ export async function runLaunch(params: RunLaunchParams, deps: LauncherDeps = {}
} else {
env = buildCodexEnv(origin, resolveCombinedCaPath(process.env), process.env);
clientArgs = buildCodexArgs(origin, routes.httpRewrites, routes.httpsRewrites, clientArgs);
const budgetArgs = await resolveCodexBudgetArgs({
model: config.codex?.model,
clientWindow: config.codex?.contextWindow,
clientAutoCompactLimit: config.codex?.autoCompactLimit,
routes: biliRoutes,
upstreamUrl: codexUpstreamUrl(config.codex),
});
if (budgetArgs.length > 0) {
clientArgs = [...budgetArgs, ...clientArgs];
console.error(`bili: codex budget aligned — ${budgetArgs.slice(2).join(", ")} (model: ${config.codex?.model})`);
}
if (injectMcp && codexConversationId) clientArgs = [...buildCodexMcpArgs(origin, codexConversationId), ...clientArgs];
}
} else {
env = directUrl
? buildClaudePluginEnv(origin, true, process.env)
: buildClaudeEnv(origin, ca, routes.httpRewrites, routes.httpsRewrites, process.env);
if (directUrl) env.BILLION_CONTEXT_PROXY = origin;
const claudeBudget = await resolveClaudeBudgetEnv({
model: nonEmpty(process.env.ANTHROPIC_MODEL) ? process.env.ANTHROPIC_MODEL : config.claude?.model,
userAutoCompactWindow: config.claude?.autoCompactWindow,
shellAutoCompactWindow: process.env.CLAUDE_CODE_AUTO_COMPACT_WINDOW,
routes: biliRoutes,
upstreamUrl: config.claude?.anthropicBaseUrl ?? "https://api.anthropic.com",
});
Object.assign(env, claudeBudget);
if (claudeBudget.CLAUDE_CODE_AUTO_COMPACT_WINDOW !== undefined) {
console.error(`bili: claude budget aligned — CLAUDE_CODE_AUTO_COMPACT_WINDOW=${claudeBudget.CLAUDE_CODE_AUTO_COMPACT_WINDOW}`);
}
if (injectMcp) {
const mcpFile = path.join(os.tmpdir(), `bili-mcp-${Date.now()}.json`);
fs.writeFileSync(mcpFile, JSON.stringify(buildMcpConfig(origin)));
Expand Down
Loading
Loading