fix: compat.roles wire role rewrite for role-rejecting upstreams (#552)

CONFIGURATION.md

@@ -107,6 +107,14 @@ Top-level keys that control how the proxy listens and behaves globally.
 - **Status:** ACTIVE
 - **Description:** Forward every request to the upstream **without** compression, tool injection, or nudging. Equivalent to `ACP_PASSTHROUGH=1`. Handy for A/B comparison against the uncompressed baseline.
 
+### `compat`
+
+- **Type:** `{ roles?: Record<string, string> }`
+- **Default:** `{}` (disabled)
+- **Status:** ACTIVE
+- **Description:** Global wire-compat role map. `roles` maps message roles to the role name your upstream accepts, e.g. `{"compat":{"roles":{"developer":"system"}}}` rewrites `developer` → `system` on the final forwarded body for upstreams that reject the `developer` role (#552, newer codex clients). Applies to `openai` chat-completions and `responses` requests; exact-match roles only, everything else in the body is untouched; re-sent compress-retry bodies carry the same rewrite. Per-provider `compat.roles` entries (see [Providers](#providers)) win per key. Default `{}` forwards bodies byte-for-byte unchanged.
+- **Learn-on-failure:** with no compat configured, an upstream `400 Invalid role: …` is auto-fixed — bili rewrites the offending role to `system`, retries once, and remembers the mapping **session-scoped** (in-memory on the session; never written to config). Later requests in that session skip the 400 round-trip. The info log emitted when the fix fires carries the permanent per-provider snippet.
+
 ### `proxy`
 
 - **Type:** `string`
@@ -171,6 +179,13 @@ A shallow key (`https://open.bigmodel.cn`) matches every path on that host. A de
 - **Status:** ACTIVE
 - **Description:** Per-provider compression overrides. This is **level 2 of 3** in the merge hierarchy — see [Compression Tuning](#compression-tuning).
 
+### `compat`
+
+- **Type:** `{ roles?: Record<string, string> }`
+- **Default:** `{}` (disabled)
+- **Status:** ACTIVE
+- **Description:** Per-provider wire-compat overrides. `roles` maps message roles to the role name this upstream accepts, e.g. `{"developer": "system"}` for upstreams that reject the `developer` role newer codex clients send (#552). Applied to the final forwarded `openai`/`responses` body — client-sent roles and bili's own injected prompt alike — and to every body the compress-retry loops re-send. Wins per key over the global `compat` block (see [Server Settings](#server-settings)). Default `{}` forwards byte-for-byte unchanged.
+
 ---
 
 ## Compression Tuning

CONFIGURATION.zh-CN.md

@@ -107,6 +107,14 @@
 - **状态：** ACTIVE
 - **说明：** 将每个请求**不经过**压缩、工具注入或 nudge，直接转发到上游。等价于 `ACP_PASSTHROUGH=1`。便于与未压缩基线做 A/B 对比。
 
+### `compat`
+
+- **类型：** `{ roles?: Record<string, string> }`
+- **默认值：** `{}`（禁用）
+- **状态：** ACTIVE
+- **说明：** 全局线上兼容角色映射。`roles` 把消息角色映射为上游接受的角色名，例如 `{"compat":{"roles":{"developer":"system"}}}` 把 `developer` → `system`，用于拒绝 `developer` 角色的上游（#552，新版 codex 客户端会发送）。作用于 `openai` chat-completions 与 `responses` 请求；仅精确匹配角色，体内其它内容不动；压缩重试重发的请求体同样携带。按 provider 的 `compat.roles`（见 [Providers](#providers)）按键优先。默认 `{}` 逐字节透明转发。
+- **失败自学习：** 未配置 compat 时，上游返回 `400 Invalid role: …` 会被自动修复 —— bili 把被拒角色改写为 `system`，重试一次，并把学到的映射记在**会话上**（仅内存，绝不写入配置）。该会话后续请求免 400 往返。修复生效时打印的 info 日志附带可永久化的 per-provider 片段。
+
 ### `proxy`
 
 - **类型：** `string`
@@ -169,6 +177,13 @@
 - **状态：** ACTIVE
 - **说明：** 按 provider 的压缩覆盖项。这是三层合并中的**第 2 层** —— 见[压缩调优](#压缩调优)。
 
+### `compat`
+
+- **类型：** `{ roles?: Record<string, string> }`
+- **默认值：** `{}`（禁用）
+- **状态：** ACTIVE
+- **说明：** 按 provider 的线上兼容覆盖。`roles` 把消息角色映射为该上游接受的角色名，例如 `{"developer": "system"}` —— 用于拒绝 `developer` 角色的上游（#552，新版 codex 客户端会发这个角色）。作用于最终转发的 `openai`/`responses` 请求体 —— 客户端发送的角色和 bili 自己注入的提示一视同仁 —— 压缩重试循环重发的请求体同样携带该改写。按键覆盖全局 `compat` 块（见[服务端设置](#服务端设置)）。默认 `{}` 逐字节透明转发。
+
 ---
 
 ## 压缩调优

README.md

@@ -355,6 +355,31 @@ So you can give ZCode its own proxy without affecting API-key clients:
 }
 ```
 
+### Wire-compat role rewrite (`compat.roles`)
+
+Some upstreams reject the `developer` role newer codex clients send on the
+Responses API (`400 Invalid role: developer`). `compat.roles` maps roles to
+what the upstream accepts — applied at the forward boundary to the final
+`openai`/`responses` body (client-sent roles **and** bili's own injected
+prompt alike), global or per-provider, default off = byte-for-byte:
+
+```jsonc
+{
+  "compat": { "roles": { "developer": "system" } },
+  "providers": {
+    "https://picky.example.com": { "compat": { "roles": { "developer": "user" } } }
+  }
+}
+```
+
+**No configuration needed for the common case.** When an upstream answers a
+request with `400 Invalid role: …`, bili auto-rewrites the offending role to
+`system`, retries the request once, and — if the retry succeeds — remembers
+the mapping **for that session only** (nothing is written to your config).
+Later requests in the session skip the 400 round-trip. The log line printed
+when the auto-fix fires includes a copy-paste per-provider snippet if you
+want the mapping permanently.
+
 ## How sessions work
 
 The proxy needs a stable per-conversation identifier to isolate compression

src/compat-roles.ts

@@ -0,0 +1,119 @@
+import type { ProviderRoutes } from "./config.js";
+import { findRoute } from "./config.js";
+
+/** A resolved wire-compat role map: source role → role name upstream accepts. */
+export type CompatRoles = Record<string, string>;
+
+/** Validate a `compat.roles`-shaped value: an object of string → string.
+ *  Non-string entries are dropped (a malformed partial never breaks the
+ *  proxy); returns undefined when there is nothing usable. */
+export function parseCompatRoles(v: unknown): Record<string, string> | undefined {
+    if (!v || typeof v !== "object" || Array.isArray(v)) return undefined;
+    const obj = v as Record<string, unknown>;
+    let out: Record<string, string> | undefined;
+    for (const [k, val] of Object.entries(obj)) {
+        if (typeof val !== "string" || val.length === 0) continue;
+        out ??= {};
+        out[k] = val;
+    }
+    return out;
+}
+
+/** Merge the wire-compat role map for one request: global `compat.roles`
+ *  (config root) overlaid by the per-provider route entry (longest-URL-prefix
+ *  match, identical to the compress-settings lookup). Provider entries win
+ *  per key; global keys not overridden still apply. Empty when unconfigured —
+ *  the default, byte-for-byte transparent forward (#552). */
+export function resolveCompatRoles(
+    routes: ProviderRoutes,
+    upstreamUrl: string | undefined,
+    globalRoles: Record<string, string> | undefined,
+): Record<string, string> {
+    const providerRoles = findRoute(routes, upstreamUrl)?.compat?.roles;
+    if (!globalRoles && !providerRoles) return {};
+    return { ...globalRoles, ...providerRoles };
+}
+
+/** Apply the role map to a serialized chat-completions or Responses body.
+ *  Rewrites exact-match roles only — position, content and every other field
+ *  are untouched. Returns the original string (no re-stringify) when nothing
+ *  matched, so the default path stays byte-identical.
+ *
+ *  Applied at the FINAL forward boundary on purpose: every emission site —
+ *  client-sent items, bili's injected compress prompt, instructions hoisting,
+ *  compress-loop items — is visible there, and the kernel already normalizes
+ *  developer→system internally (session state / block IDs / prefix-cache are
+ *  computed from pre-wire core messages), so only outbound bytes change. */
+/** Object-level role rewrite shared by the forward boundary (string body)
+ *  and the compress-retry loops (parsed body). Mutates `parsed` in place;
+ *  returns the number of roles rewritten. */
+export function applyCompatRolesJson(
+    parsed: Record<string, unknown>,
+    protocol: "openai" | "responses",
+    roles: Record<string, string>,
+): number {
+    if (Object.keys(roles).length === 0) return 0;
+    const items = protocol === "openai" ? parsed.messages : parsed.input;
+    if (!Array.isArray(items)) return 0;
+    let rewritten = 0;
+    for (const item of items) {
+        if (!item || typeof item !== "object" || Array.isArray(item)) continue;
+        const msg = item as Record<string, unknown>;
+        if (protocol === "responses" && msg.type !== undefined && msg.type !== "message") continue;
+        const role = msg.role;
+        if (typeof role !== "string") continue;
+        const mapped = roles[role];
+        if (mapped === undefined || mapped === role) continue;
+        msg.role = mapped;
+        rewritten++;
+    }
+    return rewritten;
+}
+
+/** Stop-words that follow "role" in prose ("Invalid role must be one…"):
+ *  a captured token equal to one of these means the regex grabbed a word
+ *  from the sentence, not a role name. */
+const ROLE_CAPTURE_STOPWORDS = new Set([
+    "must", "should", "be", "is", "one", "of", "the", "a", "an", "in", "for", "not", "was", "and", "or", "to", "only", "allowed", "supported", "valid", "value", "message", "messages",
+]);
+
+const ROLE_REJECTION_PATTERNS = [
+    /(?:invalid|unknown|unsupported|unrecognized|unexpected)[a-z ]{0,24}?role\s*[:=]?\s*["'`]?([a-z][a-z0-9_-]{1,31})["'`]?/i,
+    /role\s*[:=]?\s*["'`]?([a-z][a-z0-9_-]{1,31})["'`]?\s+(?:is\s+)?not\s+(?:supported|allowed|recognized|valid|accepted)/i,
+];
+
+/** Detect an upstream 400 that exists only because of a message role the
+ *  provider does not support (e.g. codex ≥0.153 sends "developer";
+ *  converting backends answer 400 "Invalid role: developer"). Conservative by
+ *  design: status must be 400, the body must blame a role, and the captured
+ *  token must look like a role name. Returns the offending role so the
+ *  caller can auto-rewrite it to a supported one. */
+export function detectRoleRejection(status: number, bodyText: string): { role: string } | null {
+    if (status !== 400) return null;
+    const head = bodyText.slice(0, 4096);
+    for (const re of ROLE_REJECTION_PATTERNS) {
+        const m = re.exec(head);
+        if (!m) continue;
+        const role = m[1].toLowerCase();
+        if (ROLE_CAPTURE_STOPWORDS.has(role)) continue;
+        return { role };
+    }
+    return null;
+}
+
+export function applyCompatRoles(
+    body: string,
+    protocol: "openai" | "responses",
+    roles: Record<string, string>,
+): { body: string; rewritten: number } {
+    if (Object.keys(roles).length === 0) return { body, rewritten: 0 };
+    let parsed: Record<string, unknown>;
+    try {
+        parsed = JSON.parse(body) as Record<string, unknown>;
+    } catch {
+        return { body, rewritten: 0 };
+    }
+    const rewritten = applyCompatRolesJson(parsed, protocol, roles);
+    if (rewritten === 0) return { body, rewritten: 0 };
+    return { body: JSON.stringify(parsed), rewritten };
+}

src/compress-loop-responses.ts

@@ -67,6 +67,9 @@ interface CompressLoopResponsesCtx {
 interface RequestOptions {
     url: string;
     headers: Record<string, string>;
+    /** Final-stage wire transform (compat.roles, #552) — same contract as the
+     *  unified loop's RequestOptions.wireTransform. */
+    wireTransform?: (body: Record<string, unknown>) => Record<string, unknown>;
 }
 
 interface FunctionCallAccumulator {
@@ -215,7 +218,7 @@ export async function compressLoopResponsesJson(
         const result = await fetchWithRetry(requestOptions.url, {
             method: "POST",
             headers: requestOptions.headers,
-            body: JSON.stringify(requestBody),
+            body: JSON.stringify(requestOptions.wireTransform ? requestOptions.wireTransform(requestBody) : requestBody),
             ...(ctx.proxyUrl ? { dispatcher: proxyDispatcher(ctx.proxyUrl) } : {}),
         }, undefined, undefined, (info) => {
             // #189: correlate the rejection with the rewrite that preceded it.

src/config.ts

@@ -5,6 +5,8 @@ import { configFile } from "./paths.js";
 import { log as loggerLog } from "./logger.js";
 import { validateHttpProxy, type ProxyFallbackOptions } from "./upstream-proxy.js";
 
+import { parseCompatRoles } from "./compat-roles.js";
+
 export function safeReadJson(path: string): unknown {
     try {
         // Strip a leading UTF-8 BOM: Windows Notepad saves UTF-8 "with BOM",
@@ -41,6 +43,12 @@ export type ProviderRoute = {
     compressProtocol?: "tools" | "marker";
     /** Per-provider compression overrides (level 2 of 3). See CompressSettings. */
     compress?: CompressSettings;
+    /** Per-provider wire-compat overrides. `roles` maps message roles to the
+     *  role name this upstream accepts (e.g. `"developer": "system"`) —
+     *  applied at the forward boundary to the FINAL wire body, covering
+     *  client-sent roles and bili's own injected prompt alike (#552). Wins
+     *  per key over the global `compat` block. */
+    compat?: { roles?: Record<string, string> };
 };
 export type ProviderRoutes = Record<string, ProviderRoute>; // key = upstream URL prefix (the /bili/<this> string)
 
@@ -234,6 +242,11 @@ export type ProxyOptions = {
         injectNudge: boolean;
     };
     promptCache: { routing: PromptCacheRouting };
+    /** Wire-compat role map (global level; per-provider `compat.roles` overlays
+     *  it per key). `{"developer":"system"}` rewrites developer→system on the
+     *  forwarded body for upstreams without the developer role (#552). Empty =
+     *  byte-for-byte transparent. */
+    compat: { roles: Record<string, string> };
     sessionHeader: string;
     log: boolean;
     debug: boolean;
@@ -373,6 +386,7 @@ export function loadOptions(env: NodeJS.ProcessEnv = process.env): ProxyOptions
         promptCache: {
             routing: parsePromptCacheRouting(env.ACP_PROMPT_CACHE_ROUTING ?? fileConfig.promptCache?.routing),
         },
+        compat: { roles: parseCompatRoles(fileConfig.compat?.roles) ?? {} },
         sessionHeader: env.ACP_SESSION_HEADER ?? fileConfig.sessionHeader ?? "x-acp-session",
         log: env.ACP_LOG !== "0" && fileConfig.log !== false,
         debug: (env.ACP_DEBUG ?? (fileConfig.debug ? "1" : "0")) === "1",
@@ -421,6 +435,10 @@ type FileConfig = {
     compress?: CompressSettings & { injectTool?: boolean; injectNudge?: boolean };
     promptCache?: { routing?: string };
     mitm?: { enabled?: boolean; domains?: string[] };
+    /** Global wire-compat block. `roles` maps message roles to the role name
+     *  upstreams accept (e.g. `{"developer":"system"}`) — applied to the
+     *  final forwarded body for openai/responses requests (#552). */
+    compat?: { roles?: Record<string, string> };
 };
 
 function nonEmpty(value: string | undefined): string | undefined {
@@ -495,11 +513,13 @@ export function parseRouteEntry(v: unknown): ProviderRoute | undefined {
     // is the KEY in the providers map (identical to the /bili/<url> string),
     // so it is NOT repeated inside the value.
     if (v && typeof v === "object" && !Array.isArray(v)) {
-        const obj = v as { models?: Record<string, ModelEntry>; proxy?: string; compressProtocol?: string; compress?: CompressSettings };
+        const obj = v as { models?: Record<string, ModelEntry>; proxy?: string; compressProtocol?: string; compress?: CompressSettings; compat?: { roles?: unknown } };
         const route: ProviderRoute = { models: obj.models };
         if (typeof obj.proxy === "string") route.proxy = obj.proxy;
         if (obj.compressProtocol === "marker" || obj.compressProtocol === "tools") route.compressProtocol = obj.compressProtocol;
         if (obj.compress) route.compress = obj.compress;
+        const compatRoles = parseCompatRoles(obj.compat?.roles);
+        if (compatRoles) route.compat = { roles: compatRoles };
         return route;
     }
     // A bare value (e.g. null) means "this upstream exists, no overrides".

src/fetch-util.ts

@@ -52,6 +52,7 @@ export async function fetchWithTimeout(
     externalSignal?: AbortSignal,
 ): Promise<{ response: Response; clearTimer: () => void }> {
     const controller = new AbortController();
+    let cleared = false;
     const armTimer = () => {
         const t = setTimeout(() => {
             liveUpstreamTimers.delete(t);
@@ -62,6 +63,11 @@ export async function fetchWithTimeout(
     };
     let timer = armTimer();
     const rearm = () => {
+        // A late body chunk can resolve after the consumer already called
+        // clearTimer (abandoned response — e.g. a failed retry whose body is
+        // never read, or a client abort). Re-arming after cleanup would leak
+        // a fresh 10-minute timer and pin the event loop (#552 e2e hang).
+        if (cleared) return;
         clearTimeout(timer);
         liveUpstreamTimers.delete(timer);
         timer = armTimer();
@@ -75,6 +81,7 @@ export async function fetchWithTimeout(
         }
     }
     const cleanup = () => {
+        cleared = true;
         clearTimeout(timer);
         liveUpstreamTimers.delete(timer);
         if (onExternalAbort && externalSignal) externalSignal.removeEventListener("abort", onExternalAbort);

src/loop/core.ts

@@ -79,6 +79,11 @@ export interface LoopCtx {
 export interface RequestOptions {
     url: string;
     headers: Record<string, string>;
+    /** Final-stage wire transform (e.g. compat.roles role rewrite, #552).
+     *  Applied to every outgoing body this loop re-sends — truncation retry,
+     *  degraded retry, rebuilt rounds — so re-sent bodies carry the same
+     *  compat the initial forward() applied. Returns the body to serialize. */
+    wireTransform?: (body: Record<string, unknown>) => Record<string, unknown>;
 }
 
 export type ParsedStreamEvent =
@@ -211,7 +216,7 @@ export async function* runCompressLoop(
             {
                 method: "POST",
                 headers: requestOptions.headers,
-                body: JSON.stringify(body),
+                body: JSON.stringify(requestOptions.wireTransform ? requestOptions.wireTransform(body) : body),
                 ...(ctx.proxyUrl ? { dispatcher: proxyDispatcher(ctx.proxyUrl) } : {}),
             },
             undefined,