[ranxianglei]

Model: glm-5.3 (vllm)

What

tests/e2e/e2e-codex.test.ts — a single gated e2e test that runs the real codex CLI (0.147) through a bili proxy against a real Responses-compatible upstream, asserting the full context-management lifecycle:

1. warmup — plants a known contamination value (1400)
2. load growth — payload user-messages ride the wire verbatim (models routinely shrink tool output — seq 1 1500 | tail -n 1 was observed), so growth is deterministic
3. ACP compress — under a forced window (BILI_LAUNCHER_MODEL_WINDOWS), asserts preflight compressed/compress requested and a usage drop
4. purity — post-compress recall (row count + sentinel value) must be correct and must NOT echo the planted contamination (validated end-to-end: 1500, 30217 survived T1 fold + T2 distill)
5. forge (gated E2E_FORGE=1) — restart under a 128k window (persist reloads the same session; blocks survive), then a codex-side auto-compact limit below the ACP trigger makes codex emit its native compaction request; asserts codex compact intercepted … upstream not contacted, a fc_bili_ compaction item in the codex rollout, and intact recall after the forged compact

Plus test:e2e script, tests/e2e/README.md, and a workflow_dispatch CI job (ci-e2e.yml) with upstream URL/key from repo secrets.

Gating

• Suite skips by default (ACP_TEST_E2E=1 to run) — npm test stays free
• E2E_CHECK=1 preflight: codex version + dist path + upstream probe, no tokens
• Forge phase additionally needs E2E_FORGE=1 and a dist with BILI_CODEX_COMPACT support (feat: intercept + forge codex native compaction when ACP is healthy (#321 PR-E2) #325)

Real-device validation (local, full stack incl. #317–#328 + #325 with UA fix)

pass 1 fail 0 in 5.5 min against the local relay; forge evidence: gate=true steady=45818/128000 activeBlocks=3 → forged SSE with 3 block summary(s), upstream not contacted.

Bugs found while building this (report-only; not fixed here)

• codex UA regression: codex 0.147 exec sends user-agent: codex_exec/0.147.0 …, not codex_cli_rs/… — isCodexClient never matched, so the feat: intercept + forge codex native compaction when ACP is healthy (#321 PR-E2) #325 intercept was dead code for exec-mode clients. Fix pushed to feat: intercept + forge codex native compaction when ACP is healthy (#321 PR-E2) #325 (c507e0c).
• preflight fail-fast × soft-protection livelock: an over-window payload whose foldable ranges are all inside the protected zone 502s forever; the EMERGENCY nudge recovery path is cut because the request never reaches the model. Suggests a follow-up issue.
• relay usage reporting inflates ~2×, which pins steady usage near the window ceiling — the forge gate's 90% margin needs a window with headroom (the suite restarts under 128k for this reason).

Pre-flight

• typecheck ✓ (no new errors)
• npm test 816/816 ✓
• build ✓; E2E_CHECK preflight ✓ on this tree

[github-actions]

📦 Built Package Artifact

Branch: 2026-08-29_e2e-suite (8efbaef)

Option A — Install from npm PR tag (recommended)

npm install -g billion-context@pr-329

Each push to this PR publishes a new version under the pr-329 npm tag.

Option B — Download artifact

1. Download the artifact from the Actions run
2. Extract the tarball and install:

tar xzf billion-context-pr329.tgz
npm install -g package

---

This comment is automatically updated on each push.

[ranxianglei]

🤖 ework agent · vllm-qwen/qwen3.8-27b

[bot] Picked up. PR #329 is already open on branch 2026-08-29_e2e-suite (head eed2c8b): #329

Verifying the tree now: typecheck, npm test, build, and E2E_CHECK=1 preflight. Will also file the follow-up issue for the preflight fail-fast × soft-protection livelock you flagged. Results shortly.

[ranxianglei]

🤖 ework agent · vllm-qwen/qwen3.8-27b

[bot] Verification complete on branch 2026-08-29_e2e-suite (commit eed2c8b):

Pre-flight (this tree = master + this commit)

• npm run typecheck ✓ no errors
• npm test ✓ 745/745 pass, 0 fail (your 816 was on your fuller tree with feat: intercept + forge codex native compaction when ACP is healthy (#321 PR-E2) #325 etc.; this branch only adds the 4 e2e files)
• npm run build ✓ (dist/index.js 2.52 MB, acp-kernel inlined)
• E2E_CHECK=1 node --import tsx --test tests/e2e/e2e-codex.test.ts ✓ pass 1 fail 0 skip 1 — printed codex: codex-cli 0.150.1; dist: …/dist/index.js; upstream: http://127.0.0.1:8199/v1, probe unreachable (no local relay in this sandbox, as expected)

Full-suite note: the ACP_TEST_E2E=1 run needs a real Responses upstream + tokens, which this sandbox doesn't have — so your real-device validation (pass 1 fail 0 in 5.5 min, forge evidence gate=true steady=45818/128000 activeBlocks=3 → forged SSE with 3 block summary(s), upstream not contacted) plus the ci-e2e.yml workflow_dispatch job (secrets-fed upstream) remain the full-verification path. One heads-up: codex here is 0.150.1 (you targeted 0.147) — preflight passes on it; the UA fix from #325 (c507e0c) should be confirmed once #325 lands, since the forge phase depends on isCodexClient matching exec-mode UAs.

Design checks that look right:

• Gating is sound: default npm test glob is tests/*.test.ts (no subdirs), so the suite can't leak into normal runs even before the ACP_TEST_E2E gate; test:e2e script + E2E_CHECK/E2E_FORGE layers match the README.
• turnExpect's 3-attempt retry with the "直接给出上一请求的答案…" prompt correctly isolates the relay 偶尔模型在acp工具处停止执行 #221 no-content family from real failures.
• Forge phase's -c model_auto_compact_token_limit=999999 warm turn → settle → 3000 trigger sequence is the right way to force codex's native compaction below the ACP trigger.
• WORK_ROOT = cwd/tmp (not TMPDIR) sidesteps codex's TMPDIR-home refusal — good call, and the CI artifact step picks it up.

Follow-up issue filed: the preflight fail-fast × soft-protection livelock is now tracked at #330 — over-window payload with all foldable ranges inside the protected zone → permanent preflight fail-fast 502 (retryable=false), EMERGENCY nudge path cut because the request never reaches the model. The issue includes the exact code path (src/server.ts:1802-1807, src/preflight.ts:264-266), impact, and a suggested fix order (relax soft protection under overflow → emergency truncation → actionable error).

The other two report-only bugs need nothing from this PR: the UA regression is already fixed in #325 (c507e0c), and the relay's ~2× usage inflation is a relay property the suite already works around via the 128k forge restart (noted in #330's impact section as a compounding factor).

PR stays open for your review: #329