Skip to content

feat: dsh native mode — full plugin via bili plugin install dsh + upgraded launcher (#941) - #942

Merged
ranxianglei merged 5 commits into
masterfrom
2026-09-18_dsh-native
Sep 18, 2026
Merged

ranxianglei merged 5 commits into
masterfrom
2026-09-18_dsh-native

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Fixes #941

What

dsh gets full plugin-mode compression in BOTH lanes, sharing one cordis plugin (src/agent/dsh-native.ts):

  • Mode 3 — bili plugin install dsh (new): appends a managed block (# bili begin/# bili end markers, user entries/comments preserved, placeholder [] replaced and restored on remove) to every ~/.dsh/profiles/*/cordis.patch.yml. At load the plugin self-spawns a proxy (attaches to a healthy instance instead of doubling), patches globalThis.fetch to rewrite model-API URLs to <proxy>/bili/<url>, registers the manifest tools (compress/decompress/acp_status) as native dsh tools, and disables dsh auto-compaction (compaction-basic → auto: false override — a patch replaces the row's whole config and dsh-base ships it config-less).
  • Mode 2 — bili dsh (upgraded): the --patch overlay now injects the SAME full plugin (was: /acp-panel-only dsh-acp.js, kept for old overlays). Under the launcher the plugin ATTACHES to the launcher's proxy — no second spawn, routing stays with the launcher (proxy envs / settings overlay), the fetch patch only stamps headers, including on already-routed /bili/-prefixed requests.

Key mechanics (verified against dsh 0.1.1-rc.2 sources)

  • Parameters pass-through: ctx.tools.register projects definition.parameters verbatim onto the model wire (defineTool merely compiles the author spec) — the proxy manifest's input_schema IS a real JSON Schema, so no conversion is needed.
  • Session binding: conversation id from ctx.agents.currentInitiator() — dsh's AsyncLocalStorage attribution, read synchronously at request time inside the agent's driver chain (withInitiator wraps the loop's kick, so LLM fetches run inside the boundary). Tool executes use exec.agent.session.id and fail loudly when agentless.
  • Plugin-mode gating (pi.ts discipline): headersFor returns nothing until the manifest registration completes — round 1 rides wire mode instead of arriving tool-less.
  • Attach precedence: kill-switches (BILLION_CONTEXT_PLUGIN=0, BILI_NATIVE_DSH=0, BILI_PROVIDER_REWRITES) > attach (BILLION_CONTEXT_ATTACH ?? BILLION_CONTEXT_PROXY) > spawn — same shape as planNativeOpencode (opencode 2.x: no-launcher npm-plugin deployment — plugins: ["billion-context"], plugin lazily spawns the proxy #809).

native-intercept changes

  • headersFor?: (url) => Record<string,string> hook; stamped on rewritten (spawn), attach-mode, and already-routed /bili/ requests; all three init.headers forms + Request-object inputs handled (withHeaders).
  • Attach mode never rewrites the URL.
  • New routedBiliModelUrl() predicate (embedded model URL from /bili/ form).

Known limitation

Manual /compact has no dsh-side event hook, so its boundary is left to the kernel's natural ingest diff (auto-compaction is off — #395 gap acceptable, noted in README).

Pre-flight

  • npm run typecheck ✅
  • npm test — 1878 pass / 0 fail (new: tests/dsh-native.test.ts ×10, native-intercept headersFor/attach ×4; launcher + plugin-agent assertions updated for the new patch content and the 6th agent)
  • npm run build ✅ (dist/agent/dsh-native.js, self-contained)

…intercept) via plugin install and launcher (#941)

- native-intercept: headersFor hook + attach mode (no rewrite, stamp only) +
  already-routed /bili/ model URLs get headers without rewriting
- dsh-native.ts: cordis plugin (inject tools/commands/agents) — attach
  (launcher/BILLION_CONTENT_ATTACH) or self-spawn proxy; manifest tools
  registered verbatim (parameters = JSON schema pass-through); headersFor
  gates plugin mode on tool readiness (pi.ts discipline); conversation id
  from ctx.agents.currentInitiator() at request time; /acp session-bound
  with latest fallback; BILI_NATIVE_DSH=0 opt-out
- plugin-install: dsh agent — managed block (# bili begin/end markers) in
  every ~/.dsh/profiles/*/cordis.patch.yml: bili-native insert +
  compaction-basic auto:false override; remove restores placeholder [];
  status counts profiles
- launcher: writeDshAcpPatch now carries dsh-native + compaction override
  (dsh-acp.js kept for old overlays)
- tests: dsh-native.test.ts (10), native-intercept headersFor/attach (4),
  launcher/plugin-agent updated
- README bilingual dsh sections
@github-actions

github-actions Bot commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-09-18_dsh-native (1beb1bf)

Option A — Install from npm PR tag (recommended)

npm install -g billion-context@pr-942

Each push to this PR publishes a new version under the pr-942 npm tag.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr942.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

收到,开始 review PR #942(dsh native mode)。已拉取分支 2026-09-18_dsh-native,接下来核对 diff、跑 typecheck/test/build、检查 CI 状态,稍后给出结论。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Review of 2026-09-18_dsh-native (head 12a0f0e) — verified against dsh's documented plugin API in this repo's sources and ran the full pre-flight locally.

What I checked

Code walkthrough

  • src/agent/dsh-native.ts — attach precedence matches planNativeOpencode (opencode 2.x: no-launcher npm-plugin deployment — plugins: ["billion-context"], plugin lazily spawns the proxy #809): kill-switches (BILLION_CONTEXT_PLUGIN=0, BILI_NATIVE_DSH=0, BILI_PROVIDER_REWRITES) > attach (BILLION_CONTEXT_ATTACH ?? BILLION_CONTEXT_PROXY) > spawn. Plugin-mode gating is correct: headersFor returns nothing until manifest registration completes, so round 1 rides wire mode instead of arriving tool-less (pi.ts discipline). Session binding via ctx.agents.currentInitiator() at request time, exec.agent.session.id for execution with a loud fail when agentless — right shape.
  • native-intercept changes: all three init.headers forms + Request-object inputs handled (withHeaders rebuilds with duplex: "half"); routedBiliModelUrl() short-circuits before readyOrigin, so already-routed /bili/ URLs never block on bootstrap. Good.
  • Launcher overlay now injects the same full plugin + compaction-basic → auto: false; plugin-install managed block preserves user entries/comments, handles the placeholder [] root, restores it on remove. Consistent across both lanes as claimed.

Local pre-flight (on the PR head)

  • npm run typecheck ✅
  • npm test: 1877 pass / 0 fail / 2 skipped
  • npm run build ✅ — dist/agent/dsh-native.js emitted, self-contained

⚠️ Windows CI failed — fixed directly on the branch

The first CI run failed only on windows-latest (22) and (24) (ubuntu + e2e jobs green; master's windows runs are green, so this PR introduced it). Both windows jobs failed the same two tests in tests/dsh-native.test.ts:

mergeDshManagedPatch: placeholder [] is replaced, comments survive   (line 37)
mergeDshManagedPatch is idempotent and rewrites a moved install path (line 65)

Root cause: both assertions hardcode POSIX file-URL literals (file:///opt/bili/dist/agent/dsh-native.js), but dshManagedPatchBlock builds the URL via pathToFileURL(path.join(root, …)). On win32 path.join("/opt/bili", …) resolves against the cwd drive with backslashes, so the generated URL never matches the literal. Classic platform-dependent test (§7.2). All other 1877 tests passed on Windows.

Fixed directly on the branch per review discipline (small, test-only fix): commit 31fdf03 — test: make dsh patch-block URL assertions platform-independent (#941). The assertions now compute the expected URL with the same pathToFileURL(path.join(…)) construction the source uses, so they hold on every platform. Verified locally: tests/dsh-native.test.ts 10/10, typecheck clean. Push landed; CI re-running (windows jobs in progress at time of writing).

Finding — needs your call: attach-mode no-rewrite breaks opencode V1

The new attach branch in patched() (src/agent/native-intercept.ts) deliberately skips URL rewriting. For dsh that's correct (the launcher owns routing / the proxy envs are stamped by the host). But this intercept is shared: opencode-native's V1 server() installs the same patch (src/agent/opencode-native.ts:594) specifically to catch providers with no explicit baseURL — the config-hook rewrite can't see SDK-default providers, which is why the fetch patch exists there (comment at lines 584–593).

Consequence after this PR: OpenCode 1.x host + bili plugin install opencode + preset BILLION_CONTEXT_PROXY/BILLION_CONTEXT_ATTACH + a provider without an explicit baseURL → those requests go direct, uncompressed, silently (previously they were rewritten into the proxy). Meanwhile the V2 route in the same entry still rewrites in attach mode (createNativeRoute, opencode-native.ts:211–224), so one entry now behaves differently per lane, and this contradicts #809's documented attach semantics ("probe + rewrite + stamp").

Scope is narrow (V1 host + attach + SDK-default provider only) but the silent-bypass is exactly what the V1 comment says the patch exists to prevent. Suggested fix: make no-rewrite a per-host opt-in flag on NativeInterceptState, set only by dsh-native, leaving opencode's attach rewriting intact. I didn't change shared wire semantics unilaterally — your call whether to fold it into this PR or split it out.

Nits (non-blocking)


Windows CI 挂的两个测试是断言里硬编码 POSIX file-URL、而源码用 pathToFileURL(path.join(...)) 在 win32 下生成不同路径导致的(已在 PR 分支直接补了平台无关的修复提交 31fdf03,CI 重跑中);另发现一个需你拍板的问题:native-intercept attach 模式不再改写 URL 会让 opencode V1 无显式 baseURL 的 provider 静默绕过压缩(与 #809 文档语义及同文件 V2 路由不一致),建议改成 dsh 专属开关;其余核对全部通过,Windows 转绿后可合并。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Follow-up — the URL-assertion fix landed (31fdf03), CI re-ran: windows (24) ✅, but windows (22) failed on two different tests, both pre-existing and unrelated to dsh:

#152: ensureRootCA writes combined-ca.pem with MITM root + public roots   (tests/ca-combined.test.ts:36)
#152: combined bundle re-merges when the root already exists              (tests/ca-combined.test.ts:48)
error: 'combined bundle must contain the MITM root CA'

Root cause — latent Windows race, not introduced by this PR

This PR touches none of ca.ts / paths.ts / mitm.ts / server startup; it's a timing-luck flake that surfaced now:

  1. tests/ca-combined.test.ts isolates itself by setting HOME, but caDir() resolves through os.homedir() (src/paths.ts:23) — and on win32 homedir() uses USERPROFILE, ignoring HOME. So on Windows its CA dir is the runner's real profile dir (C:\Users\runneradmin\.local\share\billion-context\ca).
  2. Meanwhile two other test files start real proxy servers with mitm.enabled: true without any XDG_DATA_HOME isolation: tests/blind-tunnel-stats.test.ts:52 and tests/log-mask.test.ts:355. Every such startup runs setupMitm() → ensureRootCA() eagerly (src/mitm.ts:131).
  3. Node's test runner executes test files as parallel processes. At job start, several of them see "no existing CA", each generates its own RSA root (~200 ms keygen), and they overwrite each other's root-ca.pem / combined-ca.pem in that one shared dir. The ca-combined process's bundle holds cert A while the file on disk ends up holding cert B → combined.includes(mitmRoot) fails. Both ZCode 集成:MITM-only CA bundle 覆盖系统根证书,导致 pip/git 等 TLS 校验失败 #152 assertions died in the same ~30 ms window at job start, consistent with exactly this interleaving.

Linux/macOS never hit it because there homedir() honors HOME, so every file lands in its own tmp dir. Master's green windows runs were just lucky timing. This violates §7.2 (deterministic tests) and is worth fixing regardless of this PR.

Fixed directly on the branch (test-only)

Commit 9b4e705 — test: isolate CA dir via XDG_DATA_HOME in MITM-touching test files (Windows flake):

  • tests/ca-combined.test.ts: also set XDG_DATA_HOME under its tmp home (it's checked before homedir(), so isolation now holds on every platform);
  • tests/blind-tunnel-stats.test.ts + tests/log-mask.test.ts: wrap their MITM-enabled server startups with the same XDG_DATA_HOME tmp-dir pattern tests/ws-upgrade.test.ts already uses.

Local pre-flight on the new head: full suite 1877 pass / 0 fail / 2 skipped, typecheck clean, build clean. Pushed; CI re-running. If you'd rather keep this PR strictly scoped to dsh, say so and I'll move those two test commits onto their own branch/PR — otherwise they ride along since they're what unblocks windows CI here.


第二轮 CI 暴露的另两个失败是仓库里早已存在的 Windows 竞态(CA 目录隔离在 win32 上失效:os.homedir() 不认 HOME,多个并行测试进程互相覆盖根证书),与 dsh 改动无关;已用 XDG_DATA_HOME 隔离直接修掉并提交到 PR 分支(9b4e705),本地全量测试通过,等 Windows CI 转绿。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Final status — CI is fully green on head 9b4e705 (both windows jobs ✅, ubuntu ✅, e2e-fake-codex ✅, codex-overlay ✅, build-artifact ✅), PR is clean/mergeable.

Recap of what this branch now carries:

Commit What
12a0f0e feat: dsh native mode (your original work — reviewed, mechanics verified against dsh 0.1.1-rc.2 sources)
31fdf03 test: platform-independent URL assertions in tests/dsh-native.test.ts (fixed the deterministic windows failure)
9b4e705 test: XDG_DATA_HOME CA-dir isolation in ca-combined / blind-tunnel-stats / log-mask (fixed the pre-existing windows race)

Both extra commits are test-only; if you want this PR strictly scoped to dsh, I can split them out — otherwise they're what makes windows CI honest here.

One item still open for your call (from my first review note): the shared attach-mode no-rewrite change in src/agent/native-intercept.ts silently drops compression for OpenCode 1.x + bili plugin install opencode + preset BILLION_CONTEXT_PROXY/BILLION_CONTEXT_ATTACH + provider without explicit baseURL (V1 fetch-patch used to rewrite those; V2 route still does). Suggested fix: per-host opt-in flag on NativeInterceptState, set only by dsh-native. Happy to implement it here or on a separate branch once you decide.

Aside from that decision, nothing blocks merging on my side.


最终状态:CI 全绿(含两个 Windows job),PR 可合并。我补了两个纯测试提交——一个修 PR 自身断言的平台相关失败,一个修仓库里早已存在的 Windows CA 目录竞态;唯一待你拍板的是 native-intercept attach 模式不重写 URL 会让 opencode V1 无显式 baseURL 的 provider 静默绕过压缩(建议改成 dsh 专属开关),其余无阻塞。

ework-agent and others added 2 commits September 18, 2026 19:46
… plugin (#941)

cordis validates loader entry ids across ALL merged patch layers and hard-fails
boot on duplicates (`duplicate loader entry id: bili-native`, verified against
dsh 0.1.1-rc.2). A user who ran `bili plugin install dsh` and then launched via
`bili dsh` got two `id: bili-native` inserts (persistent cordis.patch.yml block
+ launcher --patch overlay) and dsh could not start at all.

The launcher now skips its overlay when any profile carries the managed block
(dshNativeInstalled in plugin-install.ts) — the persistent block already
provides the plugin insert plus the compaction-basic override, so nothing is
lost. Fresh users (no install) keep the overlay exactly as before.

Verified with real dsh 0.1.1-rc.2: persistent-only boot passes the loader
(plugin registers, proxy spawns); persistent+overlay reproduces the crash;
overlay-only stays clean. Typecheck + full suite (1878 pass) + build green.
The #941 stamp-only attach branch silently dropped compression for
OpenCode 1.x under `bili plugin install opencode` with a preset
BILLION_CONTEXT_PROXY/BILLION_CONTEXT_ATTACH and a provider without an
explicit baseURL: the V1 `server()` surface shares the module-level
intercept state (attach=true), and the V1 fetch patch is exactly what
used to route those SDK-default-baseURL requests into the proxy.

Attach now rewrites to `${origin}/bili/${url}` exactly like spawn
(#809 probe+rewrite semantics restored). For dsh under the `bili dsh`
launcher this is doubly safe: settings-overlay URLs are already
`/bili/`-shaped and take the routed branch (headers only), and rewriting
a raw upstream URL to the loopback proxy is never proxied by the MITM
envs (an http loopback target bypasses HTTPS_PROXY/HTTP_PROXY forwarding
in every case that matters — worst case an absolute-form self-hop, which
the proxy unwraps normally).

- native-intercept: attach branch removed (unified rewrite+stamp path);
  interface docs updated
- tests: attach assertions now expect the rewrite; headersFor gating
  unchanged
@ranxianglei

Copy link
Copy Markdown
Owner Author

Review item resolved in 1beb1bf — went one step simpler than the suggested per-host flag: attach mode now rewrites like spawn mode (the stamp-only branch is gone entirely).

Why that is safe for both hosts:

  • opencode V1 — restores the old behavior exactly: server() shares the module-level state (attach=true), and the V1 fetch patch again routes SDK-default-baseURL providers into the proxy under a preset env (opencode 2.x: no-launcher npm-plugin deployment — plugins: ["billion-context"], plugin lazily spawns the proxy #809 probe+rewrite semantics).
  • dsh under bili dsh — the launcher's three routing channels are unaffected by a rewrite: settings-overlay and DEEPSEEK_BASE_URL URLs are already /bili/-shaped and take the routed branch (headers only, no rewrite); a raw upstream URL rewrites to a loopback proxy target, which proxy envs never intercept (an http loopback target bypasses HTTPS_PROXY; worst case an absolute-form self-hop via HTTP_PROXY, which the proxy unwraps normally). So the rewrite is at worst a no-op improvement (drops the MITM hop for rewritten requests).

Docs/comments updated to match; attach-mode tests now assert the rewrite. Typecheck + full suite (1878 pass) + build green.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Verified 1beb1bf — agreed, dropping the stamp-only branch entirely is cleaner than my suggested per-host flag, and it's correct on inspection:

  • opencode V1 finding resolved exactly as intended: the unified path (src/agent/native-intercept.ts:198-202) rewrites to ${origin}/bili/${url} + stamps headersFor; opencode sets no headersFor, so V1 attach gets a pure rewrite — byte-for-byte the pre-dsh: full plugin/native mode (mode 3) + upgrade launcher to plugin-mode compression (mode 2) #941/opencode 2.x: no-launcher npm-plugin deployment — plugins: ["billion-context"], plugin lazily spawns the proxy #809 behavior.
  • Fail-closed preserved: neither host sets state.respawn in attach (dsh-native.ts:196-200 spawn-only at :205; opencode-native.ts:262-267 spawn-only at :270), so a TypeError on a rewritten request re-throws instead of ever spawning against an external proxy. One nuance I accept: if the launcher's proxy dies mid-session under bili dsh, the fetch-patch channel now throws where it previously slipped through direct — but every other launcher routing channel is dead at that point too, so it's not a new failure class, just a louder one.
  • dsh loopback reasoning holds: already-routed /bili/ URLs short-circuit into the headers-only branch before the rewrite (native-intercept.ts:161-166); a raw upstream URL becomes a loopback http target that proxy envs don't forward, worst case an absolute-form self-hop the proxy unwraps.
  • Tests now assert rewrite+stamp for attach (incl. the headersFor === undefined case still rewriting), README zh/en kept in sync.

Also reviewed the other new commit 00c9ed2 (skip launcher overlay when a persistent install provides the plugin): sound — cordis' duplicate-entry-id hard-fail is real, dshNativeInstalled() checks the managed-block marker across profiles, and the persistent block carries both the plugin insert and the compaction override so nothing is lost. Edge case I checked: a mixed state (some profiles with the block) degrades gracefully — unblocked profiles just lack plugin registration, no crash path.

Pre-flight on 1beb1bf: typecheck ✅, full suite 1878 pass / 0 fail / 2 skipped, build ✅; CI fully green (both windows jobs included), PR clean/mergeable.

No open items left from my side — merge is yours to make.


核对通过:attach 专用分支整体移除后,opencode V1 的 probe+rewrite 语义精确恢复(两个宿主 attach 都不设 respawn,失败即 fail-closed、绝不对外部代理拉起新进程),dsh 侧"回环目标永不被代理 env 拦截"的推理成立,测试与中英文档同步更新;顺带核对了同批的 00c9ed2(持久安装存在时跳过 overlay,避免 cordis 重复条目硬失败)也正确。本地与 CI 全部转绿,我的 review 项清零,可以合并。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

dsh: full plugin/native mode (mode 3) + upgrade launcher to plugin-mode compression (mode 2)

1 participant