Skip to content

Install daemon updates with the npm CLI that ships with its Node - #6083

Closed
alexandre-vl wants to merge 1 commit into
get-bb:mainfrom
alexandre-vl:bb/daemon-self-update-node-npm-cli
Closed

Install daemon updates with the npm CLI that ships with its Node#6083
alexandre-vl wants to merge 1 commit into
get-bb:mainfrom
alexandre-vl:bb/daemon-self-update-node-npm-cli

Conversation

@alexandre-vl

@alexandre-vl alexandre-vl commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

Human comments

What was wrong

The host daemon installs a new bb-app with npm install -g … in two places: protocol self-update, and the server-move install. Both resolved npm through PATH, with the directory of the running Node prepended. When that Node came from an older mise install, the npm next to it is mise's bash wrapper. The wrapper runs the real npm-cli.js and then mise reshim under set -euo pipefail. Under launchd or systemd mise is not on PATH, so the wrapper exits 127 after the package has already installed. bb records a failed update, never restarts into the new build, and retries with backoff while the machine stays disconnected (#6079).

What changed

  • apps/host-daemon/src/protocol-self-update.ts: defaultInstallTarball now takes the Node executable path and runs <node> <npm-cli.js> from that Node's own install. That is the command the wrapper itself runs, without the reshim.
    • POSIX: <node dir>/../lib/node_modules/npm/bin/npm-cli.js. Windows: <node dir>\node_modules\npm\bin\npm-cli.js, the same lookup npmLaunch in apps/server/src/assets/install-machine-windows.mjs already uses.
    • When that file is missing, the installer falls back to npm on PATH, as before (layouts where npm lives elsewhere, such as distro packages).
    • PATH still gets the Node directory prepended, so lifecycle scripts find node.
  • Both callers, the self-updater and server-move/service.ts, pass process.execPath.
  • createProtocolSelfUpdater lost its unused runProcess option. The installer tests now call defaultInstallTarball directly with a fake Node install, so they no longer depend on how the test runner's own Node is laid out.
  • No wire change, so no HOST_DAEMON_PROTOCOL_VERSION bump. No CLI or docs change.

Intended side effects:

  • The wrapper's mise reshim no longer runs. A bb-app installed into mise's own prefix without BB_APP_NPM_PREFIX would not get shims for bins added in future releases until someone reshims. Machines set up by the installer pass --prefix and never relied on a reshim.
  • On Windows, an npm the user upgraded into their global prefix is no longer picked over the npm bundled with Node. The Windows installer already behaves this way.

Daemons that are already stuck can't receive this fix through self-update. Their failed attempt already installed the new files, though, so restarting the service brings them up on the new version, which includes this fix.

How you verified

  • New default tarball installer tests: 5 cases, one of them a POSIX real-process test with a failing npm shim beside Node. Against the old command selection, 4 of them fail. The regression test fails with Error: Command failed: npm install -g … mise: command not found. With the fix all 17 tests in the file pass.
  • pnpm exec turbo run typecheck --filter=@bb/host-daemon passes.
  • pnpm exec turbo run test --filter=@bb/host-daemon: 73 files, 811 passed, 7 skipped.
  • oxfmt --check and oxlint pass on the changed files.
  • End to end on Debian 13, mise Node 24.17.0, launchd's default PATH (env -i PATH=/usr/bin:/bin:/usr/sbin:/sbin), installing a running server's real /install/bb-app.tgz into a temp prefix through defaultInstallTarball:
    • before: rejected, code 127;
    • after: resolved, with bb-app 0.46.2-nightly.39620384282.1 and its bin links installed.
  • An independent review in a separate bb thread returned "ship". I applied its one test-hermeticity suggestion: the real-process test now installs into a temp --prefix.
  • Not verified:
    • a real Windows run of node.exe npm-cli.js. The real-process test is skipped on win32; the Windows branch is covered by the fixture tests and mirrors npmLaunch.
    • Homebrew, distro, Volta and fnm layouts. Where npm-cli.js isn't next to Node, they take the previous PATH fallback.

Fixes #6079

AGENT GENERATED

@alexandre-vl
Install daemon updates with the npm CLI that ships with its Node
11deedb 
The host daemon's protocol self-update and the server-move install ran
`npm` from PATH. When Node comes from an older mise install, that `npm`
is a bash wrapper that runs `mise reshim` after global installs and
exits 127 when `mise` is not on the launchd or systemd PATH, so bb
recorded a failed update after the package had installed and never
restarted.

Run `<node> <npm-cli.js>` from the running Node's own install, the same
command the wrapper runs, and fall back to `npm` on PATH when that
install has no npm CLI.

Fixes get-bb#6079
alexandre-vl added a commit to alexandre-vl/bb that referenced this pull request Oct 21, 2026
@alexandre-vl
Sync other forks, report upstream pull request health, carry get-bb#6083
f212de7
@github-actions github-actions Bot mentioned this pull request Oct 21, 2026
Open
@ymichael ymichael mentioned this pull request Oct 23, 2026
Merged
ymichael added a commit that referenced this pull request Oct 27, 2026
@ymichael @claude review
One host toolchain for executable discovery, child PATH, and proxy/CA…
88048e6 
… defaults (#6194)

## Human comments

## What was wrong

bb launches child processes from many places (server, daemon, desktop,
launcher, plugins), and each one found executables and built PATH its
own way. Under a macOS launchd service or a GUI launch, PATH is minimal,
so several things broke:

- Plugins could not run `bb` because the CLI's Node was not on PATH
(#5807).
- Daemon self-update went through a mise npm wrapper that exits 127
after a successful install, so bb treated the update as failed (#6079).
- Homebrew upgrades deleted the running Node, and bb kept using the dead
path to start workers (#5295).
- The Claude Code installation check searched only PATH, so a native
install in `~/.local/bin` that was not on PATH was reported missing and
bb kept offering to reinstall it (#6284).
- Node ignored corporate proxies and OS-trusted certificates because
`NODE_USE_ENV_PROXY` and `NODE_USE_SYSTEM_CA` were never set (#5260,
#3239).

## What changed

- **One resolver:** `HostToolchain` in
`packages/process-utils/src/host-toolchain.ts` owns executable lookup,
the child PATH and proxy/CA environment, npm selection, and replacement
of a deleted Node.
- It reads the login-shell PATH, or the user and machine registry PATH
on Windows, plus proxy variables from the shell.
- It adds the running Node runtime's directory (never the Electron
app's) and standard user tool directories (`~/.local/bin`, mise/asdf
shims, Volta, and Homebrew on macOS).
- Discovery runs in the background and is coalesced. Launches never wait
for it.
- `ENOENT`, exit 127 and lookup misses start a background refresh.
Commands are never replayed.
- **Launch helpers:** `spawnHostProcess`, `forkHostProcess`,
`execHostFile`, `execHostFileCallback`, `execHostFileBuffer` and
`prepareHostPtyCommand` in `host-process.ts` use the resolver. Core call
sites across the server, daemon, desktop, CLI, launcher, host-workspace,
host-watcher, local-open-targets and the provider kits now use these
helpers instead of `node:child_process`. The per-package lookup code is
deleted: desktop `desktop-shell-path`, daemon `runtime-shell-env`
shell/registry parsing, `selectResolvedExecutable`, and the plugin-local
`bb`/`claude` candidate lists.
- **Child environments:**
- Every child PATH gets the login-shell PATH and the standard tool
folders. When the PATH is the one bb started with, the login-shell PATH
comes first; when a caller passes a different PATH, the caller's entries
stay first. bb never infers that a PATH was meant to be exact.
- Scripts that start with `#!/usr/bin/env node` run with the `node` on
PATH.
- `npm` and `npx` run as found on PATH. Daemon self-update instead runs
`npm-cli.js` with the Node installation that owns it
(`hostToolchain.npmCli`), which avoids version-manager wrappers. Inside
the Electron desktop runtime, bb never substitutes its own runtime for
npm.
- **Network defaults:** the packaged server/daemon launchers start bb's
own Node processes with `--use-system-ca` and `--use-env-proxy`
(`@bb/config/node-network`), each only when the runtime accepts it, and
bb's forked workers and provider bridge workers inherit them. They are
flags rather than exported variables, so terminals, provider CLIs and
agent shells keep the user's own Node networking settings. Setting
`NODE_USE_SYSTEM_CA=0` or `NODE_USE_ENV_PROXY=0` opts out. `bb-app env
set` treats the proxy and CA keys as startup-only.
- **Startup:** the daemon and the packaged desktop app wait for
discovery at startup. The server starts it in the background. The `bb`
CLI never runs it.
- **Plugin SDK:** new `@get-bb/plugin-sdk/process` entry point exports
`experimental_hostToolchain`, `experimental_spawnHostProcess`,
`experimental_forkHostProcess`, `experimental_execHostFile` and
`experimental_execHostFileCallback`. Core uses the same implementation.
- SDK bumped to 0.6.86. Plugins that import the entry point now require
`>=0.6.86`.
  - `check-plugin-sdk-version.mjs` now watches `src/process.ts`.
- Added an entry to `docs/api_to_audit.md` and the Plugin Guide's
Provider bridges card.
- **Plugins:** MCP, automations, browser automation, GitHub, keep-awake,
storage retention (including the external worktree scanner), account
pool, and the Claude Code, Codex and Pi providers use the SDK helpers.
- **CLI startup:** the `bb` entry imports
`@bb/process-utils/host-process` rather than the package barrel, so
startup does not load execa.
- **Docs:** `docs/configuration.md` covers proxies and certificates and
how executable discovery works.
- **No wire changes:** `HOST_DAEMON_PROTOCOL_VERSION` is unchanged.

Behavior differences from main:

- Proxy variables set in the user's login shell now reach child
processes when the caller's environment does not set them.
- Children receive the caller's PATH extended with the login-shell PATH
and the standard tool directories.

Not addressed:

- Daemon WebSocket connections still ignore the proxy (#4957).
- Node releases without built-in environment-proxy support (before 22.21
and 24.5) still connect directly (#3239).
- A real corporate proxy or CA deployment has not been verified; the
proxy checks used local proxies and mitmproxy.
- Codex native installer (#5930) and Pi bundled-runtime install (#5943)
remain separate follow-ups.

This supersedes #6081, #6083, #5997 and #5948.

## How you verified

- New process-utils tests (`host-toolchain.test.ts`,
`host-toolchain-windows.test.ts`) cover:
  - npm install through a mise-style wrapper that exits 127 (#6079)
  - recovery of a deleted Homebrew Node (#5295)
- Claude Code installation status finding a native install outside PATH
(#6284; fails on main)
  - running a `bb` Node script with a minimal service PATH (#5807)
  - a new command found after the login PATH changes
  - retaining the last good PATH when discovery fails
  - bounded probes and no repeated probing of missing commands
  - non-blocking discovery
- caller PATH entries kept first, with the standard folders added after
them
  - caller proxy variables winning over login-shell ones in either case
  - `#!/usr/bin/env node` scripts not run through the bb runtime
- npm paired with the Node that owns it, and the PATH npm used for
ordinary launches
  - a tool in a standard folder found when the caller's PATH lacks it
  - byte-exact buffered output
  - no replay after exit 127
  - Windows registry PATH expansion and retention
- bb-app tests start the server and daemon with real environment proxies
and check that `NO_PROXY` bypasses them (#3239), and that explicit
opt-outs are kept (#5260).
- Reproduced before these fixes, and verified after:
- The launcher's `bb --version` took 1.7 s because of a synchronous
login-shell probe. It now takes 0.08 s.
- A binary git blob came back corrupted (64 bytes in, 121 out). It now
comes back byte-exact.
- `pnpm exec turbo run typecheck lint --filter='...[origin/main]'`
passes.
- `pnpm exec turbo run test --filter='[origin/main]'` passes for every
changed package except the desktop Electron smoke test below. Under that
parallel load, the MCP timeout test and the server builtin hot-reload
test missed their timing windows; both pass when run alone.
- `check-plugin-sdk-version.mjs` and `oxfmt --check` pass.
- Desktop `preload-build` (Electron smoke) times out locally with both
this branch's and main's desktop sources, so it is environmental.
- Manual checks on real installs:
- **Windows 11 ARM64 VM** (bb-app from this branch, started by its
sign-in scheduled task): Codex (`codex.exe`), Claude Code (`claude.exe`)
and Pi (`pi.cmd`) are detected with their on-disk names. Updating Pi
through bb (`bb machine provider-cli install <host> pi --action update`)
ran `npm.cmd` in a pty and moved Pi from 1.3.0 to 1.4.0. Signed-in
Claude Code and Codex turns run `node`, `npm` and `bb`, and `where.exe`
finds node, npm, bb, claude, codex and pi. A bb terminal resolves
`codex`, `claude`, `pi`, `node` and `npm` from the registry PATH.
- **Packaged macOS desktop app launched with launchd's environment**
(`PATH=/usr/bin:/bin:/usr/sbin:/sbin`): the server and daemon receive
the login-shell PATH; Codex under `~/.bun/bin` and Pi under nvm are
detected; Claude Code, Codex and Pi turns each run `git --version`,
`node --version` and `bb --version` successfully. The packaged `bb` CLI
runs with `PATH=/usr/bin:/bin` (#5807). This check found that exporting
`NODE_USE_*` made `bb --version` inside agent shells print proxy
warnings and system-certificate errors; switching to flags fixed it.
- **Main vs branch environment diff** (both packaged apps launched with
launchd's environment): in a bb terminal, Claude Code and Codex agent
shells, and the server, daemon, bridge, Claude and Codex processes, the
only differences are the added PATH fallbacks and the network flags on
bb's own processes. `node`, `npm`, `git`, `bb`, `codex`, `claude` and
`pi` resolve to the same files. This check found the Electron app
directory on child PATHs; it is now excluded.
- **Proxy:** with `HTTPS_PROXY` in the data directory's `env.json`, the
branch routes the server's npm version check and the bridge workers'
Claude, Codex, Cursor and OpenCode usage fetches through the proxy; main
sends the same requests directly. On Linux, a TLS-intercepting mitmproxy
whose CA is only in the system store works with the default flags and
fails with `NODE_USE_SYSTEM_CA=0`. This check found that bridge workers
lacked the flags; they now inherit them.
- **Linux** (Ubuntu 24.04 container, Node from mise only on the
login-shell PATH, launched with systemd's default PATH): providers are
found through mise shims, a Pi update through bb ran mise's npm, a bb
terminal resolves every tool, and a machine running its daemon like the
systemd unit self-updated from protocol 255 to a bumped 256 server and
reconnected.
- **Daemon self-update:** a machine enrolled with the manual installer,
running a supervised `--auto-update` daemon under launchd's PATH,
updated from protocol 252 to a locally bumped 253 server: it installed
the server's package into its private prefix with the Node-owned npm,
restarted, and reconnected. The global npm prefix was untouched.

Fixes #6079
Fixes #5807
Fixes #5295
Fixes #5260
Fixes #6284

> AGENT GENERATED

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: review <review@local>
alexandre-vl added a commit to alexandre-vl/bb that referenced this pull request Oct 30, 2026
@alexandre-vl
Drop the layers upstream merged and the carries they no longer need
b4fbe20 
- bb/daemon-self-update-node-npm-cli (get-bb#6083): superseded by
  upstream get-bb#6194.
- bb/project-default-source-host (get-bb#6054) and
  bb/explain-failed-automations-thr_agcnkvvvke (get-bb#6005): merged upstream;
  each merges into main with main's own tree.
- carry/automation-environment-composer: it only reconciled the composer
  branch with explain-failed, and the updated
  bb/automation-environment-composer merges without it.
- carry/github-plugin-inbox: it breaks bb-plugin-github#typecheck on
  current main; the updated github-plugin-inbox carries the inbox itself.
@github-actions github-actions Bot closed this Nov 5, 2026
@github-actions

github-actions Bot commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

This is an automated message. We have closed this pull request because it has been open for more than two weeks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Daemon self-update fails with exit 127 when node comes from an older mise install: the npm wrapper's mise reshim is not on the service PATH

1 participant

@alexandre-vl