From fac6a2c025d352323a8418179a5e12714dd0ae19 Mon Sep 17 00:00:00 2001 From: Alexey Zimarev Date: Wed, 10 Jun 2026 16:43:27 +0200 Subject: [PATCH 01/16] docs: spec for daemon service supervisor (install/uninstall/start/stop) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Per-user OS service integration so the daemon auto-restarts after an external SIGKILL (jetsam/OOM/crash) — the root cause traced on 2026-06-10: exit 137 with zero death-rattle output = uncatchable kill, reproduced under detached tmux during a memory-pressure jetsam storm. launchd LaunchAgent (macOS), systemd --user unit (Linux), Task Scheduler logon task (Windows), behind an IServiceManager seam with pure unit-text generators for testing. Co-Authored-By: Claude Opus 4.8 (1M context) --- ...-06-10-daemon-service-supervisor-design.md | 282 ++++++++++++++++++ 1 file changed, 282 insertions(+) create mode 100644 docs/superpowers/specs/2026-06-10-daemon-service-supervisor-design.md diff --git a/docs/superpowers/specs/2026-06-10-daemon-service-supervisor-design.md b/docs/superpowers/specs/2026-06-10-daemon-service-supervisor-design.md new file mode 100644 index 000000000..e92569bb2 --- /dev/null +++ b/docs/superpowers/specs/2026-06-10-daemon-service-supervisor-design.md @@ -0,0 +1,282 @@ +# Daemon service supervisor (install/uninstall/start/stop) + +## Problem + +The `kcap` daemon is a long-running process, but nothing supervises it. Users +run it via `kcap daemon start -d` (a one-shot detached spawn) or under tmux, and +neither survives the process dying. + +A debugging session on 2026-06-10 traced a recurring "daemon crashes silently" +report to its root cause: the daemon exits with code **137 (SIGKILL)** and the +`death-rattle` instrumentation emits **nothing** — proving the kill comes from +*outside* the .NET runtime and is uncatchable in-process. The evidence: + +- Exit 137 = 128 + 9 (SIGKILL). +- Zero `[death-rattle]` lines in 4001 log lines (no managed handler ran). +- The daemon was **idle for 49 minutes** before the kill, so nothing it did + triggered it. +- The macOS unified log showed a `JETSAM_REASON_MEMORY_IDLE_EXIT` storm at the + same time (launchd SIGKILLing dozens of idle processes), driven by a + concurrent heavy `dotnet` build/test run spiking memory pressure. +- It happened under **detached tmux**, which falsified the earlier "Warp block + recycling" hypothesis — tmux only shields against terminal-driven signals + (SIGHUP/SIGTERM), not kernel memory reclamation (jetsam) or the Linux OOM + killer. + +**There is no in-process fix** — you cannot survive SIGKILL. The fix is +operational resilience: run the daemon under a real per-user service supervisor +that auto-restarts it after any death (jetsam, OOM, crash, logout/login), on all +three platforms. + +## Scope + +Add a `service` command group under `kcap daemon` that registers, deregisters, +and controls the daemon as a **per-user** OS service: + +``` +kcap daemon service install [--name N] [--max-agents N] [--server-url URL] [--no-start] +kcap daemon service uninstall [--name N] +kcap daemon service start [--name N] +kcap daemon service stop [--name N] +kcap daemon service status [--name N] +``` + +Per-user / login-session scope on every platform: + +| Platform | Mechanism | Unit location | +|---|---|---| +| macOS | launchd **LaunchAgent** | `~/Library/LaunchAgents/io.kurrent.kcap.daemon[.].plist` | +| Linux | systemd **`--user`** unit (templated) | `~/.config/systemd/user/kcap-daemon@.service` | +| Windows | Task Scheduler **logon task** | task `kcap-daemon[-]` | + +The existing ad-hoc lifecycle (`kcap daemon start [-d]`, `stop`, `status`, +`logs`, `doctor`) is **unchanged**. Service management is additive. + +### Out of scope (explicitly deferred) + +- **System-wide / boot scope** (LaunchDaemon, systemd system unit, true Windows + Service running as SYSTEM). Rejected: the daemon needs the user's `$HOME`, + resolved kcap profile, login keychain, and an interactive session to spawn + Claude/Codex PTY agents. A session-0 service breaks PTY agent spawning. +- **systemd linger** (`loginctl enable-linger`) so the unit runs while logged + out. Per-user/login scope means "stops at logout" is acceptable; a future + `--linger` flag can add it. +- **Auto-reinstall on `npm update`.** The platform-package binary path is stable + across updates; `daemon doctor` will flag a unit whose binary path no longer + exists (see below). + +## Architecture + +### `IServiceManager` seam (in `Capacitor.Cli`) + +Lives in `Capacitor.Cli` (not `Capacitor.Cli.Core`): its only consumer is +`DaemonCommands` (Cli), and the side-effecting apply uses `ProcessHelpers` +(Cli) — Core must not depend on Cli. `Capacitor.Cli.Tests.Unit` already +references the Cli project and tests Cli-project classes, so this is unit-testable. +Mirrors the existing interface seams in the codebase (`IPtyProcessFactory`, +`IHostedAgentLauncher`). Each implementation splits a **pure unit-text +generator** (testable) from a **thin side-effecting apply** (shells out to the +platform tool). + +```csharp +public record ServiceSpec( + string Name, + string DaemonBinaryPath, // absolute path to kcap-daemon, resolved at install + string LogPath, // stable log file passed as --log-file + IReadOnlyList ExtraArgs); // optional baked overrides (--max-agents, --server-url) + +public enum ServiceState { NotInstalled, Installed, Running } + +public interface IServiceManager { + string Describe(); // e.g. "launchd LaunchAgent" + string GenerateUnit(ServiceSpec spec); // PURE — the primary tested seam + ServiceState Status(string name); + void Install(ServiceSpec spec, bool startNow); + void Uninstall(string name); + void Start(string name); + void Stop(string name); +} +``` + +Selection: + +```csharp +public static class ServiceManagerFactory { + public static IServiceManager ForCurrentOs(); // Launchd | Systemd | WindowsScheduledTask + // throws PlatformNotSupportedException otherwise +} +``` + +Implementations: `LaunchdServiceManager`, `SystemdServiceManager`, +`WindowsScheduledTaskServiceManager`. + +### What the unit runs + +The unit execs the **`kcap-daemon` binary directly** — the OS service manager is +the sole supervisor (no `kcap daemon start` CLI layer, no PID-file dance; the +daemon's own `flock` still prevents duplicate names). + +The unit **pins only**: + +- `--name ` — must be deterministic so it matches the `flock` slot. +- `--log-file ` — stable log location (`daemon.log`, or + `daemon-.log` for a named instance), via `PathHelpers.ConfigPath`. + +Everything else (`server-url`, `max-agents`, claude/codex paths) keeps resolving +from the active profile + env **at each launch** — `DaemonRunner.RunAsync` +already does `AppConfig.ResolveActiveProfile(args)` and reads these from the +resolved profile. So editing the profile does **not** require a reinstall. +`--max-agents` / `--server-url` on `install` are optional overrides baked into +`ExtraArgs` for users who want them frozen. + +### Restart semantics (the subtle part) + +Restart must fire on **crash / SIGKILL** (jetsam, OOM) but **not** on a clean +stop or clean exit. All three managers distinguish a *failure* from a deliberate +*stop*: + +| Platform | Restart-on-failure config | Clean stop (no restart) | +|---|---|---| +| launchd | `KeepAlive = { SuccessfulExit = false }` | `launchctl kill SIGTERM` → daemon cooperatively exits 0 → not restarted | +| systemd | `Restart=on-failure`, `RestartSec=5` | `systemctl --user stop` → systemd knows it's a stop, not a failure | +| schtasks | task settings `RestartCount` + `RestartInterval=PT1M` | `schtasks /End` → terminated, not a failure exit | + +A SIGKILL is an abnormal exit on all three → it **does** restart (the jetsam/OOM +case we are fixing). A `service stop` keeps the unit installed but down; it +returns at next login (launchd `RunAtLoad`, systemd still `enable`d) or via +`service start`. Only `service uninstall` deregisters. + +### Per-verb → platform command mapping + +| verb | macOS | Linux | Windows | +|---|---|---|---| +| install | write plist; `launchctl bootstrap gui/ ` (`RunAtLoad` starts it); follow with `launchctl kill SIGTERM` if `--no-start` | write unit; `systemctl --user daemon-reload`; `enable --now kcap-daemon@` (or `enable` only if `--no-start`) | `schtasks /Create /TN /XML /F`; `/Run` unless `--no-start` | +| uninstall | `launchctl bootout gui//