Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 18 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -666,17 +666,23 @@ host. The long-term shape is captured in
[ADR-011](adrs/011-deployment-artifact-format.md): CI publishes a
portable cosign-signed tarball; `install.sh` cosign-verifies and
extracts it without an SDK on the host. PR C1 (#248) shipped Option A
as the pragmatic short-term default. The flip to Option B as the
installer's default is tracked separately and gated on one release
cycle of green CI publishing the signed manifest + tarball.
as the pragmatic short-term interim default. **The D4 default-flip (#249)
has since landed: `--from-release` is now the installer default.**

#### `--from-release` (ADR-011 Option B opt-in)
#### `--from-release` (ADR-011 Option B — default since #249)

D3 of the ADR-011 rollout (#249) adds an opt-in release-tarball mode to
`install.sh`. The default (no flag) remains source-build for backward
compatibility — the default-flip ships in D4 once one release cycle of
the D2 publishing pipeline has gone green and the D3 smoke test has run
on a fresh macOS + Ubuntu host.
`install.sh` with no mode flag now installs from the cosign-signed release
tarball (ADR-011 Option B). Source builds are the explicit opt-in escape
hatch: `--from-source` requires `--i-accept-unverified-source` to
acknowledge that a local tree carries no cosign chain (see the source-build
note below). On a **first** release-mode install `--version vX.Y.Z` is
required (`latest` is permitted only on upgrades).

> **Windows (`install.ps1`)** does not yet have a `--from-release` path — it
> builds from source unconditionally (no cosign verification). Windows
> release-path parity is tracked in
> [#367](https://github.com/psmfd/agent-expertise-api/issues/367); until then
> the default trust posture differs from macOS/Linux.

```bash
./scripts/install.sh --from-release --version vX.Y.Z # fetch + cosign-verify + extract
Expand Down Expand Up @@ -740,8 +746,9 @@ bundle (#256).
Quick start (macOS / Linux / WSL):

```bash
./scripts/install.sh # per-user install, fdd publish (source build, current D3 default for back-compat)
./scripts/install.sh --from-release --version vX.Y.Z # ADR-011 release-mode: cosign-verify a published tarball, no SDK on host
./scripts/install.sh --version vX.Y.Z # DEFAULT (ADR-011): cosign-verify a published tarball, no SDK on host
./scripts/install.sh # DEFAULT on upgrades: fetches the latest signed release
./scripts/install.sh --from-source --i-accept-unverified-source # opt-in: build from local tree (no cosign chain)
edit ~/.config/expertise-api/secrets.env # set ConnectionStrings__DefaultConnection
./scripts/migrate.sh # apply EF Core migrations (idempotent)
./scripts/expertise-apictl status # daily-use service control
Expand Down
2 changes: 1 addition & 1 deletion adrs/011-deployment-artifact-format.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Cosign-signed published tarball over SDK-on-host for Archetype A2 install

- Status: accepted (commitment to migrate; default flip tracked separately)
- Status: accepted; **default flip landed** (#249, 2026-07-06 — `install.sh` defaults to `--from-release`, `--from-source` gated behind `--i-accept-unverified-source`)
- Date: 2026-05-22
- Companion: `scripts/install.sh`, `.github/workflows/release.yml`, [README §Archetype A2](../README.md#archetype-a2-native-os-service-install-no-docker)
- Tracking issue: [#245](https://github.com/TheSemicolon/agent-expertise-api/issues/245)
Expand Down
31 changes: 21 additions & 10 deletions docs/install-upgrade-track.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Persistent TODO for the multi-PR effort that gets the native-service install path (Archetype A2) ready for pi-agent integration. Updated at the end of every working session.

**Last updated:** 2026-07-06 (E3 from-release smoke #260 landed; Debian/Ubuntu bootstrap #246 landed via #335; headless boot + macOS LaunchDaemon #145 landed via #307; migrate log-flush #263 and env-export #262 bugs closed. Remaining: D4 default-flip #249 and RHEL/Fedora bootstrap #247.)
**Last updated:** 2026-07-06 (D4 default-flip #249 landed — `install.sh` now defaults to `--from-release`. Earlier same day: E3 smoke #260, Debian bootstrap #335, headless/LaunchDaemon #307, migrate bugs #263/#262. Remaining: RHEL/Fedora bootstrap #247 only.)
**Driver issue:** [#230 readiness sweep](https://github.com/psmfd/agent-expertise-api/issues/230)
**Goal:** make `scripts/install.sh` safe, upgrade-aware, and capable of bootstrapping host dependencies on macOS (primary) and Linux, so the pi-agent integration session has a turn-key target.

Expand All @@ -21,17 +21,18 @@ Persistent TODO for the multi-PR effort that gets the native-service install pat
- **Headless boot + macOS LaunchDaemon (#145): DONE** via #307; macOS `--system` hard-errors
until LaunchDaemon support is present (#291).
- **migrate log-flush (#263) and env-export (#262) bugs: CLOSED.**
- `install.sh` **still defaults to `--from-source`** — the D4 default-flip has not landed
(see Remaining stack).
- **D4 default-flip (#249): DONE** — `install.sh` now defaults to `--from-release`
(cosign-verified tarball); `--from-source` is the opt-in escape hatch and requires
`--i-accept-unverified-source`. The macOS gate (E3 red under #366) was satisfied by a
local `verify-release.sh` + bsdtar-extract run against v1.2.0 on Apple Silicon (cosign
verify PASS, sha bind PASS, osx-arm64 ONNX native present) in lieu of the #366-blocked
CI smoke.
- `install.sh` **now defaults to `--from-release`.**

### Remaining stack

1. **D4 default-flip (#249, OPEN)** — flip `install.sh`'s default from `--from-source` to
`--from-release`. The E3 gate is now satisfied (from-release smoke exists and runs); the
flip itself is the remaining action, subject to ADR-011's binding-flip rule (one release
cycle of green E3 on `dev`).
2. **C3 RHEL/Fedora bootstrap (#247, OPEN)** — `dnf` equivalent of the Debian module (#246),
scoped to `cosign + bsdtar` per ADR-011 Option B. Independent of D4; no shared files.
1. **C3 RHEL/Fedora bootstrap (#247, OPEN)** — `dnf` equivalent of the Debian module (#246),
scoped to `cosign + bsdtar` per ADR-011 Option B. The last open install-hardening item.

### Other open follow-ups (lower priority)

Expand Down Expand Up @@ -76,7 +77,7 @@ Persistent TODO for the multi-PR effort that gets the native-service install pat
| D1 | csproj `<RuntimeIdentifiers>` + `<RollForward>` precondition | #249 | ☑ merged as `a54aabf` (#251) | One-line csproj addition. Hard precondition for D2 portable publish (ONNX RID natives). Zero behavior change to docker/helm/`dotnet run`/existing A2 installs. |
| D2 | release.yml portable publish + manifest + cosign sign-blob | #249 | ☑ merged as `6929bdb` (#252) | `scripts/build/generate-manifest.sh` + 30-assertion test suite; release.yml publish/manifest/sign/upload steps; README §Supply-chain verification stanza for the tarball-via-manifest recipe; CI `release-manifest-generator` job seeds #166 incrementally. First D2 production run gates the D4 default-flip per ADR-011. |
| D3 | install.sh `--from-release` opt-in (cosign verify + bsdtar extract + downgrade defense + runtime semver tighten) | #249 | ☑ merged as `88d20b1` (#258) | Load-bearing PR. Ships `scripts/verify-release.sh` (sanctioned cosign-verify entrypoint), `scripts/lib/release-consumer.sh` (fetch/verify/inspect/extract/markers), install.sh wiring (`--from-release` / `--from-source` / `--version` / `--allow-downgrade` / `--accept-republished-version` / `--skip-release-api-crosscheck`), and `tests/install/test-release-tarball-flow.sh` (39 assertions). Pre-PR fan-out folded 2 HIGH + 4 MED + 5 LOW. Follow-up flags deferred to #255 (`--tarball-url`) and #256 (`--allow-offline-verify`); SemVer §11 comparator deferred to #257 (closed by feat/semver-aware-downgrade-defense). |
| D4 | Cosign as managed dep in `bootstrap-*.sh`; default flip `--from-source` → `--from-release` | #249 | ◐ macOS half landed (#267, `549e6cc`); Linux halves bundle with C2/C3; default-flip still gated | macOS half adds `_macos_ensure_cosign` to `bootstrap-macos.sh` (Homebrew, idempotent, honors any cosign on PATH). bsdtar intentionally not installed (macOS `/usr/bin/tar` IS bsdtar; release-consumer.sh::rc_select_tar already auto-detects). Debian cosign folded into #335; RHEL half pending in C3 (#247). E3 (#260) is now merged and running, so the gate is satisfied — **the default-flip itself (#249, OPEN) is the remaining action.** D2 production-verified by the v1.0.0 cut on 2026-05-23. |
| D4 | Cosign as managed dep in `bootstrap-*.sh`; default flip `--from-source` → `--from-release` | #249 | ☑ **default-flip landed** — `install.sh` defaults to `--from-release`; `--from-source` requires `--i-accept-unverified-source` | macOS cosign dep landed (#267, `549e6cc`); Debian cosign folded into #335; RHEL half pending in C3 (#247). Default-flip: unset `INSTALL_MODE` resolves to release; source mode gated behind the ack flag. macOS gate (E3 red under #366) satisfied by a local `verify-release.sh` + bsdtar run against v1.2.0 on Apple Silicon (cosign PASS, sha bind PASS, osx-arm64 ONNX native present). |
| E1 | install.sh end-to-end smoke (Linux / systemd-user / postgres-17 in privileged container) | #166 | ☑ merged as `122542d` (#261) | Ships `scripts/test/Dockerfile.install-smoke` (debian13 + systemd + postgres17 + dotnet-sdk-10) + `scripts/test/test-install-smoke.sh` (12-assertion harness) + new `install-smoke-linux` job in `ci.yml`. Covers the primary AC of #166 (Linux + Postgres + service install + restart + health + orphan check). Container approach mirrors the existing apictl-restart-race-debian13 precedent (privileged systemd-in-container with cgroup v2 unified mount + host-uid 1000 user); deviates from #166's suggested GHA `services:` shape because the smoke needs systemd-user for service-install/restart parity with real hosts, which GHA's host runner does not give cleanly. Harness is OS-agnostic by design so E2 (macOS) reuses it without modification. First run on dev: 1m58s, all 12 assertions PASS. |
| E2 | macOS install.sh smoke (`brew install postgresql@17`, reuses E1 harness) | #259 | ☑ merged as `731cba3` (#264) | Closes the stretch AC of #166. Adds `install-smoke-macos` to `ci.yml` (macos-latest / arm64, brew-installed postgresql@17 + pgvector, no privileged container). Reuses `scripts/test/test-install-smoke.sh` — the macOS branches seeded in E1 (`start_postgres_macos`, Darwin psql dispatch, BSD-pgrep guard) ran for the first time and exposed one harness bug: hardcoded `DOTNET_ROOT=/usr/share/dotnet` overrode the GHA-runner's `/Users/runner/.dotnet`, breaking the FDD apphost. Fixed in the same PR (honor caller `DOTNET_ROOT`; per-OS fallback). Pre-PR fan-out (code-review + security + linter) PASS; two Info-level hardening notes folded in (postgresql@14 collision check + pgvector hard-fail probe). First green run on dev: 1m58s wall clock, 12/12 assertions PASS. |
| E3 | `--from-release` install.sh smoke (both OSes; gates D4 default-flip) | #260 | ☑ merged (#260, asset-name fix #293) — `.github/workflows/install-smoke-from-release.yml` runs `install-smoke-linux-from-release` + `install-smoke-macos-from-release` | Runs `install.sh --from-release --version <latest-tag>` against the latest cosign-signed release. Path-filter trigger on `scripts/install.sh`, `scripts/verify-release.sh`, `scripts/lib/release-consumer.sh`, `.github/workflows/release.yml`. Asserts D3 post-install markers (`.install-mode == release`, `.install-version-semver` non-empty with correct appVersion, `.install-history` non-empty). The D4 gate is now satisfied. |
Expand Down Expand Up @@ -181,6 +182,16 @@ Status legend: ☐ todo · ◐ in progress · ☑ merged · ✗ abandoned.

## Session log

- **2026-07-06 (later)** — **D4 default-flip (#249) landed.** `install.sh` now defaults to
`--from-release` (unset `INSTALL_MODE` resolves to release); `--from-source` is the opt-in
escape hatch and requires `--i-accept-unverified-source`. Migrated the source-default
assumption out of the test suite (`run_install` in test-upgrade-roundtrip; the smoke
harness source leg + both `--system` calls; flow-test cases 17/18 + new 18b). The ADR-011
macOS gate — E3 red under #366 (Homebrew bottle) — was satisfied out-of-band by a local
`verify-release.sh` + bsdtar-extract of v1.2.0 on Apple Silicon (cosign verify PASS, sha
bind PASS, `runtimes/osx-arm64/native/libonnxruntime.dylib` present). Only #247 (RHEL
bootstrap) remains on the readiness track.

- **2026-07-06** — Tracker refreshed to current state (no new install work this session; audit
of issue states + org-URL fix). Since the 2026-05-23 v1.0.0 cut, the readiness track
advanced: E3 `--from-release` smoke landed (#260, asset-name fix #293); Debian/Ubuntu
Expand Down
59 changes: 35 additions & 24 deletions scripts/install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
# [--rid RID] [--fix-line-endings]
# [--allow-system-prefix]
# [--install-deps] [--upgrade-deps]
# [--from-release | --from-source [--i-accept-unverified-source]]
# [--from-release | --from-source --i-accept-unverified-source]
# [--version vX.Y.Z|latest]
# [--allow-downgrade] [--accept-republished-version]
# [--skip-release-api-crosscheck]
Expand All @@ -23,16 +23,15 @@
#
# Install mode (ADR-011, issue #249):
#
# * --from-release Fetch cosign-signed portable tarball from GHCR
# Releases. Cosign verifies the manifest; manifest's
# sha256 binds the tarball. Default mode after the
# D4 flip gate clears. Requires --version on FIRST
# release-mode install (--version latest only
# permitted on upgrades).
# * --from-source Build from local source tree (current default for
# back-compat with pre-D3 callers). Becomes opt-in
# after the D4 flip; --i-accept-unverified-source
# will be mandatory then. Accepted silently in D3.
# * --from-release DEFAULT (D4 flip, #249). Fetch cosign-signed portable
# tarball from GitHub Releases. Cosign verifies the
# manifest; the manifest's sha256 binds the tarball.
# Requires --version on the FIRST release-mode install
# (--version latest only permitted on upgrades).
# * --from-source Opt-in escape hatch: build from the local source tree
# (no cosign chain). REQUIRES --i-accept-unverified-source
# to acknowledge the reduced trust posture. For devs
# iterating on a clone and air-gapped operators.
# * --version vX.Y.Z Pin the release tag (release-mode only). Required
# on first install. "latest" permitted on upgrades.
# * --allow-downgrade Permit ${incoming} < ${prior}. Cosign verify still
Expand Down Expand Up @@ -115,15 +114,15 @@ FIX_LINE_ENDINGS=0
ALLOW_SYSTEM_PREFIX=0
INSTALL_DEPS=0
UPGRADE_DEPS=0
# D3 (#249) — release-mode args. Defaults preserve pre-D3 behavior (source
# mode silent default) until the D4 flip gate clears (one release cycle of
# D2 green + D3 smoke test green on macOS+Linux). See ADR-011.
INSTALL_MODE="" # "" (= source, silent default in D3) | release | source
# Release-mode args (#249). Since the D4 default-flip an unset INSTALL_MODE
# resolves to release (cosign-verified tarball); --from-source is opt-in and
# requires --i-accept-unverified-source. See ADR-011.
INSTALL_MODE="" # "" (resolves to release, D4 default-flip) | release | source
REQUESTED_VERSION="" # vX.Y.Z, X.Y.Z, or "latest" — release mode only
ALLOW_DOWNGRADE=0
ACCEPT_REPUBLISHED_VERSION=0
SKIP_RELEASE_API_CROSSCHECK=0
ACCEPT_UNVERIFIED_SOURCE=0 # no-op in D3; mandatory after D4 default-flip
ACCEPT_UNVERIFIED_SOURCE=0 # mandatory with --from-source since the D4 default-flip (#249)
MIGRATE_TIMEOUT=300 # wall-time limit for the migrate verb; 0 = unbounded

usage() { sed -n '2,80p' "$0" | sed 's/^# \{0,1\}//'; }
Expand Down Expand Up @@ -175,15 +174,25 @@ fi
[[ "${PUBLISH_MODE}" == "fdd" || "${PUBLISH_MODE}" == "scd" ]] \
|| err "--publish-mode must be 'fdd' or 'scd'"

# D3 (#249) — install-mode validation. Silent default in D3 = source (= "")
# for back-compat. After the D4 flip gate clears, the default becomes
# release and --from-source will require --i-accept-unverified-source.
# Install-mode validation (#249). Since the D4 default-flip an unset
# INSTALL_MODE resolves to release (cosign-verified); --from-source is the
# opt-in escape hatch and requires --i-accept-unverified-source.
#
# D3 pre-PR (shell-expert LOW): the arg-parse loop currently lets
# `--from-release --from-source` (or vice versa) silently take the LAST
# value of INSTALL_MODE. Refuse the conflict so the operator's intent is
# never resolved silently.
if [[ "${INSTALL_MODE}" == "release" ]] && [[ "${ACCEPT_UNVERIFIED_SOURCE}" == "1" ]]; then
# D4 default-flip (#249, ADR-011): the default install mode is now the
# cosign-verified release tarball. Resolve an unset INSTALL_MODE to release
# FIRST — so the accept-flag guard below also catches the implicit-default
# case (--i-accept-unverified-source with no mode flag), not just explicit
# --from-release. --from-source is the opt-in escape hatch (else branch).
if [[ -z "${INSTALL_MODE}" ]]; then
INSTALL_MODE="release"
fi
# --i-accept-unverified-source is meaningful only with --from-source. Reject it
# in any non-source (cosign-verified) mode rather than silently ignoring intent.
if [[ "${INSTALL_MODE}" != "source" ]] && [[ "${ACCEPT_UNVERIFIED_SOURCE}" == "1" ]]; then
err "--i-accept-unverified-source is meaningful only with --from-source (release mode is cosign-verified)"
fi
if [[ "${INSTALL_MODE}" == "release" ]]; then
Expand All @@ -202,9 +211,11 @@ else
if (( ALLOW_DOWNGRADE == 1 || ACCEPT_REPUBLISHED_VERSION == 1 || SKIP_RELEASE_API_CROSSCHECK == 1 )); then
err "--allow-downgrade / --accept-republished-version / --skip-release-api-crosscheck are release-mode-only flags"
fi
# Touch ACCEPT_UNVERIFIED_SOURCE so shellcheck sees it referenced; will
# become load-bearing after the D4 default-flip.
: "${ACCEPT_UNVERIFIED_SOURCE}"
# D4 default-flip (#249): --from-source now builds from an unverified local
# tree (no cosign chain), so it must be an explicit, acknowledged choice.
if (( ACCEPT_UNVERIFIED_SOURCE != 1 )); then
err "--from-source builds from an unverified local source tree (no cosign verification). Pass --i-accept-unverified-source to acknowledge the reduced trust posture, or omit --from-source to use the default cosign-verified --from-release path."
fi
fi

# ---------------------------------------------------------------------------
Expand Down Expand Up @@ -1215,7 +1226,7 @@ main() {
# D3 (#249) — post-swap mode/semver/history markers. Written AFTER
# atomic_swap so they only reflect committed installs (a rollback path
# that runs cleanup() before SUCCESS=1 will not have touched them).
rc_write_post_install_markers "${INSTALL_MODE:-source}"
rc_write_post_install_markers "${INSTALL_MODE:-release}"
SUCCESS=1

log "install complete"
Expand Down
Loading
Loading