Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/host-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,9 @@ Presence is the weaker half of this contract. Both host defects this fleet has a

**`docker`'s floor is read from the engine, not from the CLI banner.** The `docker` on `PATH` inside a WSL distribution can be a separately packaged client that talks to Docker Desktop's engine over the integration socket, and the two then carry different versions: a host was recorded with a distribution client at `29.1.3` against an engine at `29.7.2`, so `docker --version` failed a floor the engine cleared comfortably. The gate therefore asks the daemon first, with `docker version --format '{{.Server.Version}}'`, and falls back to the `docker --version` banner only where that exits non-zero, which is what a stopped or unreachable daemon does. The **Present when** column above still names the banner, because presence is what it answers and a host with the daemon stopped still has `docker` installed.

**The rest of the table takes the distribution's package, and one more does not.** `git` and the Python interpreter come from the distribution, because each keeps up well enough that a second source buys nothing and costs a repository to trust. `jq` is the same on a current Debian or Ubuntu, which carries a version at or above the floor, and the upstream release binary is the answer only where it does not. `uv` is published by its authors as a release archive and packaged by neither distribution, so upstream is the only source there is. Where a repository needs `node`, the distribution's package trails upstream by whole release lines, so it comes from the NodeSource repository on the line upstream currently carries as long term support. Where a repository needs `dotnet`, the distribution's feed is preferred where it carries an SDK and Microsoft's feed is the fallback, because mixing the two is what breaks a host rather than either one alone, and Microsoft's carries `amd64` only.
**The rest of the table takes the distribution's package, and one more does not.** `git` and the Python interpreter come from the distribution, because each keeps up well enough that a second source buys nothing and costs a repository to trust. `jq` is the same on a current Debian or Ubuntu, which carries a version at or above the floor, and the upstream release binary is the answer only where it does not. `uv` is published by its authors as a release archive and packaged by neither distribution, so upstream is the only source there is. Where a repository needs `node`, the distribution's package trails upstream by whole release lines, so it comes from the NodeSource repository on the line upstream currently carries as long term support. Where a repository needs `dotnet`, the distribution's feed is preferred where it carries an SDK and Microsoft's feed is the fallback, because mixing the two is what breaks a host rather than either one alone, and Microsoft's carries `amd64` only. Where a repository needs `pwsh`, that same feed is the only source, since no distribution packages it, and the tooling installs it as the `powershell` tool under `--optional`, with `pwsh` the command that tool provides.

Neither `node` nor `dotnet` is in the table above, deliberately: they serve the repositories that need them rather than the fleet contract, and a repository needing one declares it in a `host-tools.json` of its own, which [`scripts/host_gate.py`][host-gate] merges over this one. The merge tightens only, so a repository may raise a floor or add one and may not lower or remove one.
Neither `node`, `dotnet`, nor `pwsh` is in the table above, deliberately: they serve the repositories that need them rather than the fleet contract, and a repository needing one declares it in a `host-tools.json` of its own, which [`scripts/host_gate.py`][host-gate] merges over this one. The merge tightens only, so a repository may raise a floor or add one and may not lower or remove one.

**A host being stood up needs no Python.** The tooling under [`host-setup/`][host-setup-dir] is shell and PowerShell, deliberately, because requiring an interpreter to upgrade a package or install a tool would make the first step of standing a host up depend on the thing that step exists to provide. The Python floor above is a development requirement, meaning [`scripts/`][scripts-dir] and [`spec/`][spec-dir], and a host that only runs services never has to meet it. `bootstrap.sh` needs `curl` and `tar`, both of which a base install carries or can install without a network tool of its own. `bootstrap.ps1` needs only `tar.exe`, which has shipped with Windows since 1803, and installs its one further dependency, PowerShell 7, itself through `winget`. The one exception is the skills step at the end of a stand-up, which drives the Python installer in [`scripts/`][scripts-dir], and it runs last for exactly that reason: `install-tools` has provided the interpreter by then, and run alone on a host without one it stops and names the tools step as its prerequisite.

Expand Down
6 changes: 4 additions & 2 deletions host-setup/linux/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ Each script is LF with a shebang, and its executable bit is tracked in git. A fr
The Windows registry has one source because `winget` tracks upstream. Here the distro package trails upstream on `gh`, on `node`, and on `uv`, so a tool comes from whichever source keeps up:

- **The distro**, for `git` and `python`, where apt's own package is current enough.
- **An upstream apt repository**, for `gh`, `node`, and `docker`, and for `dotnet` as a fallback, where upstream publishes one.
- **An upstream apt repository**, for `gh`, `node`, `docker`, and `powershell`, and for `dotnet` as a fallback, where upstream publishes one.
- **A released binary into `/usr/local/bin`**, for `jq`, `uv`, and `git-restore-mtime`, where upstream publishes no repository.

No version is written into the script. Each upstream is asked what it carries now, so the script does not go stale between releases. Every step is idempotent, so a keyring or sources file is written only when its content differs. A re-run repairs drift rather than assuming a clean host.
Expand All @@ -59,14 +59,16 @@ A report changes nothing and reads the apt cache as it stands. An available vers

An install or upgrade collects a tool whose install fails and carries on, so one failure does not strand the rest of the run. A refusal is different and ends the run. An unverifiable keyring, a checksum mismatch, or a declined prompt stops everything, because continuing past one would install something nobody vouched for.

## Docker, node, and dotnet
## Docker, node, dotnet, and powershell

**Inside a WSL distribution, docker comes only from Docker Desktop's own WSL integration, never from installing `docker-ce`.** A native install would run a second engine beside Desktop's. `--install` and `--upgrade` therefore always skip it there and point at Docker Desktop's settings instead. The skip counts as success only where `docker` already answers, so a run cannot exit clean having found nothing working. On a native host, the conflicting packages Docker's own uninstall list names are removed first. Non-root use (`usermod -aG docker`) is left to the operator, as a group choice rather than a question of presence.

**Installing `node` displaces distro packages.** The upstream package carries `npm` itself and conflicts with the distro's `npm` and `nodejs-doc`. The script asks apt what it would remove and puts that list in front of the operator first. Asking apt beats naming the conflicts here, because the conflict set belongs to the upstream package and changes without notice. The major line installed is whatever upstream currently marks LTS, read from its release index at run time.

**For `dotnet`, the distro feed is the default and Microsoft's feed is the fallback.** The fallback is added only where the distro carries no SDK at all, because mixing the two feeds is what breaks a host. Microsoft's feed carries amd64 only, so any other architecture without a distro SDK is a named skip. The default set is the newest SDK line the feed carries. `--optional` adds every other line, for a host that builds against more than one.

**`powershell` is optional and comes from Microsoft's feed, the same one `dotnet` falls back to.** The distro never carries a package, so there is no distro version to prefer and no mixing concern to weigh. It joins the default selection only under `--optional`, and naming it on the command line selects it either way. Like `dotnet`, the feed carries amd64 only, so any other architecture is a named skip. It is not part of the fleet contract, which is why it stays optional: a repository that needs `pwsh` opts in rather than every host carrying it.

## The Sudo Credential Cache

`install-tools.sh --sudo-timestamp` shares one sudo credential cache across the invoking user's terminals. Sudo's default is one cache per terminal, so a `sudo -v` answered in one terminal does nothing for a program started in another. The action installs no tool and reports on none. It sits here because a host stand-up already runs this script.
Expand Down
59 changes: 55 additions & 4 deletions host-setup/linux/install-tools.sh
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,8 @@ readonly SUDOERS_FILE="/etc/sudoers.d/90-host-setup-sudo-timestamp"
readonly SUDO_TIMESTAMP_TIMEOUT=60

# Managed tools, in dependency order: node asks jq to read the upstream release index.
readonly TOOLS=(git gh jq git-restore-mtime node python uv docker dotnet)
# The one optional member, powershell, joins the default selection only under --optional.
readonly TOOLS=(git gh jq git-restore-mtime node python uv docker dotnet powershell)

# Package sets.
# The default set is what a tool needs to be useful, and the optional set is what is useful often enough to name but not always wanted, installed only with --optional.
Expand Down Expand Up @@ -66,7 +67,8 @@ usage() {
Usage: install-tools.sh [options] [tool ...]

Installs the host tools the fleet's repositories expect, from upstream where the distro package
trails upstream. With no tool named, every managed tool is selected.
trails upstream. With no tool named, every managed tool is selected, and the optional tool
joins it under --optional.

Actions, the last one given wins, default --report:
-r, --report Report installed and available versions, change nothing
Expand All @@ -80,7 +82,7 @@ Actions, the last one given wins, default --report:
Options:
-n, --dry-run Print the commands instead of running them
-y, --yes Do not prompt before changing the host
-o, --optional Include the optional package set, where a tool has one
-o, --optional Include the optional tool and the optional package sets, where a tool has one

Versions read as apt versions for an apt-managed tool and as upstream versions for a standalone
binary, so a column compares like with like. A report reads the apt cache as it stands and does
Expand Down Expand Up @@ -675,6 +677,17 @@ dotnet_feed() {
return 1
fi

microsoft_feed
}

# Register Microsoft's apt repository, the feed powershell always needs and dotnet falls back to when the distro carries no SDK.
# The upstream deb is the whole registration, keyring and sources file in one, so this installs it rather than reimplementing it with install_keyring and write_sources.
microsoft_feed() {
# The feed already configured is the fast path, so a repeat run or a run selecting both dotnet and powershell does not re-install the deb and force the apt refresh this file reserves for a changed sources file.
if [[ -f "$SOURCES_DIR/microsoft-prod.sources" || -f "$SOURCES_DIR/microsoft-prod.list" ]]; then
return 0
fi

local deb="$TMP_DIR/packages-microsoft-prod.deb"
fetch -o "$deb" "https://packages.microsoft.com/config/$DISTRO_ID/$DISTRO_VERSION/packages-microsoft-prod.deb" ||
die "Microsoft publishes no feed for $DISTRO_ID $DISTRO_VERSION"
Expand Down Expand Up @@ -702,6 +715,24 @@ dotnet_install() {
done < <(dotnet_sdk_packages)
}

# --- powershell ---

powershell_source() { printf 'packages.microsoft.com'; }

powershell_version() { apt_installed_version powershell; }
powershell_target() { apt_candidate_version powershell; }

# The distro never carries a powershell package, so Microsoft's feed is the only source, and there is no dotnet-style mixing concern to decide against.
powershell_install() {
if [[ $ARCH != "amd64" ]]; then
warn "Microsoft's feed carries amd64 only, skipping powershell on $ARCH"
return 0
fi

microsoft_feed
apt_install powershell
}

# --- Tool dispatch ---

tool_function() {
Expand Down Expand Up @@ -814,6 +845,12 @@ tool_note() {
note "docker" "this is a WSL distribution, docker here comes only from Docker Desktop's own WSL integration, never from installing docker-ce directly, so --install/--upgrade skip it"
fi
;;
powershell)
# A host without the feed reads no candidate at all, since the distro never carries a powershell package, so the note says why rather than reading as absent.
if [[ ! -f "$SOURCES_DIR/microsoft-prod.sources" && ! -f "$SOURCES_DIR/microsoft-prod.list" ]]; then
note "powershell" "Microsoft's repository is not configured, so no package is available yet"
fi
;;
*) ;;
esac

Expand Down Expand Up @@ -991,7 +1028,12 @@ list_tools() {
log "Managed tools:"
local tool
for tool in "${TOOLS[@]}"; do
printf ' %-18s %s\n' "$tool" "$("$(tool_function "$tool" source)")"
# The one optional member is marked here, since --list shows the whole registry, and its line says what the default selection leaves out.
if [[ $tool == "powershell" ]]; then
printf ' %-18s %s (optional)\n' "$tool" "$("$(tool_function "$tool" source)")"
else
printf ' %-18s %s\n' "$tool" "$("$(tool_function "$tool" source)")"
fi
done
log ""
log "Package sets:"
Expand Down Expand Up @@ -1306,6 +1348,15 @@ parse_args() {
done
else
SELECTED=("${TOOLS[@]}")
if [[ $WITH_OPTIONAL == false ]]; then
# The optional member joins the default selection only under --optional, while naming it on the command line still selects it, since an explicit request is never dropped.
local -a filtered=()
local tool
for tool in "${TOOLS[@]}"; do
[[ $tool == "powershell" ]] || filtered+=("$tool")
done
SELECTED=("${filtered[@]}")
fi
fi
}

Expand Down