Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions docs/host-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ This section is the **contract**: which tools a host needs and which repo proced
| `gh` | the PR and review loop, `gh api` queries, `repo-config/configure.sh` | `gh --version` | **2.47.0**, measured |
| Python 3 | `scripts/` and `spec/` (standard library only, no packages to install) | `python3 --version`, or `py -3 --version` on native Windows | **3.13**, target |
| `jq` | the ruleset normalizer in `repo-config/configure.sh`, the ruleset diff in [`AUDIT.md`][audit] section 6, and payload regeneration | `jq --version` | **1.7**, target |
| `docker` | the four linters, which run as pinned images rather than local installs | `docker --version` | none |
| `docker` | the four linters, which run as pinned images rather than local installs | `docker --version` | **29.6.2**, target |
| `uv` / `uvx` | coverage runs, and the Python toolchain (`ruff`, `pyright` or `mypy`) in a Python repo | `uv --version` | **0.12.2**, target |

The **Floor** column exists because presence and sufficiency are different questions and the answer to the first was being read as the answer to the second. A tool below its floor still answers `--version`, so every other column reports it as fine while `scripts/host_gate.py` fails it. The kind is named beside the number, since a **measured** floor sits above a version known to break a documented procedure and gives a failing host a defect to point at, where a **target** floor names the version the repo's toolchain is configured for and does not. The next section carries the reasoning behind each one.
Expand All @@ -39,7 +39,9 @@ Presence is the weaker half of this contract. Both host defects this fleet has a

**`git-restore-mtime` must not come from it either, where a repo uses it.** Debian and Ubuntu package **2022.12**, which shells out to `git whatchanged`. Current `git` refuses that without a hidden opt-in flag a caller cannot pass through, so the tool restores nothing, prints its ordinary statistics and **exits 0**. A deploy keyed on mtimes then ships a full copy and reports success. Take the upstream release from [git-tools][git-tools-link], or in CI the [action][git-restore-mtime-action-link] that vendors it. Note the direction of that interaction: a **newer** `git` is the trigger rather than the remedy, so a host old enough to still allow `whatchanged` hides the defect rather than avoiding it. No procedure in this repo needs the tool, so the gate declares it **optional** and skips it when absent.

**The rest of the table takes the distribution's package, and two more do not.** `git`, `docker` and the Python interpreter come from the distribution, because each keeps up well enough that a second source buys nothing and costs a repository to trust. `jq` is the same on a current Debian or Ubuntu, which carries a version at or above the floor, and the upstream release binary is the answer only where it does not. `uv` is published by its authors as a release archive and packaged by neither distribution, so upstream is the only source there is. Where a repository needs `node`, the distribution's package trails upstream by whole release lines, so it comes from the NodeSource repository on the line upstream currently carries as long term support. Where a repository needs `dotnet`, the distribution's feed is preferred where it carries an SDK and Microsoft's feed is the fallback, because mixing the two is what breaks a host rather than either one alone, and Microsoft's carries `amd64` only.
**`docker` must not come from the distribution's own package either, with one exception.** Debian and Ubuntu package `docker.io`, an older build that trails and conflicts with `docker-ce`, so [`host-setup/linux/install-tools.sh`][host-setup-dir] removes it and installs from Docker's own apt repository at [download.docker.com][docker-install-link] instead, the same shape it already uses for `gh` and `node`. The exception is a **WSL distribution**, where the only sanctioned source is Docker Desktop's own WSL integration (Settings, Resources, WSL integration, on the Windows side, reported read-only by [`setup-wsl.ps1`][host-setup-windows]) and a native install is refused outright, with no override: running `docker-ce` directly inside a WSL distribution risks a second engine beside Desktop's own. On native **Windows**, `winget` already tracks upstream Desktop releases, so neither hazard arises there.

**The rest of the table takes the distribution's package, and one more does not.** `git` and the Python interpreter come from the distribution, because each keeps up well enough that a second source buys nothing and costs a repository to trust. `jq` is the same on a current Debian or Ubuntu, which carries a version at or above the floor, and the upstream release binary is the answer only where it does not. `uv` is published by its authors as a release archive and packaged by neither distribution, so upstream is the only source there is. Where a repository needs `node`, the distribution's package trails upstream by whole release lines, so it comes from the NodeSource repository on the line upstream currently carries as long term support. Where a repository needs `dotnet`, the distribution's feed is preferred where it carries an SDK and Microsoft's feed is the fallback, because mixing the two is what breaks a host rather than either one alone, and Microsoft's carries `amd64` only.

Neither `node` nor `dotnet` is in the table above, deliberately: they serve the repositories that need them rather than the fleet contract, and a repository needing one declares it in a `host-tools.json` of its own, which [`scripts/host_gate.py`][host-gate] merges over this one. The merge tightens only, so a repository may raise a floor or add one and may not lower or remove one.

Expand Down Expand Up @@ -305,6 +307,7 @@ A host that fails any row is not ready for the procedure that row names, and the

[cli-install-link]: https://github.com/cli/cli/blob/trunk/docs/install_linux.md
[cli-link]: https://cli.github.com/
[docker-install-link]: https://docs.docker.com/engine/install/
[git-restore-mtime-action-link]: https://github.com/chetan/git-restore-mtime-action
[git-tools-link]: https://github.com/MestreLion/git-tools
[keys-link]: https://github.com/settings/keys
88 changes: 87 additions & 1 deletion host-setup/linux/install-tools.sh
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ readonly SOURCES_DIR="/etc/apt/sources.list.d"
readonly BIN_DIR="/usr/local/bin"

# Managed tools, in dependency order: node asks jq to read the upstream release index.
readonly TOOLS=(git gh jq git-restore-mtime node python uv dotnet)
readonly TOOLS=(git gh jq git-restore-mtime node python uv docker dotnet)

# Package sets.
# The default set is what a tool needs to be useful, and the optional set is what is useful often enough to name but not always wanted, installed only with --optional.
Expand All @@ -30,6 +30,8 @@ APT_REFRESHED=false
APT_DIRTY=false
DISTRO_ID=""
DISTRO_VERSION=""
CODENAME=""
IS_WSL=false
ARCH=""
SUDO=()
SELECTED=()
Expand Down Expand Up @@ -98,6 +100,7 @@ detect_host() {

DISTRO_ID="${ID:-}"
DISTRO_VERSION="${VERSION_ID:-}"
CODENAME="${VERSION_CODENAME:-}"
if [[ $DISTRO_ID != "debian" && $DISTRO_ID != "ubuntu" ]]; then
Comment thread
ptr727 marked this conversation as resolved.
[[ " ${ID_LIKE:-} " == *" debian "* ]] ||
die "Unsupported distribution \"${DISTRO_ID:-unknown}\", this script installs on Debian and Ubuntu based hosts"
Expand All @@ -108,6 +111,11 @@ detect_host() {
command -v apt-get > /dev/null || die "apt-get not found, this script installs apt packages"
ARCH=$(dpkg --print-architecture)

# WSL has no kernel of its own, and docker there comes only from Docker Desktop's own WSL integration, never a native install.
if grep -qi microsoft /proc/version 2> /dev/null || [[ -n ${WSL_DISTRO_NAME:-} ]]; then
IS_WSL=true
fi

if [[ $EUID -ne 0 ]]; then
command -v sudo > /dev/null || die "Not running as root and sudo is not installed"
SUDO=(sudo)
Expand Down Expand Up @@ -526,6 +534,79 @@ uv_install() {
run_root install -m 0755 "$TMP_DIR/uv-$triple/uvx" "$BIN_DIR/uvx"
}

# --- docker ---

# Matches docker_install: a WSL distribution never reaches download.docker.com, so the report says where docker actually comes from there instead.
docker_source() {
if [[ $IS_WSL == true ]]; then
printf "Docker Desktop's WSL integration"
else
printf 'download.docker.com'
fi
}

# Read directly from the CLI rather than from apt_installed_version docker-ce, unlike gh and node.
# On a WSL distribution using Docker Desktop's own WSL integration, docker is a working command with no docker-ce apt package behind it at all, and reading the apt package version would misreport that working install as absent.
# This also matches exactly what scripts/host_gate.py's own probe and pattern read.
docker_version() {
command -v docker > /dev/null || return 0
docker --version 2> /dev/null | sed -n 's/^Docker version \([0-9][0-9.]*\).*/\1/p'
}

# Stripped of the epoch and the Debian package revision apt_candidate_version otherwise carries (e.g. "5:29.7.2-1~debian.13~trixie"), so this compares like for like against docker_version's plain CLI reading rather than against dpkg's own packaging metadata.
docker_target() {
local raw
raw=$(apt_candidate_version docker-ce)
[[ -z $raw ]] && return 0
raw="${raw#*:}"
printf '%s' "${raw%%-*}"
}

# Old and conflicting packages named here, per Docker's own uninstall list.
# Debian and Ubuntu never ship a package named docker-ce, so unlike gh and node there is no distro package the upstream one could be confused with, and tool_configured needs no entry for it.
docker_install() {
# The only sanctioned source inside a WSL distribution is Docker Desktop's own WSL integration, confirmed with the maintainer as a hard rule with no override.
# A native install here would run a second engine beside Desktop's, so this is always a skip rather than an install, on the same pattern dotnet_feed uses for an architecture Microsoft's feed does not carry.
if [[ $IS_WSL == true ]]; then
warn "This is a WSL distribution, and docker here comes only from Docker Desktop's own WSL integration, never from installing docker-ce directly. Enable it in Docker Desktop under Settings, Resources, WSL integration, or check it from Windows with setup-wsl.ps1 -Status. Skipping the native install."
# A skip is success only where the integration already answers, since --install/--upgrade otherwise exits 0 having neither installed docker nor found it working.
command -v docker > /dev/null && return 0
warn "docker is not on PATH here either, so Docker Desktop's WSL integration is not enabled for this distribution yet."
return 1
fi
[[ -n $CODENAME ]] ||
die "/etc/os-release names no VERSION_CODENAME, so the Docker apt repository's suite cannot be worked out"

local -a conflicts=(docker.io docker-doc docker-compose docker-compose-v2 docker-buildx podman-docker containerd runc)
local -a present=()
local pkg
for pkg in "${conflicts[@]}"; do
package_installed "$pkg" && present+=("$pkg")
done
if [[ ${#present[@]} -gt 0 ]]; then
log " Removing ${#present[@]} conflicting package(s): ${present[*]}"
run_root apt-get remove -y "${present[@]}"
fi

ensure_prerequisites
remove_stale "$SOURCES_DIR/docker.list"

if install_keyring "https://download.docker.com/linux/$DISTRO_ID/gpg" \
"$KEYRING_DIR/docker.gpg" \
"https://download.docker.com/linux/$DISTRO_ID/dists/$CODENAME/InRelease" true; then
APT_DIRTY=true
fi

if write_sources "docker" "https://download.docker.com/linux/$DISTRO_ID" "$CODENAME" "stable" \
"$KEYRING_DIR/docker.gpg"; then
APT_DIRTY=true
fi
Comment thread
ptr727 marked this conversation as resolved.

apt_install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

# Non-root use (usermod -aG docker $USER) is left to the operator, the same way this file leaves orphaned dependencies to a later apt autoremove: it is a user/group choice rather than a question of whether the tool is present and current.
}

# --- dotnet ---

dotnet_source() {
Expand Down Expand Up @@ -705,6 +786,11 @@ tool_note() {
note "$tool" "upstream repository not configured, the available version is the distro's"
fi
;;
docker)
if [[ $IS_WSL == true ]]; then
note "docker" "this is a WSL distribution, docker here comes only from Docker Desktop's own WSL integration, never from installing docker-ce directly, so --install/--upgrade skip it"
fi
;;
*) ;;
esac

Expand Down
4 changes: 3 additions & 1 deletion host-setup/windows/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,8 @@ pwsh -NoProfile -File ..\bootstrap.ps1 -Help

Docker's own `docker-desktop` distribution is excluded from every distribution listing, since it is Docker's rather than one an operator installed.

`install-tools.ps1` checks, before installing or upgrading `docker`, that `wsl.exe` is present and reports a WSL version at or above `2.1.5`, Docker Desktop's own documented floor for the platform it depends on. Where either is not the case it skips `docker` and names the exact remedy (`wsl --install --no-distribution`, or `upgrade-host.ps1 -Wsl`) rather than installing against a platform Docker Desktop cannot use, or Windows-feature-installing on the caller's behalf. It never runs `wsl --install` or `wsl --update` itself: those stay `upgrade-host.ps1 -Wsl` and a person's own choice, since an update restarts every distribution and neither belongs as a side effect of installing a different tool. The same check surfaces as a note under `-Report`, read-only, before a caller ever runs `-Install`.

## Differences From the Linux Tooling

| Linux | Windows | Why |
Expand All @@ -88,7 +90,7 @@ Docker's own `docker-desktop` distribution is excluded from every distribution l
| `install-tools.sh` carries four functions per tool | `install-tools.ps1` carries one registry record per tool | Every source is `winget`, so the per-tool variation those functions exist for does not arise |
| Actions, the last one given wins | Actions, name one | A `param()` block records which switches were given and not their order, and refusing beats silently discarding an intent |
| `git-restore-mtime` is managed | not managed | The spec declares it not applicable on Windows, since it serves a Linux deploy path |
| `docker` is not managed | `docker` is managed | `Docker.DockerDesktop` is one winget package, where the Linux answer differs by host role |
| `install-tools.sh` refuses docker entirely inside a WSL *distribution* | `install-tools.ps1` checks the WSL *platform* version before installing docker | A WSL distribution takes docker only from Docker Desktop's own WSL integration, and Windows needs WSL2 present for Docker Desktop's own backend |
| `sudo` re-runs a command as root | nothing elevates | `winget` raises UAC per installer, which is the path with the fewest failures |
| `unmanaged` means the upstream repository is unconfigured | `unmanaged` means the tool is on `PATH` and winget knows no package for it | The same question, by a different mechanism |
| `credential.helper cache --timeout=3600` | `credential.helper manager`, and only where unset | Git Credential Manager ships with Git for Windows |
Expand Down
64 changes: 64 additions & 0 deletions host-setup/windows/install-tools.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -434,6 +434,59 @@ function Add-ToolNote {
if ($Tool.Name -eq 'dotnet' -and -not $script:WITH_OPTIONAL) {
note 'dotnet' "optional set not selected: $($Tool.Optional -join ', ')"
}
if ($Tool.Name -eq 'docker') {
$wslProblem = Test-WslReadyForDocker
if ($wslProblem) { note 'docker' $wslProblem }
}
}

# --- WSL ---

# Docker Desktop's own documented floor for the WSL platform, per docs.docker.com/desktop/features/wsl.
$DOCKER_WSL_FLOOR = '2.1.5'

# Every wsl.exe call goes through here, because wsl.exe emits UTF-16 by default and its output then reads as NUL separated characters.
# Mirrored from upgrade-host.ps1 rather than shared with it, on the same rule as the rest of this directory: a script here has to stay independently fetchable.
function Invoke-Wsl {
param([Parameter(ValueFromRemainingArguments)][string[]]$Arguments)
$previous = $env:WSL_UTF8
try {
$env:WSL_UTF8 = '1'
$text = (& wsl.exe @Arguments 2>&1 | Out-String -Width 500)
if ($text.Contains([char]0)) { $text = $text -replace "`0", '' }
return $text
} finally {
if ($null -eq $previous) { Remove-Item Env:\WSL_UTF8 -ErrorAction SilentlyContinue }
else { $env:WSL_UTF8 = $previous }
}
}

# Just the "WSL version" line, as a value Compare-HostVersion can read.
# Get-WslVersion in upgrade-host.ps1 concatenates WSL, kernel and WSLg into one display string instead, which serves a report rather than a comparison.
function Get-WslPlatformVersion {
if (-not (Get-Command wsl.exe -ErrorAction SilentlyContinue)) { return $null }
$text = Invoke-Wsl '--version'
if ($LASTEXITCODE -ne 0) { return $null }
if ($text -match '(?m)^WSL version:\s*(\S+)\s*$') { return $Matches[1] }
return $null
}

# What stands between this host and installing or upgrading docker, or $null where nothing does.
# Read-only: this never runs wsl --install or wsl --update itself.
# Those stay a person's own action through upgrade-host.ps1 -Wsl and setup-wsl.ps1, confirmed with the maintainer as the boundary.
# A WSL platform update restarts every distribution, and neither action belongs as a silent side effect of installing a different tool.
function Test-WslReadyForDocker {
if (-not (Get-Command wsl.exe -ErrorAction SilentlyContinue)) {
return "wsl.exe was not found, and Docker Desktop needs WSL2. Install it with: wsl --install --no-distribution"
}
$version = Get-WslPlatformVersion
if (-not $version) {
return "the WSL version could not be read, and Docker Desktop needs WSL $($script:DOCKER_WSL_FLOOR) or later. Update it with: host-setup\windows\upgrade-host.ps1 -Wsl"
}
if ((Compare-HostVersion $version $script:DOCKER_WSL_FLOOR) -lt 0) {
return "WSL is at $version, and Docker Desktop needs $($script:DOCKER_WSL_FLOOR) or later. Update it with: host-setup\windows\upgrade-host.ps1 -Wsl"
}
return $null
}

# --- Actions ---
Expand Down Expand Up @@ -494,6 +547,17 @@ function Invoke-ToolApply {
return
}

# Docker Desktop needs WSL2 already present and current, and does not install or update it itself.
# A WSL gap has nothing to do with any other tool in this run, so it is collected here on the same rule as a failed install rather than ending the whole run.
if ($ToolName -eq 'docker') {
$wslProblem = Test-WslReadyForDocker
if ($wslProblem) {
warn "docker skipped, $wslProblem"
$script:FAILED += $ToolName
return
}
}

# Naming a scope the installed copy does not sit in would add a second copy beside it, so the removal is asked for rather than done on the way past.
if ($script:WANT_SCOPE -and $state.Rows.Count -gt 0 -and $state.Scope.Count -gt 0 -and
$state.Scope -notcontains $script:WANT_SCOPE -and $script:MODE -ne 'reinstall') {
Expand Down
Loading