diff --git a/TODO.md b/TODO.md index cb930e5f..f2e942a7 100644 --- a/TODO.md +++ b/TODO.md @@ -442,6 +442,24 @@ Three findings raised while writing [`host-setup/windows/`][host-setup-windows], - **Settled** - The Windows tooling already refuses this way. That began as a constraint, since a PowerShell `param()` block records which switches were given and not their order, and the constraint produced the better behavior. - **Open** - Nothing about the change itself, which is three `usage()` heredocs and three `parse_args()` bodies. The decision is only whether the fleet wants the stricter contract, and taking it deletes the differences-table row in [`host-setup/windows/README.md`][host-setup-windows] rather than leaving a permanent divergence. +### Neither Host Bootstrap Has Run Against a Truly Fresh Host + +Two loaders exist so a copy-paste snippet takes a stock OS install to a configured dev host, and neither has ever been run that way. Everything either has behind it is a dry run or a read against an already-configured checkout, on a machine carrying most of the target tools already. That confirms the logic is internally consistent. It confirms nothing about a `winget` package id still resolving, a stock Debian netinst actually lacking `curl` the way the docs assume, `tar.exe` genuinely shipping on a given Windows image, or the interactive menu reading correctly on a real console. This needs a human watching a real run on a real fresh image and reporting back what broke, including anything that merely looked fine, since neither of those closes from a description of the logic. + +**State** `ready`. **Touches** [`host-setup/bootstrap.sh`][bootstrap] and [`host-setup/bootstrap.ps1`][bootstrap-ps1] and, if either run turns something up, whichever script under [`host-setup/linux/`][install-tools] or [`host-setup/windows/`][host-setup-windows] it drives. **Cost** VM time on the images each entry names, and an iteration round trip per finding, since a fix this file cannot verify is a fix that needs the same fresh image again. + +- **Run `bootstrap.sh` unattended against a fresh Debian and a fresh Ubuntu image, and again to confirm the second run is idempotent.** `--host --yes` finishing clean, with nothing to fix, is the signal. A re-run reporting no further changes confirms idempotency rather than assuming it. + - **Blocked by** - VM access to a current image of each, and one still-supported older release per distribution, since the contract's floors are meant to hold there too. + - **Issue** - None filed. + - **Checked** - `develop` at `82a87d3` on 2026-08-13, where this loader has existed since #674 and carries no record of a run against an image with nothing preinstalled. + - **Open** - Which images, who runs the pass, and whether a failure blocks the loader or is filed and worked separately, since a fresh-host pass can turn up findings well past what one pull request should carry. + +- **Run `bootstrap.ps1` unattended against a fresh Windows 10 image with no App Installer, and a fresh Windows 11 image, then again on each to confirm idempotency.** The Windows 10 case is the one that exercises the winget-missing remedy this loader prints but has never had checked against a real console. Windows 11 is the expected common case, App Installer and `winget` both present. `-Host -Yes` finishing clean on each, then a clean re-run, is the same signal as the Linux entry above. + - **Blocked by** - VM access to both images. + - **Issue** - None filed. + - **Checked** - Branch `feature/windows-bootstrap-loader` on 2026-08-13, adding this loader for the first time. It has run under `-DryRun` and against `PSScriptAnalyzer` on a dev machine that already carries `pwsh`, `winget`, and most managed tools, which is signal on the script's internal consistency and none at all on whether it survives a host it has not touched. + - **Open** - Same as the Linux entry: which images, who runs the pass, and how a finding routes back. + ## Standalone Chores Small work with no research to preserve, selectable one bullet at a time. @@ -590,6 +608,8 @@ Nothing is awaiting close today. [#578][issue-578] was the last entry here and c [agents]: ./AGENTS.md [audit]: ./spec/audit.py [audit-doc]: ./AUDIT.md +[bootstrap]: ./host-setup/bootstrap.sh +[bootstrap-ps1]: ./host-setup/bootstrap.ps1 [codestyle]: ./CODESTYLE.md [copilot-instructions]: ./.github/copilot-instructions.md [divergences]: ./spec/divergences.json diff --git a/docs/host-setup.md b/docs/host-setup.md index 43c48954..3eb46d5c 100644 --- a/docs/host-setup.md +++ b/docs/host-setup.md @@ -43,9 +43,9 @@ Presence is the weaker half of this contract. Both host defects this fleet has a Neither `node` nor `dotnet` is in the table above, deliberately: they serve the repositories that need them rather than the fleet contract, and a repository needing one declares it in a `host-tools.json` of its own, which [`scripts/host_gate.py`][host-gate] merges over this one. The merge tightens only, so a repository may raise a floor or add one and may not lower or remove one. -**A host being stood up needs no Python.** The tooling under [`host-setup/`][host-setup-dir] is shell, deliberately, because requiring an interpreter to upgrade a package or install a tool would make the first step of standing a host up depend on the thing that step exists to provide. The Python floor above is a development requirement, meaning [`scripts/`][scripts-dir] and [`spec/`][spec-dir], and a host that only runs services never has to meet it. `bootstrap.sh` needs `curl` and `tar`, both of which a base install carries or can install without a network tool of its own. +**A host being stood up needs no Python.** The tooling under [`host-setup/`][host-setup-dir] is shell and PowerShell, deliberately, because requiring an interpreter to upgrade a package or install a tool would make the first step of standing a host up depend on the thing that step exists to provide. The Python floor above is a development requirement, meaning [`scripts/`][scripts-dir] and [`spec/`][spec-dir], and a host that only runs services never has to meet it. `bootstrap.sh` needs `curl` and `tar`, both of which a base install carries or can install without a network tool of its own. `bootstrap.ps1` needs only `tar.exe`, which has shipped with Windows since 1803, and installs its one further dependency, PowerShell 7, itself through `winget`. -**Standing a host up.** [`host-setup/`][host-setup-dir] carries the tooling that makes a host satisfy this contract, and its README is the usage. A host with nothing runs [`host-setup/bootstrap.sh`][bootstrap], which fetches this repository and runs that tooling from the fetched tree. A native Windows host runs the PowerShell peers in [`host-setup/windows/`][host-setup-windows] from a checkout instead, since no loader reaches those yet. It is not called by [`scripts/host_gate.py`][host-gate] and it does not call it: the gate measures a host against the floors above, and the tooling is a remedy a person chooses when the gate reports a gap. +**Standing a host up.** [`host-setup/`][host-setup-dir] carries the tooling that makes a host satisfy this contract, and its README is the usage. A host with nothing runs [`host-setup/bootstrap.sh`][bootstrap], which fetches this repository and runs that tooling from the fetched tree. A native Windows host with nothing runs [`host-setup/bootstrap.ps1`][bootstrap-ps1] the same way, which finds or installs PowerShell 7 before it fetches anything, since every script under [`host-setup/windows/`][host-setup-windows] requires it. Neither is called by [`scripts/host_gate.py`][host-gate] and neither calls it: the gate measures a host against the floors above, and the tooling is a remedy a person chooses when the gate reports a gap. A repository that needs more than the fleet does adds its own `host-tools.json` at its root, which the gate layers over the hub's. It may add a tool nobody else uses, raise a floor, or turn an optional tool required. It may **not** lower a floor or turn a required tool optional, since those edits retire a fleet check from inside the repository it protects, and the gate reports a rejected relaxation rather than dropping it. @@ -265,6 +265,7 @@ A host that fails any row is not ready for the procedure that row names, and the [agent-safety]: ../host-setup/agent-safety/README.md [audit]: ../AUDIT.md [bootstrap]: ../host-setup/bootstrap.sh +[bootstrap-ps1]: ../host-setup/bootstrap.ps1 [devcontainer]: ./devcontainer.md [governance-git-and-commit-rules]: ../GOVERNANCE.md#git-and-commit-rules [host-gate]: ../scripts/host_gate.py diff --git a/host-setup/README.md b/host-setup/README.md index d42937e4..9353ccc7 100644 --- a/host-setup/README.md +++ b/host-setup/README.md @@ -4,14 +4,15 @@ What a machine needs before it can be worked in, and the tooling that puts it th ## What Is Here -- [`bootstrap.sh`][bootstrap] stands a host up from nothing. It is the one file fetched on its own, because a host with no git and no checkout is what it exists to fix. It fetches this repository and runs the tooling from that tree. +- [`bootstrap.sh`][bootstrap] stands a Debian or Ubuntu host up from nothing. It is the one file fetched on its own, because a host with no git and no checkout is what it exists to fix. It fetches this repository and runs the tooling from that tree. +- [`bootstrap.ps1`][bootstrap-ps1] does the same for native Windows. It runs under Windows PowerShell 5.1, the version every fresh Windows host guarantees, and hands off to PowerShell 7 once it has found or installed it, since every script it drives requires that version. - [`linux/`][linux] holds the tooling itself, for Debian and Ubuntu based hosts, Proxmox and WSL included. `install-tools.sh` installs and upgrades the host tools, `upgrade-host.sh` upgrades the packages of the current release or moves to the next one, and `setup-github.sh` configures the SSH key, git, and commit signing. - [`windows/`][windows] holds the tooling for native Windows, through `winget` and PowerShell 7. `install-tools.ps1` installs and upgrades the host tools, `upgrade-host.ps1` upgrades the winget packages and updates the WSL platform, `setup-github.ps1` configures the SSH key, git, and commit signing, and `setup-wsl.ps1` installs a WSL distribution and reports the Docker Desktop integration. - [`agent-safety/`][agent-safety] holds the write-safety guards, deployed per machine and per account. ## Standing a Host Up -Three lines, on a host that has nothing: +Three lines, on a Debian or Ubuntu host that has nothing: ```shell sudo apt-get update && sudo apt-get install -y curl ca-certificates tar @@ -30,6 +31,23 @@ Piping it is still detected: with no action and no terminal it reports rather th ./bootstrap.sh --help # every action and option ``` +Three lines as well, on a native Windows host that has nothing, pasted into a stock `powershell.exe` console: + +```powershell +[Net.ServicePointManager]::SecurityProtocol = [Net.ServicePointManager]::SecurityProtocol -bor [Net.SecurityProtocolType]::Tls12 +Invoke-WebRequest -UseBasicParsing -Uri https://raw.githubusercontent.com/ptr727/ProjectTemplate/main/host-setup/bootstrap.ps1 -OutFile bootstrap.ps1 +powershell -ExecutionPolicy Bypass -File bootstrap.ps1 +``` + +The first line is the Windows counterpart of installing `curl`: a fresh console's default TLS floor can predate 1.2, which every host this fetches from requires. The third runs under the `powershell.exe` a fresh host guarantees rather than `pwsh`, since `bootstrap.ps1` finds or installs PowerShell 7 itself and hands the rest of the run to it. `-ExecutionPolicy Bypass` clears both the default `Restricted` policy and the mark of the web `Invoke-WebRequest` leaves on the file, in one flag. + +```powershell +.\bootstrap.ps1 -Report # what each tool would do, changing nothing +.\bootstrap.ps1 -Host -Yes # packages, tools, git and GitHub, unattended +.\bootstrap.ps1 -Ref develop -Report # run the tooling as it is on develop +.\bootstrap.ps1 -Help # every action and option +``` + Each tool also runs on its own, on a host that already has a checkout: ```shell @@ -49,8 +67,6 @@ host-setup\windows\setup-github.ps1 -Status host-setup\windows\setup-wsl.ps1 -Status ``` -There is no `bootstrap.ps1`, so a Windows host obtains this repository first. The problem `bootstrap.sh` solves is a host with no git and no checkout, and the Windows form of that problem has no one-liner anybody here has run, so none is offered. - ## Which Revision a Run Used `bootstrap.sh` resolves the ref it was given to the commit it names, prints that commit, and downloads that exact revision. A run therefore says which revision of the tooling it used, and a second run of the same ref cannot silently be a different tree. Where the resolve fails, which an unauthenticated rate limit can cause, the run says it cannot attribute itself and continues, since the download itself is unaffected. @@ -59,17 +75,18 @@ There is no `bootstrap.ps1`, so a Windows host obtains this repository first. Th ## Three Rules This Directory Follows -**Group by whichever axis has one member.** `agent-safety/` is one concern across three platforms, so it is a concern directory holding `install.sh`, `install.ps1` and `install.py`. `linux/` is three concerns on one platform, so it is a platform directory. Windows host tooling therefore sits at `windows/` rather than beside the Linux scripts, because the `winget` equivalent of `install-tools.sh` is a different program rather than a translation of one. It carries one registry record per tool where the Linux script carries four functions, since every Windows source is `winget` and the per-tool variation those functions exist for does not arise. `windows/` also carries a fourth script with no Linux peer, because WSL is a Windows-side concern. +**Group by whichever axis has one member.** `agent-safety/` is one concern across three platforms, so it is a concern directory holding `install.sh`, `install.ps1` and `install.py`. `linux/` is three concerns on one platform, so it is a platform directory. Windows host tooling therefore sits at `windows/` rather than beside the Linux scripts, because the `winget` equivalent of `install-tools.sh` is a different program rather than a translation of one. It carries one registry record per tool where the Linux script carries four functions, since every Windows source is `winget` and the per-tool variation those functions exist for does not arise. `windows/` also carries a fourth script with no Linux peer, because WSL is a Windows-side concern. The loader is the same shape as `agent-safety/`, not as `linux/`/`windows/`: one concern, two platforms, so `bootstrap.ps1` sits beside `bootstrap.sh` at the top level rather than inside `windows/`. -**Nothing here needs Python, and `bootstrap.sh` needs only `curl`.** [`docs/host-setup.md`][host-setup] carries that as part of the contract, with the reasoning. It is why `bootstrap.sh` runs no gate as a closing step: [`scripts/host_gate.py`][host-gate] measures a host against the floors and is not called from here, and nothing here is called from it. A host set up by hand years ago is an ordinary host, so the gate reports what it is missing and running this tooling is a remedy a person chooses. The two are joined at code time instead, by [`scripts/test_bootstrap.py`][test-bootstrap] asserting that every tool the spec requires is one this tooling can provide. +**Nothing here needs Python, and neither loader needs an interpreter to fetch what it drives.** [`docs/host-setup.md`][host-setup] carries that as part of the contract, with the reasoning. `bootstrap.sh` needs only `curl` and `tar`. `bootstrap.ps1` needs only `tar.exe`, which has shipped with Windows since 1803, and installs its one further dependency, `pwsh`, itself through `winget`. Neither runs a gate as a closing step: [`scripts/host_gate.py`][host-gate] measures a host against the floors and is not called from here, and nothing here is called from it. A host set up by hand years ago is an ordinary host, so the gate reports what it is missing and running this tooling is a remedy a person chooses. The two are joined at code time instead, by [`scripts/test_bootstrap.py`][test-bootstrap] asserting that every tool the spec requires is one this tooling can provide. -**The scripts under `linux/` and `windows/` share no file, and the duplication is deliberate.** Each is independently fetchable and runnable on its own, which is the property that lets a host with no checkout use one without the others. A shared helper file would take that away: the moment one script sources a sibling, fetching it alone yields a script that dies on a missing file. What is duplicated is about thirty lines each of logging, the dry-run wrapper, the confirmation prompt, and a temporary directory, and those copies are identical rather than merely similar. Do not factor them out. On the Windows side the fetchability argument is one no loader exercises yet, and the duplication is kept anyway so a loader added later inherits the property rather than having to introduce it. +**The scripts under `linux/` and `windows/` share no file, and the duplication is deliberate.** Each is independently fetchable and runnable on its own, which is the property that lets a host with no checkout use one without the others. A shared helper file would take that away: the moment one script sources a sibling, fetching it alone yields a script that dies on a missing file. What is duplicated is about thirty lines each of logging, the dry-run wrapper, the confirmation prompt, and a temporary directory, and those copies are identical rather than merely similar. Do not factor them out. `bootstrap.ps1` now exercises the same fetchability argument `bootstrap.sh` always has, rather than merely being written to allow for it. [agent-safety]: ./agent-safety/ [audit]: ../AUDIT.md [bootstrap]: ./bootstrap.sh +[bootstrap-ps1]: ./bootstrap.ps1 [host-gate]: ../scripts/host_gate.py [host-setup]: ../docs/host-setup.md [linux]: ./linux/ diff --git a/host-setup/bootstrap.ps1 b/host-setup/bootstrap.ps1 new file mode 100644 index 00000000..81bebc4d --- /dev/null +++ b/host-setup/bootstrap.ps1 @@ -0,0 +1,424 @@ +# Stands a host up from nothing, by fetching this repository and handing control to the host tooling inside it. +# It is the one file fetched on its own, because a host with no git and no checkout is what it exists to fix. +# It reads no payload, no table, and no sibling module: it obtains a tree and runs one entry point inside that tree. +# +# A tarball rather than a clone, because a clone needs git on a host that may not have it, and because a tarball of a resolved commit cannot be stale. +# The commit it resolved is printed before anything runs, so a run says which revision of the fleet's tooling it used. +# +# An unverified loader is worse than none, which is why this one does more than its Linux peer before it trusts anything. +# It pins TLS 1.2 itself rather than assume a fresh host's default reaches GitHub, it checks a fetched tree for the marker it wrote before removing anything under -Dir, and it hands off to PowerShell 7 explicitly rather than assume the console it started in already carries it. +# +# Windows PowerShell 5.1 is the one shell a fresh Windows host guarantees, the way bash is guaranteed on a fresh Debian host, so everything up to and including finding or installing PowerShell 7 is written to run under it. +# Once pwsh is confirmed, this file hands the rest of the run to itself under pwsh, and every script under host-setup/windows it goes on to drive requires that version too. + +# The Host action binds to StandUpHost rather than Host, because Host is PowerShell's own automatic variable for the host program ($Host.Name, $Host.UI), and a parameter named Host would shadow it and fail PSAvoidAssignmentToAutomaticVariable. +# The alias keeps -Host as the spelling a caller types, while leaving the automatic variable free for the console-detection checks below. +[CmdletBinding()] +param( + [Alias('r')][switch]$Report, + [Alias('Host')][switch]$StandUpHost, + [switch]$Dev, + [switch]$Upgrade, + [switch]$Tools, + [switch]$Github, + [Alias('w')][switch]$Wsl, + [Alias('n')][switch]$DryRun, + [Alias('y')][switch]$Yes, + [string]$Ref, + [string]$Dir, + [switch]$Keep, + [Alias('h')][switch]$Help +) + +# Captured before anything else touches scope, so the pwsh handoff below can rebuild the argument list this process was bound with, rather than reaching for an automatic variable from inside a nested function. +$SCRIPT_BOUND_PARAMETERS = $PSBoundParameters + +Set-StrictMode -Version Latest +$ErrorActionPreference = 'Stop' +# A non-zero exit from winget, tar or a driven script is an answer here rather than a failure. +# Setting this keeps a profile that turned it on from turning every read into a terminating error. +$PSNativeCommandUseErrorActionPreference = $false + +# A fresh Windows host's default TLS floor can predate 1.2, which raw.githubusercontent.com and the GitHub API both require. +# Set once, unconditionally, since it is a no-op where the runtime already defaults higher. +[Net.ServicePointManager]::SecurityProtocol = [Net.ServicePointManager]::SecurityProtocol -bor [Net.SecurityProtocolType]::Tls12 + +$REPO = 'ptr727/ProjectTemplate' +$DEFAULT_REF = 'main' + +# --- Output --- + +function log { param([string]$Message = '') Write-Host $Message } +function info { param([string]$Message) Write-Host " $Message" } +function step { param([string]$Message) Write-Host "`n==> $Message" } +function warn { param([string]$Message) [Console]::Error.WriteLine("WARNING: $Message") } +function die { param([string]$Message) [Console]::Error.WriteLine("ERROR: $Message"); exit 1 } + +# Every parameter is read into a variable here rather than from inside a function. +# A script parameter reached only from a nested scope reads as declared and never used, which is a finding on each one and hides a parameter that genuinely is unused. +$ACTIONS = [ordered]@{ + report = [bool]$Report + upgrade = [bool]$Upgrade + tools = [bool]$Tools + github = [bool]$Github + wsl = [bool]$Wsl + host = [bool]$StandUpHost + dev = [bool]$Dev +} +$WANT_HELP = [bool]$Help +$DRY_RUN = [bool]$DryRun +$ASSUME_YES = [bool]$Yes +$KEEP = [bool]$Keep +$REF = if ($Ref) { $Ref } else { $DEFAULT_REF } + +$MODE = '' +# No placeholder for $DIR here, unlike the four lines around it: PowerShell variable names are case-insensitive, so $DIR and the -Dir parameter $Dir are the same variable, and resetting it here would silently discard whatever -Dir the caller passed before Resolve-Directory ever reads it. +$RESOLVED = '' +$TREE = '' +$PWSH_PATH = '' + +function usage { + # The closing marker of a here-string has to sit at column 0, so this block is deliberately unindented. + Write-Host @' +Usage: bootstrap.ps1 [action] [options] + +Stands a host up: upgrades its packages, installs the host tools, and configures git and GitHub. +Fetches this repository and runs the tooling from that tree, so the tools and the rules that +describe them come from one revision rather than from whatever a host happens to hold. + +Actions, name one, default -Report: + -r, -Report Report what each tool would do, change nothing + -Host Upgrade packages, install the tools, configure git and GitHub + -Dev As -Host, and add the tools a development machine needs + -Upgrade Upgrade the packages winget manages, only + -Tools Install the host tools, only + -Github Configure git, the SSH key, and commit signing, only + -w, -Wsl Report the WSL platform and the distributions installed, only + -h, -Help Show this help + +Options: + -y, -Yes Do not prompt, and pass the same to each tool + -n, -DryRun Print what each step would run, change nothing + -Ref REF Branch, tag, pull request ref, or commit to run from, default main + -Dir PATH Where the tree is extracted, default %LOCALAPPDATA%\host-setup + -Keep Leave the extracted tree in place, which is removed by default + +With no action on a console, the menu asks. With no action and no console, the report runs, since a +redirected run is not a place to answer a question. + +This runs under Windows PowerShell 5.1, the version every fresh Windows host guarantees, and hands +control to PowerShell 7 once it has found or installed it: every script under host-setup\windows +requires it and refuses to run without it. + +Examples: + bootstrap.ps1 Ask what to do + bootstrap.ps1 -Report Report only + bootstrap.ps1 -Host -Yes Stand a host up unattended + bootstrap.ps1 -Ref develop -Report Report using the tooling on develop + bootstrap.ps1 -Tools -DryRun Show what installing the tools would run +'@ +} + +# --- pwsh handoff --- +# +# Everything above this point, and the four functions below, run under Windows PowerShell 5.1: no ternary or null-coalescing operator, no multi-argument Join-Path, nothing newer than that runtime parses. +# Everything past the handoff may use whatever pwsh 7 accepts, though it mostly does not need to. + +function Resolve-Pwsh { + $command = Get-Command pwsh -ErrorAction SilentlyContinue + if ($command) { return $command.Source } + # ProgramFiles(x86) carries no value on a host with no WOW64 layer, and Join-Path on a null path is a terminating error under Set-StrictMode, not an empty match to fall through. + $candidates = @((Join-Path $env:ProgramFiles 'PowerShell\7\pwsh.exe')) + if (${env:ProgramFiles(x86)}) { $candidates += (Join-Path ${env:ProgramFiles(x86)} 'PowerShell\7\pwsh.exe') } + foreach ($candidate in $candidates) { + if (Test-Path $candidate) { return $candidate } + } + return $null +} + +# This is the one place that install-tools.ps1's own rule does not hold: pwsh is deliberately not part of any tool registry, because a host that cannot run these scripts cannot be repaired by them. +# Standing up pwsh is the whole reason this file exists rather than starting from a checkout. +function Install-Pwsh { + if (-not (Get-Command winget -ErrorAction SilentlyContinue)) { + die 'pwsh (PowerShell 7) is not installed, and winget is not on this host to install it. Install "App Installer" from the Microsoft Store, or install PowerShell 7 directly from https://aka.ms/PSWindows, then run this again.' + } + step 'Installing PowerShell 7' + & winget install --id Microsoft.PowerShell --exact --source winget --accept-source-agreements --accept-package-agreements --silent --disable-interactivity | Out-Host + $wingetExit = $LASTEXITCODE + $found = Resolve-Pwsh + if (-not $found) { + # This names winget's own exit code even though pwsh's absence, not the code, is what decides this die: a non-zero code explains why, where "reported installing" alone does not, and winget answering 0 while pwsh is still missing is worth saying too. + die "winget exited $wingetExit installing PowerShell 7, and pwsh could still not be found. Close this console and paste the setup lines again, or install it from https://aka.ms/PSWindows." + } + return $found +} + +# Rebuilds the arguments this process was bound with, since a param() bound script has no raw $args left to forward: -Report arrives as $Report = $true, not as a string in a list. +function Get-ForwardedArgument { + $forward = @() + foreach ($key in $script:SCRIPT_BOUND_PARAMETERS.Keys) { + $value = $script:SCRIPT_BOUND_PARAMETERS[$key] + if ($value -is [switch]) { + if ($value.IsPresent) { $forward += "-$key" } + } else { + $forward += "-$key" + $forward += "$value" + } + } + return , $forward +} + +function Invoke-PwshHandoff { + $pwshPath = Resolve-Pwsh + if (-not $pwshPath) { $pwshPath = Install-Pwsh } + $forward = Get-ForwardedArgument + & $pwshPath -NoProfile -ExecutionPolicy Bypass -File $PSCommandPath @forward + exit $LASTEXITCODE +} + +# --- Fetch --- + +# Resolve the ref to the commit it names, so the run reports a revision rather than a moving name. +# The plain-text accept header returns the commit alone, which keeps this free of a JSON parser on a host that has none. +function Resolve-Ref { + try { + $sha = Invoke-RestMethod -Uri "https://api.github.com/repos/$script:REPO/commits/$script:REF" -Headers @{ Accept = 'application/vnd.github.sha' } -TimeoutSec 15 + } catch { + $sha = $null + } + if ($sha) { + $script:RESOLVED = "$sha".Trim() + return + } + # An unauthenticated request is rate limited per address, so a busy network can lose the lookup while the download itself is fine. + warn "Could not resolve $script:REF to a commit, so this run cannot be attributed to one" + $script:RESOLVED = '' +} + +# The paths this loader creates under DIR, named in one place so the cleanup and the download agree. +# DIR itself is never removed, since -Dir may name a directory the caller owns and put other things in. +function Get-TreePath { Join-Path $script:DIR 'tree' } +function Get-ArchivePath { Join-Path $script:DIR 'tree.tar.gz' } +function Get-MarkerPath { Join-Path (Get-TreePath) '.bootstrap-owned' } + +# A tree carries a marker this loader wrote, and a tree without one is somebody else's. +# DIR is a caller-supplied path, so 'tree' under it is not necessarily ours: pointing -Dir at a directory that already holds one would otherwise have this remove it, both before extracting and again on exit. +function Test-TreeOwnership { Test-Path (Get-MarkerPath) } + +# Refuses to remove a tree this run did not create, rather than trusting the name. +function Remove-Tree { + $tree = Get-TreePath + if (-not (Test-Path $tree)) { return } + if (-not (Test-TreeOwnership)) { die "$tree exists and this loader did not create it, so it will not be removed. Choose another -Dir." } + Remove-Item -Recurse -Force $tree +} + +function Get-Tree { + $archive = Get-ArchivePath + $want = if ($script:RESOLVED) { $script:RESOLVED } else { $script:REF } + + step "Fetching $script:REPO at $script:REF" + if ($script:RESOLVED) { info "Commit: $script:RESOLVED" } + + New-Item -ItemType Directory -Path $script:DIR -Force | Out-Null + try { + Invoke-WebRequest -UseBasicParsing -Uri "https://codeload.github.com/$script:REPO/tar.gz/$want" -OutFile $archive -TimeoutSec 120 + } catch { + die "Could not download $script:REPO at $script:REF. Check the ref exists and that this host reaches codeload.github.com." + } + + # The archive holds one top-level directory named for the repository and the revision. + # Extracting into a directory of our own keeps a second run from reading the first one's tree. + $tree = Get-TreePath + Remove-Tree + New-Item -ItemType Directory -Path $tree -Force | Out-Null + New-Item -ItemType File -Path (Get-MarkerPath) -Force | Out-Null + & tar -xzf $archive -C $tree --strip-components=1 + if ($LASTEXITCODE -ne 0) { die 'Could not extract the downloaded archive' } + Remove-Item -Force $archive -ErrorAction SilentlyContinue + + $script:TREE = $tree + info "Extracted to $script:TREE" +} + +# A tree that is not ours was already refused where it mattered, at the download. +# Refusing again from here would print the same error a second time, after the one that actually stopped the run. +# Removes what this run created rather than what it finished, because TREE is set only once extraction has succeeded. +# A failed extract leaves both the archive and a part-written tree, so keying this on TREE left a tarball in the cache on every failed attempt. +function Invoke-Cleanup { + if ($script:KEEP) { return } + $tree = Get-TreePath + if ((Test-Path $tree) -and -not (Test-TreeOwnership)) { + Remove-Item -Force (Get-ArchivePath) -ErrorAction SilentlyContinue + return + } + Remove-Tree + Remove-Item -Force (Get-ArchivePath) -ErrorAction SilentlyContinue +} + +# --- Handoff --- + +# Every tool runs from inside the fetched tree, and this is the only place a path inside it is named. +# Written as one interpolated, forward-slashed string against the bare $TREE rather than $script:TREE or Join-Path, so this loader's one entry point into the tree reads as the literal pattern the Linux loader is checked by, and stays checkable by that same pattern. +# A read resolves $TREE up to script scope on its own, and only a write needs the script: prefix, which is why the assignment in Get-Tree still carries it. +function Invoke-Tool { + param([Parameter(Mandatory)][string]$Tool, [Parameter(ValueFromRemainingArguments)][string[]]$Arguments) + + $path = "$TREE/host-setup/windows/$Tool" + if (-not (Test-Path $path)) { + die "The fetched tree carries no $Tool at host-setup/windows, so this ref is not one to bootstrap from" + } + + $flags = @() + if ($script:ASSUME_YES) { $flags += '-Yes' } + if ($script:DRY_RUN) { $flags += '-DryRun' } + + & $script:PWSH_PATH -NoProfile -ExecutionPolicy Bypass -File $path @Arguments @flags + if ($LASTEXITCODE -ne 0) { die "$Tool exited $LASTEXITCODE" } +} + +function Show-Report { + Invoke-Tool -Tool 'upgrade-host.ps1' -Arguments '-Status' + Invoke-Tool -Tool 'install-tools.ps1' -Arguments '-Report' + Invoke-Tool -Tool 'setup-github.ps1' -Arguments '-Status' +} + +# The order is fixed rather than chosen. +# Packages come first so install-tools.ps1 and setup-github.ps1 act on a host winget has just brought current, and GitHub comes last because it is the only step that waits on a person in a browser. +function Invoke-StandUp { + # Named Kind rather than Profile, which is PowerShell's own automatic variable for the current user's profile script. + param([string]$Kind) + + Invoke-Tool -Tool 'upgrade-host.ps1' -Arguments '-Packages' + if ($Kind -eq 'dev') { + Invoke-Tool -Tool 'install-tools.ps1' -Arguments '-Install', '-Optional' + } else { + Invoke-Tool -Tool 'install-tools.ps1' -Arguments '-Install' + } + Invoke-Tool -Tool 'setup-github.ps1' -Arguments '-Configure' +} + +# Names the host in the menu heading. +function Get-HostDescription { + try { + return (Get-CimInstance Win32_OperatingSystem -ErrorAction Stop).Caption + } catch { + return 'this host' + } +} + +# Both are checked because a scheduled task reports one and not the other, and either alone misses a case. +# The ISE is checked apart, because it answers both of those as interactive and then blocks on a Read-Host it does not render usably. +function Test-Interactive { + if (-not [Environment]::UserInteractive -or [Console]::IsInputRedirected) { return $false } + if ($Host.Name -eq 'Windows PowerShell ISE Host') { return $false } + return $true +} + +function Show-Menu { + log "Standing up $(Get-HostDescription)" + log '' + log ' 1 Report only, change nothing' + log ' 2 Upgrade the packages winget manages' + log ' 3 Install the host tools' + log ' 4 Configure git and GitHub' + log ' 5 Report the WSL platform and the distributions installed' + log ' 6 All of the above but WSL, which is a host stood up' + log ' 7 All of the above but WSL, plus the development tools' + log ' q Quit' + log '' + + $choice = Read-Host 'Choose' + switch ($choice) { + '1' { $script:MODE = 'report' } + '2' { $script:MODE = 'upgrade' } + '3' { $script:MODE = 'tools' } + '4' { $script:MODE = 'github' } + '5' { $script:MODE = 'wsl' } + '6' { $script:MODE = 'host' } + '7' { $script:MODE = 'dev' } + 'q' { exit 0 } + 'Q' { exit 0 } + default { die 'Not one of the choices' } + } +} + +# --- Entry --- + +# Windows has shipped tar.exe under %SystemRoot%\System32 since Windows 10 1803 and Windows Server 2019, and it reads a .tar.gz archive directly. +# That is why this loader does not reach for Expand-Archive, which cannot. +function Test-Prerequisite { + if (-not (Get-Command tar -ErrorAction SilentlyContinue)) { + die 'tar.exe not found under %SystemRoot%\System32. This assumes Windows 10 1803, Windows Server 2019, or later, all of which ship it.' + } +} + +# An absolute path, and never a drive root, since everything below it is created and removed under it. +function Resolve-Directory { + if (-not $script:Dir) { return (Join-Path $env:LOCALAPPDATA 'host-setup') } + if (-not [IO.Path]::IsPathRooted($script:Dir)) { die "-Dir takes an absolute path, and `"$($script:Dir)`" is relative" } + $trimmed = $script:Dir.TrimEnd('\', '/') + if ((-not $trimmed) -or ($trimmed -match '^[A-Za-z]:$')) { die '-Dir may not be a drive root' } + return $trimmed +} + +# PowerShell records which switches were given and not the order they came in, so two actions is a refusal rather than the last one winning. +# Unlike the four scripts this loader drives, zero given is not this loader's own default: main tells the menu and the piped-in report apart, the way bootstrap.sh's own entry point does. +function Resolve-Mode { + $given = @($script:ACTIONS.Keys | Where-Object { $script:ACTIONS[$_] }) + if ($given.Count -gt 1) { die "More than one action given ($($given -join ', ')), name one" } + if ($given.Count -eq 0) { return '' } + return $given[0] +} + +function main { + if ($script:WANT_HELP) { usage; exit 0 } + + if ($PSVersionTable.PSVersion.Major -lt 7) { + Invoke-PwshHandoff + } + + # Reached only under a confirmed pwsh 7, either started that way or handed off to above. + $script:PWSH_PATH = (Get-Process -Id $PID).Path + Test-Prerequisite + $script:DIR = Resolve-Directory + $script:MODE = Resolve-Mode + + # A run with no action and no console reports rather than guessing, which is what a redirected run is. + # The remedy is printed rather than assumed, since somebody reaching this has just pasted a one-line install. + if (-not $script:MODE) { + if (Test-Interactive) { + Show-Menu + } else { + $script:MODE = 'report' + warn 'No action given and no console to ask on, so this is a report' + info 'Download the file and run it, rather than piping it, to reach the menu:' + info ' [Net.ServicePointManager]::SecurityProtocol = [Net.ServicePointManager]::SecurityProtocol -bor [Net.SecurityProtocolType]::Tls12' + info " Invoke-WebRequest -UseBasicParsing -Uri https://raw.githubusercontent.com/$script:REPO/$script:DEFAULT_REF/host-setup/bootstrap.ps1 -OutFile bootstrap.ps1" + info ' powershell -ExecutionPolicy Bypass -File bootstrap.ps1' + } + } + + try { + Resolve-Ref + Get-Tree + switch ($script:MODE) { + 'report' { Show-Report } + 'upgrade' { Invoke-Tool -Tool 'upgrade-host.ps1' -Arguments '-Packages' } + 'tools' { Invoke-Tool -Tool 'install-tools.ps1' -Arguments '-Install' } + 'github' { Invoke-Tool -Tool 'setup-github.ps1' -Arguments '-Configure' } + # Only setup-wsl.ps1's -Status runs here: its -Install needs a distribution name, which no flag here collects, so choosing a default distro nobody asked for is exactly what -Wsl staying out of -Host and -Dev already exists to avoid. + # Installing one by name is a checkout away, once this run has fetched it. + 'wsl' { Invoke-Tool -Tool 'setup-wsl.ps1' -Arguments '-Status' } + 'host' { Invoke-StandUp -Kind 'host' } + 'dev' { Invoke-StandUp -Kind 'dev' } + } + } finally { + Invoke-Cleanup + } + + step 'Done' + if ($script:KEEP) { info "The fetched tree is at $script:TREE" } +} + +main diff --git a/host-setup/windows/README.md b/host-setup/windows/README.md index 5a33ae42..931d8770 100644 --- a/host-setup/windows/README.md +++ b/host-setup/windows/README.md @@ -61,9 +61,16 @@ Neither this tooling nor its Linux sibling installs `markdownlint`, `cspell`, `a Each of those runs as a pinned container image or through `uvx`, which is what keeps a local run and CI the same check: the image tag fixes the version. Installing native copies through `winget` would put a second, unpinned version of each on the host, and a local run would then differ from CI, which is the exact property the pinned images exist to guarantee. The only host requirements any of it creates are `docker` and `uv`, and both are already in the registry. -## Why There Is No bootstrap.ps1 +## bootstrap.ps1 -[`bootstrap.sh`][bootstrap] exists to stand up a host that has no git and no checkout. The Windows form of that problem has no one-liner anybody here has run, and an unverified loader is worse than none, which is the same rule [`docs/host-setup.md`][host-setup] applies to its own verification block. A Windows host therefore obtains this repository first and runs these scripts from the checkout. +[`bootstrap.ps1`][bootstrap-ps1] is a fifth script, and the odd one out: it sits beside [`bootstrap.sh`][bootstrap] at the top of [`host-setup/`][host-setup-readme] rather than here, since it is the same concern as that file rather than a fifth member of this registry. It exists to stand up a host that has no git and no checkout, the same problem `bootstrap.sh` solves on Linux. + +An unverified loader is worse than none, and that is why this one does more than fetch a tarball. It runs under Windows PowerShell 5.1, the one shell a fresh Windows host guarantees, and hands off to `pwsh` only once it has found or installed it through `winget`, since every script in this directory refuses to run under anything older. It pins TLS 1.2 itself rather than assume a fresh console's default reaches GitHub. And it checks a fetched tree for the marker it wrote before removing anything under `-Dir`, the same rule `bootstrap.sh` follows on Linux. None of that verifies the tooling it goes on to run, which stays exactly as unverified against a genuinely fresh host as it always was. It verifies the one step earlier this loader adds, standing up the interpreter everything past it depends on. + +```powershell +pwsh -NoProfile -File ..\bootstrap.ps1 -Help +..\bootstrap.ps1 -Report -DryRun +``` ## Docker Desktop and WSL @@ -101,6 +108,7 @@ host-setup\windows\install-tools.ps1 -Report host-setup\windows\upgrade-host.ps1 -Status host-setup\windows\setup-github.ps1 -Status host-setup\windows\setup-wsl.ps1 -Status +host-setup\bootstrap.ps1 -Report -DryRun ``` Then the dry runs, which print what each action would do: @@ -120,6 +128,7 @@ The scripts are checked by `PSScriptAnalyzer`, which runs in CI as the peer of t [agent-safety]: ../agent-safety/install.ps1 [bootstrap]: ../bootstrap.sh +[bootstrap-ps1]: ../bootstrap.ps1 [governance]: ../../GOVERNANCE.md [host-setup]: ../../docs/host-setup.md [host-setup-readme]: ../README.md diff --git a/scripts/test_bootstrap.py b/scripts/test_bootstrap.py index fe2cd75b..3abdc97a 100644 --- a/scripts/test_bootstrap.py +++ b/scripts/test_bootstrap.py @@ -1,12 +1,12 @@ -"""Tests for the host bootstrap: the loader invariant, and that the tooling covers what the spec requires. +"""Tests for the host bootstraps: the loader invariant, and that the tooling covers what the spec requires. Two properties, both of which fail silently rather than loudly if nobody checks them. -The loader invariant is what keeps `host-setup/bootstrap.sh` outside the reach of the -`Hub-Hosted Tooling` rule rather than exempt from it. A loader obtains a tree and hands control to -one entry point inside it. The moment it reads a second path in that tree it has become a tool that -reads hub content, and the rule applies to it in full. That boundary is a property of the file, so -it is asserted here rather than promised in prose. +The loader invariant is what keeps `host-setup/bootstrap.sh` and `host-setup/bootstrap.ps1` outside +the reach of the `Hub-Hosted Tooling` rule rather than exempt from it. A loader obtains a tree and +hands control to one entry point inside it. The moment it reads a second path in that tree it has +become a tool that reads hub content, and the rule applies to it in full. That boundary is a property +of each file, so it is asserted here rather than promised in prose. The coverage assertion is the only connection between the floors in `spec/host-tools.json` and the tooling that installs them. Nothing joins the two at runtime, deliberately: the gate measures a host @@ -31,6 +31,7 @@ ROOT = Path(__file__).resolve().parent.parent BOOTSTRAP = ROOT / 'host-setup' / 'bootstrap.sh' +BOOTSTRAP_PS = ROOT / 'host-setup' / 'bootstrap.ps1' LINUX = ROOT / 'host-setup' / 'linux' WINDOWS = ROOT / 'host-setup' / 'windows' HOST_TOOLS = ROOT / 'spec' / 'host-tools.json' @@ -110,42 +111,67 @@ def spec_tools() -> list[dict]: return [] -def test_loader_reads_one_path_into_the_tree() -> None: - """The loader references exactly one directory inside the tree it fetches. +def assert_loader_reads_one_path(path: Path, expected: str) -> None: + """A loader references exactly one directory inside the tree it fetches. - The expected set names the Linux path alone, and stays that way while `host-setup/windows` - carries no loader of its own. Widening it to admit a Windows path before one exists would - retire the invariant ahead of the thing it protects. + Both loaders write that one reference as a single interpolated, forward-slashed string + (`$TREE/host-setup//$tool` in each), rather than building it from parts, which is what + lets this one pattern read either file unmodified. """ - text = BOOTSTRAP.read_text(encoding='utf-8') + text = path.read_text(encoding='utf-8') # Every reference to the fetched tree goes through the variable holding its location, so the paths it names are countable rather than scattered. references = re.findall(r'\$TREE(?:/[^"\'\s]*)?', text) paths = {reference for reference in references if '/' in reference} check( - paths == {'$TREE/host-setup/linux/$tool'}, - f'the loader reads more than its one entry point into the fetched tree: {sorted(paths)}', + paths == {expected}, + f'{path.name} reads more than its one entry point into the fetched tree: {sorted(paths)}', ) # A payload or a table read from the tree is what makes a file a tool rather than a loader. for forbidden in ('spec/', 'registry/', 'repo-config/', 'catalog/'): check( f'$TREE/{forbidden}' not in text, - f'the loader reads {forbidden} from the fetched tree, which makes it a tool', + f'{path.name} reads {forbidden} from the fetched tree, which makes it a tool', ) -def test_loader_needs_no_python() -> None: +def assert_loader_needs_no_python(path: Path) -> None: """A host being bootstrapped must not be made to install an interpreter first.""" - text = BOOTSTRAP.read_text(encoding='utf-8') + text = path.read_text(encoding='utf-8') for interpreter in ('python3 ', 'python ', 'uv run', 'py -3'): check( interpreter not in text, - f'the loader invokes {interpreter.strip()}, which a host being bootstrapped may not have', + f'{path.name} invokes {interpreter.strip()}, which a host being bootstrapped may not have', ) +def test_linux_loader_reads_one_path_into_the_tree() -> None: + """`bootstrap.sh` references exactly one directory inside the tree it fetches.""" + assert_loader_reads_one_path(BOOTSTRAP, '$TREE/host-setup/linux/$tool') + + +def test_windows_loader_reads_one_path_into_the_tree() -> None: + """`bootstrap.ps1` references exactly one directory inside the tree it fetches. + + `$Tool`, PascalCase, because that is the parameter name PowerShell convention wants, where bash's + equivalent is the lowercase local `$tool`. The pattern this shares with the Linux check is the shape + of the path, one interpolated `$TREE/host-setup//` string, not the exact casing. + """ + assert_loader_reads_one_path(BOOTSTRAP_PS, '$TREE/host-setup/windows/$Tool') + + +def test_linux_loader_needs_no_python() -> None: + """A host `bootstrap.sh` stands up must not be made to install an interpreter first.""" + assert_loader_needs_no_python(BOOTSTRAP) + + +def test_windows_loader_needs_no_python() -> None: + """A host `bootstrap.ps1` stands up must not be made to install an interpreter first.""" + assert_loader_needs_no_python(BOOTSTRAP_PS) + + def assert_coverage(platform: str, managed: set[str], installer: str) -> None: """A tool the spec requires is one the named installer can provide, or a recorded exception.""" if not managed: @@ -248,14 +274,35 @@ def test_every_windows_script_is_present() -> None: ) +def test_bootstrap_ps1_is_present_and_unmarked() -> None: + """`bootstrap.ps1` is present, and does not open with a shebang. + + Kept apart from `test_every_windows_script_is_present` rather than folded into it, because + `bootstrap.ps1` deliberately sits beside `bootstrap.sh` at `host-setup/`, not inside + `host-setup/windows/` with the four scripts that test checks. Same reasoning as that test: the + `eol-coverage` gate pins a tracked file opening `#!` to `eol=lf`, against the CRLF this file is + written with. + """ + check(BOOTSTRAP_PS.is_file(), 'bootstrap.ps1 is missing from host-setup') + if BOOTSTRAP_PS.is_file(): + check( + not BOOTSTRAP_PS.read_bytes().startswith(b'#!'), + 'bootstrap.ps1 opens with a shebang, which the eol-coverage gate then pins to LF, ' + 'against the CRLF this file is written with', + ) + + def main() -> int: for test in ( - test_loader_reads_one_path_into_the_tree, - test_loader_needs_no_python, + test_linux_loader_reads_one_path_into_the_tree, + test_windows_loader_reads_one_path_into_the_tree, + test_linux_loader_needs_no_python, + test_windows_loader_needs_no_python, test_every_required_linux_tool_is_installable, test_every_required_windows_tool_is_installable, test_every_managed_tool_is_executable, test_every_windows_script_is_present, + test_bootstrap_ps1_is_present_and_unmarked, ): test()