diff --git a/docs/docs/reference/security-faq.md b/docs/docs/reference/security-faq.md index 5825730d58..f4eba04c3b 100644 --- a/docs/docs/reference/security-faq.md +++ b/docs/docs/reference/security-faq.md @@ -140,10 +140,16 @@ A single `first_launch` event is sent containing only: - The installed version (e.g., "0.5.9") - Whether this is a fresh install or upgrade (boolean) +- Which installer was used (`curl`, `powershell`, `npm`, `vscode-extension`, `local`, or `unknown` — what every upgrade from a version predating this field reports) - Your anonymous machine ID (random UUID) No code, queries, file paths, or personal information is included. This event helps us understand adoption and is fully opt-out-able. +The install scripts (`altimate.sh/install`, `install.ps1`), the npm postinstall, and the VS Code extension's installer send nothing themselves and contact no telemetry endpoint. They only record the version and installer name to a local file that the CLI reads on its next run, so the opt-out above decides whether anything is ever transmitted. + +!!! warning "One caveat on the config-file opt-out" + The environment variables (`ALTIMATE_TELEMETRY_DISABLED`, `OPENCODE_DISABLE_TELEMETRY`) are always honoured. The `telemetry.disabled` **config key** is read during telemetry startup, which can run before the CLI's config is resolvable — and in that case startup currently proceeds with telemetry enabled. A user who has opted out via the config key alone may therefore still have this event transmitted. Use an environment variable if you need a guarantee. + ## What happens when I authenticate via a well-known URL? When you run `altimate auth login `, the CLI fetches `/.well-known/altimate-code` to discover the server's auth command. Before executing anything: diff --git a/docs/docs/reference/telemetry.md b/docs/docs/reference/telemetry.md index d28a484475..10b443630e 100644 --- a/docs/docs/reference/telemetry.md +++ b/docs/docs/reference/telemetry.md @@ -38,7 +38,7 @@ We collect the following categories of events: | `feature_suggestion` | A post-connection feature suggestion is shown (suggestion_type, suggestions_shown, warehouse_type — no user input) | | `sql_execute_failure` | A SQL execution fails (warehouse type, query type, error message, PII-masked SQL — no raw values) | | `core_failure` | An internal tool error occurs (tool name, category, error class, truncated error message, PII-safe input signature, and optionally masked arguments — no raw values or credentials) | -| `first_launch` | Fired once on first CLI run after installation. Contains version and is_upgrade flag. No PII. | +| `first_launch` | Fired once on the first CLI run after an install or upgrade, triggered by a marker file the installer wrote — the installers themselves send nothing and contact no telemetry endpoint. Contains the installed version, `is_upgrade`, and `install_method` (`curl`, `powershell`, `npm`, `vscode-extension`, `local` for `install --binary`, or `unknown` for markers written before the field existed). `vscode-extension` starts appearing only once an extension build containing the marker write ships, so a zero share for it means the extension has not rolled out yet rather than no extension installs. No PII. **Reading `is_upgrade`:** it means "this machine had run altimate-code before", probed as whether `~/.altimate/machine-id` already existed — *not* "a binary was already present". A reinstall onto a machine that ever ran the CLI reports `is_upgrade: true`, and `altimate uninstall` leaves `machine-id` in place, so a metric excluding upgrades counts installs **per previously-unseen machine** and undercounts reinstalls onto known ones. (`is_upgrade` is a boolean in the event schema; it arrives in Application Insights `customDimensions` as a string, so KQL filters read `tostring(customDimensions.is_upgrade) != "true"`.) Delivery is at-most-once: the marker is deleted before the event flushes, so a process that dies first loses that install rather than re-firing it every launch. Local `--binary` installs report `version: "local"`. | | `task_outcome_signal` | Behavioral quality signal at session end — accepted, error, abandoned, or cancelled. Includes tool count, step count, duration, and last tool category. No user content. | | `task_classified` | Intent classification of the first user message using keyword matching — category (e.g. `debug_dbt`, `write_sql`, `optimize_query`), confidence score, and detected warehouse type. No user text is sent — only the classified category. | | `tool_chain_outcome` | Aggregated tool execution sequence at session end — ordered tool names (capped at 50), error count, recovery count, final outcome, duration, and cost. No tool arguments or outputs. | diff --git a/install b/install index e2962f2f49..5698bddde8 100755 --- a/install +++ b/install @@ -487,11 +487,55 @@ install_from_binary() { chmod 755 "$dest_path" } +# Write the same post-install marker that npm's postinstall.mjs writes, so the +# CLI emits its `first_launch` telemetry event on the next run. Without this the +# curl install path — the one advertised at altimate.sh/install — produces no +# install event at all, and every curl user is invisible in install metrics. +# +# The path MUST match welcome.ts's data-dir resolution ($XDG_DATA_HOME, falling +# back to ~/.local/share) on every platform, including Windows: the CLI reads it +# via Node's os.homedir() and never consults %LOCALAPPDATA%. +# +# No network call and no identifier is written here — this only hands the CLI the +# version it was installed at. Whether anything is ever sent remains entirely up +# to the CLI's existing telemetry opt-out gates. +# $1 — install_method to record. Must be a value in the CLI's allowlist +# (packages/opencode/src/cli/welcome.ts); anything else reports as "unknown". +write_install_marker() { + local marker_source="$1" + local data_dir="${XDG_DATA_HOME:-$HOME/.local/share}/altimate-code" + # An empty marker is deleted unread by the CLI, so fall back to "unknown" + # rather than losing the install: $specific_version is empty whenever the + # GitHub API could not be reached (see check_version). + local marker_version="${specific_version:-unknown}" + mkdir -p "$data_dir" 2>/dev/null || return 0 + # Companion first, trigger last: the CLI returns early unless .installed-version + # exists, then consumes .install-source. Trigger-first would let a CLI starting in + # between report install_method "unknown", and a truncated .installed-version is + # deleted unread — losing the install rather than just its attribution. + printf '%s' "$marker_source" > "$data_dir/.install-source" 2>/dev/null || return 0 + # The trigger is published atomically. Companion-first alone only closes the + # "attribution lost" window; a plain redirect truncates before filling, so a CLI + # starting mid-write can still observe an EMPTY .installed-version, which it + # deletes unread — losing the install itself. mv within one directory is atomic. + local tmp="$data_dir/.installed-version.$$" + printf '%s' "${marker_version#v}" > "$tmp" 2>/dev/null || return 0 + mv -f "$tmp" "$data_dir/.installed-version" 2>/dev/null || { rm -f "$tmp" 2>/dev/null; return 0; } +} + if [ -n "$binary_path" ]; then install_from_binary + # Attributed as "local", not "curl": --binary installs a file the caller already + # had (dev build, air-gapped artifact) and sets specific_version="local", so + # folding it into the curl metric would misreport both source and version. + # Still recorded — it is a real install — just not a curl one. + write_install_marker "local" else check_version download_and_install + # Only reached when an install actually happened: check_version exits 0 early + # when the requested version is already present. + write_install_marker "curl" fi diff --git a/install.ps1 b/install.ps1 index 886c2c1be0..73efc64d58 100644 --- a/install.ps1 +++ b/install.ps1 @@ -304,6 +304,71 @@ if (-not $needsBaseline) { } } +# --------------------------------------------------------------------------- +# Post-install marker (install telemetry) +# --------------------------------------------------------------------------- +# Mirrors npm's postinstall.mjs (and ./install's write_install_marker) so the CLI +# emits its `first_launch` event on the next run; without it this install path is +# invisible in install metrics. +# +# A function, not inline code, so the Pester suite can AST-extract and execute it +# against a temp profile the same way it does Test-Checksum. The subprocess tests +# deliberately stop the installer before this point, so inline code here would have +# no runtime coverage on the riskiest of the writers. +function Write-InstallMarker { + param([string]$Version) + + # The directory MUST match welcome.ts's resolution - $XDG_DATA_HOME, else + # \.local\share - because the CLI reads it through Node's os.homedir() and + # never looks at %LOCALAPPDATA%. Writing to LOCALAPPDATA here would be silently + # ignored at read time. + # + # No network call and no identifier is written; only the installed version is + # recorded. The CLI's existing telemetry opt-out gates still decide whether + # anything is ever sent. + # + # EVERYTHING is inside the try, path computation included. $ErrorActionPreference is + # "Stop", and Join-Path resolves provider-qualified paths - so a null or empty + # $env:USERPROFILE (pwsh on non-Windows, a stripped service profile) or an + # XDG_DATA_HOME naming a non-existent PSDrive raises a TERMINATING error. Computed + # outside the try, that error would abort the installer after the binary is placed + # but before the PATH registry write and the "Get started" output, leaving the user + # with an installed binary that is not on PATH. [IO.Path]::Combine also keeps + # PSDrive resolution out of it entirely. + try { + $dataRoot = if ($env:XDG_DATA_HOME) { $env:XDG_DATA_HOME } else { [IO.Path]::Combine($env:USERPROFILE, ".local", "share") } + $dataDir = [IO.Path]::Combine($dataRoot, "altimate-code") + New-Item -ItemType Directory -Force -Path $dataDir | Out-Null + # The CLI deletes an empty marker without reporting, so fall back to "unknown" + # when the version could not be resolved (GitHub API unreachable). + $markerVersion = if ($Version) { $Version -replace '^v', '' } else { "unknown" } + # -NoNewline: the CLI trims, but keep the file byte-identical to the npm path. + # + # -Encoding ascii, not utf8: the documented entrypoint is `powershell -c "irm ... | iex"`, + # i.e. Windows PowerShell 5.1, where `-Encoding utf8` prepends a UTF-8 BOM. Both values are + # ASCII by construction, so ascii is lossless here and cannot emit one. The CLI's .trim() + # happens to strip a leading BOM (U+FEFF is JS whitespace), but the install-source value is + # matched against a fixed allowlist and must not depend on that. + # + # Companion first, trigger last. The CLI returns early unless .installed-version + # exists, then consumes .install-source - so writing the trigger first would let a + # CLI starting in between report install_method "unknown". Set-Content also + # truncates before writing, and an empty .installed-version is deleted unread, + # which would lose the install outright. + Set-Content -Path ([IO.Path]::Combine($dataDir, ".install-source")) -Value "powershell" -NoNewline -Encoding ascii + # Trigger published atomically: Set-Content truncates before writing, so a CLI + # starting mid-write could observe an EMPTY .installed-version and delete it + # unread, losing the install. Move-Item within one directory is atomic. + $tmpMarker = [IO.Path]::Combine($dataDir, ".installed-version.tmp") + Set-Content -Path $tmpMarker -Value $markerVersion -NoNewline -Encoding ascii + Move-Item -Force -Path $tmpMarker -Destination ([IO.Path]::Combine($dataDir, ".installed-version")) + } catch { + # Non-fatal - a missing marker only costs us the install event, never the install. + } +} + +Write-InstallMarker -Version $specificVersion + # --------------------------------------------------------------------------- # PATH (user scope, via registry + broadcast) # --------------------------------------------------------------------------- diff --git a/packages/opencode/script/postinstall.mjs b/packages/opencode/script/postinstall.mjs index ee22894c42..8376a49862 100644 --- a/packages/opencode/script/postinstall.mjs +++ b/packages/opencode/script/postinstall.mjs @@ -238,7 +238,20 @@ function writeUpgradeMarker(version) { const xdgData = process.env.XDG_DATA_HOME || path.join(os.homedir(), ".local", "share") const dataDir = path.join(xdgData, "altimate-code") fs.mkdirSync(dataDir, { recursive: true }) - fs.writeFileSync(path.join(dataDir, ".installed-version"), version.replace(/^v/, "")) + // Companion first, trigger last. `.installed-version` is what the CLI keys on: + // it returns early unless that file exists, then consumes `.install-source`. + // Trigger-first left two windows — a CLI starting in between reports + // install_method "unknown", and writeFileSync truncates before writing, so a + // reader could observe an EMPTY `.installed-version` and delete it unread, + // losing the install outright. Matches `install` and `install.ps1`. + fs.writeFileSync(path.join(dataDir, ".install-source"), "npm") + // Trigger published atomically: writeFileSync truncates before filling, so a CLI + // starting mid-write could observe an EMPTY `.installed-version` and delete it + // unread, losing the install. renameSync within one directory is atomic. + const versionPath = path.join(dataDir, ".installed-version") + const tmpPath = `${versionPath}.${process.pid}.tmp` + fs.writeFileSync(tmpPath, version.replace(/^v/, "")) + fs.renameSync(tmpPath, versionPath) } catch { // Non-fatal — the CLI just won't show a welcome banner } diff --git a/packages/opencode/src/altimate/telemetry/index.ts b/packages/opencode/src/altimate/telemetry/index.ts index e7f14db537..45dce393fe 100644 --- a/packages/opencode/src/altimate/telemetry/index.ts +++ b/packages/opencode/src/altimate/telemetry/index.ts @@ -474,6 +474,11 @@ export namespace Telemetry { session_id: string version: string is_upgrade: boolean + // altimate_change — which installer wrote the marker. Recorded by the + // installer itself; "unknown" when the marker predates this field or the + // source file was unreadable. Without it, curl and npm installs are + // indistinguishable in the same metric. + install_method: "curl" | "powershell" | "npm" | "vscode-extension" | "local" | "unknown" } // altimate_change end // altimate_change start — telemetry for skill management operations diff --git a/packages/opencode/src/cli/welcome.ts b/packages/opencode/src/cli/welcome.ts index dab265049a..f8ffd3f954 100644 --- a/packages/opencode/src/cli/welcome.ts +++ b/packages/opencode/src/cli/welcome.ts @@ -9,6 +9,59 @@ import { Telemetry } from "../altimate/telemetry" const APP_NAME = "altimate-code" const MARKER_FILE = ".installed-version" +// altimate_change start — written alongside MARKER_FILE by whichever installer ran +// (postinstall.mjs, install, install.ps1) so first_launch can attribute the install. +const SOURCE_FILE = ".install-source" +// "vscode-extension" is the dominant installer by volume: the VS Code extension's +// native installer pulls the binary straight from GitHub releases, bypassing npm and +// both shell scripts (it stopped spawning `curl | bash` because EDR tooling flagged +// it — vscode-dbt-power-user#2049). It writes the marker so those installs land here +// rather than going uncounted. +// "local" is `install --binary ` — a dev build or air-gapped artifact the caller +// already had. Recorded separately so it cannot inflate the curl metric (that path also +// reports version "local"). +const INSTALL_METHODS = ["curl", "powershell", "npm", "vscode-extension", "local"] as const +type InstallMethod = (typeof INSTALL_METHODS)[number] | "unknown" + +/** Remove SOURCE_FILE, tolerating absence. Kept separate so the empty-marker path can + * clear an orphan without pretending to read a value it will not use. + * + * rmSync with recursive, not unlinkSync: unlinkSync throws EPERM/EISDIR on a directory, + * so a directory-shaped `.install-source` would survive every launch and keep the read + * path returning "unknown" forever. force:true also absorbs the ordinary ENOENT case. */ +function clearInstallSource(dataDir: string): void { + try { + fs.rmSync(path.join(dataDir, SOURCE_FILE), { force: true, recursive: true }) + } catch { + // Removal refused (read-only dir, EACCES on the parent) — nothing further to do. + } +} + +/** + * Read the installer that wrote the marker, then remove the file so it stays in + * lockstep with MARKER_FILE — a stale value must never be attributed to a later + * install whose installer did not write one. + * + * Returns "unknown" for a missing, unreadable, or unrecognized value: the marker + * predates this field on upgrade from an older version, and an unrecognized + * string must not reach the event as a free-form value. + * + * The unlink runs in `finally`, so a file that exists but cannot be READ (EACCES, + * a directory in its place) is still cleared. Leaving it behind would let a stale + * value be attributed to the next install whose installer wrote only the version. + */ +function readInstallMethod(dataDir: string): InstallMethod { + const sourcePath = path.join(dataDir, SOURCE_FILE) + try { + const raw = fs.readFileSync(sourcePath, "utf-8").trim() + return (INSTALL_METHODS as readonly string[]).includes(raw) ? (raw as InstallMethod) : "unknown" + } catch { + return "unknown" + } finally { + clearInstallSource(dataDir) + } +} +// altimate_change end /** Resolve the data directory at call time (respects XDG_DATA_HOME changes in tests). */ function getDataDir(): string { @@ -27,32 +80,62 @@ function getDataDir(): string { */ export function showWelcomeBannerIfNeeded(): void { try { - const markerPath = path.join(getDataDir(), MARKER_FILE) + const dataDir = getDataDir() + const markerPath = path.join(dataDir, MARKER_FILE) if (!fs.existsSync(markerPath)) return const installedVersion = fs.readFileSync(markerPath, "utf-8").trim() if (!installedVersion) { fs.unlinkSync(markerPath) + // altimate_change — clear the companion file too, so an orphaned source value + // cannot be attributed to a later install. Both install scripts write "unknown" + // rather than an empty version, so this path should now only be reachable from + // a truncated or hand-edited marker. + clearInstallSource(dataDir) return } - // Remove marker first to avoid showing twice even if display fails + // Remove marker first to avoid showing twice even if display fails. + // + // altimate_change — this makes first_launch deliberately at-most-once: the marker + // is gone before Telemetry flushes, so a process that dies inside the flush + // interval loses the event permanently rather than re-firing it on every + // subsequent launch. That trade is intentional — an offline or crash-looping + // machine repeating first_launch would corrupt install counts far worse than a + // rare miss. Do NOT make deletion contingent on a successful flush. fs.unlinkSync(markerPath) - // altimate_change start — "upgrade" means the machine-id file already existed before this - // launch. Probe existence with existsSync only — do NOT mint here. Minting is left to - // Telemetry.doInit() (its job, not the welcome banner's); the first_launch machine_id is - // attached at flush time from telemetry module state, so it does not depend on minting here. + // altimate_change start — `is_upgrade` means "this machine had run altimate-code before", + // probed as: did ~/.altimate/machine-id exist? Probe with existsSync only — do NOT mint + // here. Minting is Telemetry.doInit()'s job; the first_launch machine_id is attached at + // flush time from telemetry module state, so it does not depend on minting here. + // + // src/index.ts calls this function BEFORE Telemetry.init() precisely so this probe cannot + // race the mint. Keep that order. + // + // KNOWN SEMANTIC LIMIT: this is "has prior run", not "was the binary absent". A machine + // that ever ran the CLI — an old curl install, a trial — reports is_upgrade: true even + // when an installer genuinely placed a new binary, and `altimate uninstall` leaves + // machine-id in place (cli/cmd/uninstall.ts removes data/cache/config/state, not + // ~/.altimate). Install dashboards filtering `is_upgrade != "true"` therefore undercount + // reinstalls onto known machines. Deliberate: it keeps the count deduplicated per machine. + // The installers know the true answer (the VS Code extension already distinguishes + // first-install from auto-update) — carrying that in the marker would be the fix if a + // strict install count is ever required. Documented in docs/docs/reference/telemetry.md. // // FIXME(telemetry-init-config-opt-out): doInit() may run before Instance.provide() has made // Config.get() resolvable (see the try/catch around Config.get in telemetry/index.ts::doInit // — the catch branch proceeds with telemetry enabled). A user who opted out via the - // `telemetry.disabled` config key — with no env var set — can therefore still get a - // machine-id minted on first launch. The env-var opt-out (ALTIMATE_TELEMETRY_DISABLED / - // OPENCODE_DISABLE_TELEMETRY) is unaffected — that check does not need Instance context. - // Pre-existing (not introduced by this release); calling it out explicitly here rather than - // leaving the earlier "(tracked separately)" wording, which claimed a tracking issue that - // does not currently exist. + // `telemetry.disabled` config key — with no env var set — can therefore still have early + // events transmitted. The env-var opt-out (ALTIMATE_TELEMETRY_DISABLED / + // OPENCODE_DISABLE_TELEMETRY) is unaffected — that check needs no Instance context. + // Pre-existing and NOT specific to this event: every event emitted from CLI middleware + // shares the same gate. It is called out here because this event's volume grew ~30x when + // the shell and extension installers began writing the marker, so the exposure window is + // now routinely hit rather than theoretical. Fixing it belongs in telemetry init (make + // Config resolvable there, or adopt an explicit fail-closed policy module-wide) rather + // than in this function, which cannot resolve config without duplicating the merge and + // JSONC semantics of config/config.ts. const machineIdPath = path.join(os.homedir(), ".altimate", "machine-id") const isUpgrade = fs.existsSync(machineIdPath) // altimate_change end @@ -64,6 +147,7 @@ export function showWelcomeBannerIfNeeded(): void { session_id: "", version: installedVersion, is_upgrade: isUpgrade, + install_method: readInstallMethod(dataDir), }) // altimate_change end diff --git a/packages/opencode/src/index.ts b/packages/opencode/src/index.ts index 69da571611..444860b412 100644 --- a/packages/opencode/src/index.ts +++ b/packages/opencode/src/index.ts @@ -120,15 +120,23 @@ let cli = yargs(args) } // altimate_change end + // altimate_change start - welcome banner on first run after install/upgrade + // + // MUST run before Telemetry.init(). The banner derives `first_launch.is_upgrade` + // by probing whether ~/.altimate/machine-id already exists, and init() mints that + // file. Ordering it first makes the probe unconditionally correct instead of + // depending on doInit() happening to yield at `await Config.get()` before the + // mint — an invariant an added await would silently break, flipping every install + // to is_upgrade: true. Telemetry.track() buffers until init completes, so nothing + // is lost by emitting before init. + showWelcomeBannerIfNeeded() + // altimate_change end + // altimate_change start - telemetry init // Initialize telemetry early so events from MCP, engine, auth are captured. // init() is idempotent — safe to call again later in session prompt. Telemetry.init().catch(() => {}) // altimate_change end - - // altimate_change start - welcome banner on first run after install/upgrade - showWelcomeBannerIfNeeded() - // altimate_change end }) .usage("") .completion("completion", "generate shell completion script") diff --git a/packages/opencode/test/cli/welcome.test.ts b/packages/opencode/test/cli/welcome.test.ts index 0c87cf0bba..dd82b9e6b5 100644 --- a/packages/opencode/test/cli/welcome.test.ts +++ b/packages/opencode/test/cli/welcome.test.ts @@ -1,7 +1,8 @@ -import { describe, test, expect, beforeEach, afterEach, mock } from "bun:test" +import { describe, test, expect, beforeEach, afterEach, mock, spyOn } from "bun:test" import fs from "fs" import path from "path" import os from "os" +import { Telemetry } from "@/altimate/telemetry" describe("showWelcomeBannerIfNeeded", () => { let tmpDir: string @@ -70,4 +71,157 @@ describe("showWelcomeBannerIfNeeded", () => { const { showWelcomeBannerIfNeeded } = await import("../../src/cli/welcome") expect(() => showWelcomeBannerIfNeeded()).not.toThrow() }) + + // altimate_change start — first_launch is the only install metric, and after AI-8448 the curl and + // PowerShell installers feed it too. These assert the two fields the install dashboard reads. + describe("first_launch event", () => { + const dataFiles = (version = "1.2.3", source?: string) => { + const dir = path.join(tmpDir, "altimate-code") + fs.writeFileSync(path.join(dir, ".installed-version"), version) + if (source !== undefined) fs.writeFileSync(path.join(dir, ".install-source"), source) + return dir + } + + /** + * The machine-id probe reads os.homedir(), which must be stubbed rather than + * driven through $HOME: Bun resolves homedir() once at startup and ignores + * later mutation of process.env.HOME. Without this the result depends on + * whether the developer running the suite has ever launched the CLI. + */ + function withHome(home: string, fn: () => T): T { + const spy = spyOn(os, "homedir").mockImplementation(() => home) + try { + return fn() + } finally { + spy.mockRestore() + } + } + + function captureEvents() { + const events: Telemetry.Event[] = [] + spyOn(Telemetry, "track").mockImplementation((e: Telemetry.Event) => { + events.push(e) + }) + return events + } + + afterEach(() => mock.restore()) + + test("a machine with no prior identity reports is_upgrade false — the brand-new-install signal", async () => { + dataFiles() + const events = captureEvents() + const cleanHome = fs.mkdtempSync(path.join(os.tmpdir(), "welcome-home-")) + + const { showWelcomeBannerIfNeeded } = await import("../../src/cli/welcome") + withHome(cleanHome, () => showWelcomeBannerIfNeeded()) + + const e = events[0] as any + expect(e.type).toBe("first_launch") + expect(e.is_upgrade).toBe(false) + expect(e.version).toBe("1.2.3") + fs.rmSync(cleanHome, { recursive: true, force: true }) + }) + + test("a pre-existing machine-id reports is_upgrade true", async () => { + dataFiles() + const events = captureEvents() + const usedHome = fs.mkdtempSync(path.join(os.tmpdir(), "welcome-home-")) + fs.mkdirSync(path.join(usedHome, ".altimate"), { recursive: true }) + fs.writeFileSync(path.join(usedHome, ".altimate", "machine-id"), "8f1c0c4e-0a5e-4f4e-9c1a-2b3c4d5e6f70") + + const { showWelcomeBannerIfNeeded } = await import("../../src/cli/welcome") + withHome(usedHome, () => showWelcomeBannerIfNeeded()) + + expect((events[0] as any).is_upgrade).toBe(true) + fs.rmSync(usedHome, { recursive: true, force: true }) + }) + + test("attributes the installer that wrote the marker and consumes the source file", async () => { + const dir = dataFiles("1.2.3", "curl") + const events = captureEvents() + const cleanHome = fs.mkdtempSync(path.join(os.tmpdir(), "welcome-home-")) + + const { showWelcomeBannerIfNeeded } = await import("../../src/cli/welcome") + withHome(cleanHome, () => showWelcomeBannerIfNeeded()) + + expect((events[0] as any).install_method).toBe("curl") + // Left behind, it would be attributed to the next install whose installer wrote none. + expect(fs.existsSync(path.join(dir, ".install-source"))).toBe(false) + fs.rmSync(cleanHome, { recursive: true, force: true }) + }) + + test("attributes the VS Code extension's native installer", async () => { + // Highest-volume installer: it pulls from GitHub releases directly, so without + // this value its installs would report "unknown" and be indistinguishable from + // pre-field markers. + dataFiles("1.2.3", "vscode-extension") + const events = captureEvents() + const cleanHome = fs.mkdtempSync(path.join(os.tmpdir(), "welcome-home-")) + + const { showWelcomeBannerIfNeeded } = await import("../../src/cli/welcome") + withHome(cleanHome, () => showWelcomeBannerIfNeeded()) + + expect((events[0] as any).install_method).toBe("vscode-extension") + fs.rmSync(cleanHome, { recursive: true, force: true }) + }) + + test("an absent source file reports unknown rather than dropping the event", async () => { + dataFiles("1.2.3") + const events = captureEvents() + const cleanHome = fs.mkdtempSync(path.join(os.tmpdir(), "welcome-home-")) + + const { showWelcomeBannerIfNeeded } = await import("../../src/cli/welcome") + withHome(cleanHome, () => showWelcomeBannerIfNeeded()) + + // Upgrades from a version whose installer predates the source file land here. + expect((events[0] as any).install_method).toBe("unknown") + fs.rmSync(cleanHome, { recursive: true, force: true }) + }) + + test("an unrecognised source value cannot mint a new dimension", async () => { + dataFiles("1.2.3", "hand-edited-nonsense") + const events = captureEvents() + const cleanHome = fs.mkdtempSync(path.join(os.tmpdir(), "welcome-home-")) + + const { showWelcomeBannerIfNeeded } = await import("../../src/cli/welcome") + withHome(cleanHome, () => showWelcomeBannerIfNeeded()) + + expect((events[0] as any).install_method).toBe("unknown") + fs.rmSync(cleanHome, { recursive: true, force: true }) + }) + + test("clears a directory-shaped source file rather than leaving it forever", async () => { + // unlinkSync throws EPERM/EISDIR on a directory. If it were used here, a + // directory-shaped .install-source would survive every launch and pin + // install_method to "unknown" permanently. + const dir = path.join(tmpDir, "altimate-code") + fs.writeFileSync(path.join(dir, ".installed-version"), "1.2.3") + fs.mkdirSync(path.join(dir, ".install-source"), { recursive: true }) + const events = captureEvents() + const cleanHome = fs.mkdtempSync(path.join(os.tmpdir(), "welcome-home-")) + try { + const { showWelcomeBannerIfNeeded } = await import("../../src/cli/welcome") + withHome(cleanHome, () => showWelcomeBannerIfNeeded()) + + expect((events[0] as any).install_method).toBe("unknown") + expect(fs.existsSync(path.join(dir, ".install-source"))).toBe(false) + } finally { + fs.rmSync(cleanHome, { recursive: true, force: true }) + } + }) + + test("an empty marker emits nothing and clears the orphaned source file", async () => { + const dir = dataFiles("", "curl") + const events = captureEvents() + const cleanHome = fs.mkdtempSync(path.join(os.tmpdir(), "welcome-home-")) + + const { showWelcomeBannerIfNeeded } = await import("../../src/cli/welcome") + withHome(cleanHome, () => showWelcomeBannerIfNeeded()) + + expect(events).toHaveLength(0) + expect(fs.existsSync(path.join(dir, ".install-source"))).toBe(false) + fs.rmSync(cleanHome, { recursive: true, force: true }) + }) + }) + // altimate_change end }) diff --git a/packages/opencode/test/install/install-telemetry.test.ts b/packages/opencode/test/install/install-telemetry.test.ts new file mode 100644 index 0000000000..12c34bfaa1 --- /dev/null +++ b/packages/opencode/test/install/install-telemetry.test.ts @@ -0,0 +1,340 @@ +/** + * altimate_change — install telemetry (AI-8448). + * + * `first_launch` is the only install metric, and it is triggered by a marker file rather than by + * the installer talking to the network. Before this, only npm's postinstall wrote that marker, so + * moving the advertised install path to altimate.sh/install took installs out of instrumentation + * and showed up as a dip in the dashboard. + * + * Coverage is in two layers. Source-level assertions (matching windows-install.test.ts) pin the + * details that fail silently — the data-dir expression, the "unknown" fallback, ordering relative + * to the install dispatch. Those cannot catch a syntax error or a bad variable scope, so the + * "executed" describe below runs the real bash installer through its `--binary` path (no network, + * no GitHub) against a throwaway HOME and asserts the files the CLI actually reads. + * + * install.ps1 gets source-level assertions here plus a syntax parse and real execution in + * test/windows/install.Tests.ps1, which AST-extracts Write-InstallMarker and runs it against a + * temp profile. NOTE: the Pester suite's *subprocess* tests deliberately stop the installer + * early (via -Help / an unknown -Version) so nothing is downloaded, so they never reach the + * marker block — the AST-extracted tests are its only runtime coverage. That suite runs under + * pwsh, not powershell.exe, so Windows PowerShell 5.1 — the documented entrypoint and the + * reason for -Encoding ascii — is still not exercised anywhere. + */ +import { describe, expect, test, afterEach, spyOn, mock } from "bun:test" +import { readFileSync, existsSync, mkdtempSync, rmSync, writeFileSync, readdirSync } from "node:fs" +import { spawnSync } from "node:child_process" +import { join } from "node:path" +import os from "os" +import path from "path" +import { Telemetry } from "@/altimate/telemetry" + +const REPO_ROOT = join(import.meta.dir, "..", "..", "..", "..") +const INSTALL_SH = readFileSync(join(REPO_ROOT, "install"), "utf-8") +const INSTALL_PS1 = readFileSync(join(REPO_ROOT, "install.ps1"), "utf-8") +const WELCOME_SRC = readFileSync(join(REPO_ROOT, "packages/opencode/src/cli/welcome.ts"), "utf-8") + +describe("install — post-install marker", () => { + test("writes the marker the CLI reads", () => { + expect(INSTALL_SH).toMatch(/\.installed-version/) + expect(INSTALL_SH).toMatch(/write_install_marker/) + }) + + test("resolves the data dir exactly as welcome.ts does", () => { + // welcome.ts: XDG_DATA_HOME, else /.local/share, then /altimate-code. + expect(INSTALL_SH).toMatch(/\$\{XDG_DATA_HOME:-\$HOME\/\.local\/share\}\/altimate-code/) + expect(WELCOME_SRC).toMatch(/XDG_DATA_HOME \|\| path\.join\(os\.homedir\(\), "\.local", "share"\)/) + }) + + test("falls back to a non-empty version when the release could not be resolved", () => { + // An empty marker is deleted unread (welcome.ts), so an unresolved version would + // otherwise lose the install entirely — the exact case check_version leaves empty. + expect(INSTALL_SH).toMatch(/specific_version:-unknown/) + }) + + test("records the install method it was given", () => { + expect(INSTALL_SH).toMatch(/\.install-source/) + expect(INSTALL_SH).toMatch(/printf '%s' "\$marker_source"/) + // The parameter must actually be bound, or set -u aborts the install. + expect(INSTALL_SH).toMatch(/local marker_source="\$1"/) + }) + + test("marker is written after the install actually happened, not before", () => { + // Ordering matters twice: check_version exits 0 early when the requested version is + // already present (no install, so no event), and a marker written ahead of a failed + // download would report an install that never landed. + const dispatch = INSTALL_SH.indexOf(" download_and_install") + // Both branches call it after their install step; neither before. + const curlCall = INSTALL_SH.indexOf('write_install_marker "curl"') + const localCall = INSTALL_SH.indexOf('write_install_marker "local"') + const binaryDispatch = INSTALL_SH.indexOf(" install_from_binary") + expect(dispatch).toBeGreaterThan(0) + expect(curlCall).toBeGreaterThan(dispatch) + expect(localCall).toBeGreaterThan(binaryDispatch) + }) + + test("publishes the companion before the trigger", () => { + const start = INSTALL_SH.indexOf("write_install_marker() {") + // Code only — the comment above the writes names .installed-version first while + // explaining why it must be written last. + const fn = INSTALL_SH.slice(start, INSTALL_SH.indexOf("\n}", start)) + .split("\n") + .filter((l) => !l.trim().startsWith("#")) + .join("\n") + const source = fn.indexOf(".install-source") + const version = fn.indexOf(".installed-version") + expect(source).toBeGreaterThan(0) + expect(version).toBeGreaterThan(source) + }) + + test("publishes the trigger atomically, not with a truncating write", () => { + // Companion-first only closes the attribution window. A plain redirect truncates + // before filling, so a reader mid-write can see an EMPTY .installed-version and + // delete it unread — losing the install. All four writers use temp+rename. + const start = INSTALL_SH.indexOf("write_install_marker() {") + const fn = INSTALL_SH.slice(start, INSTALL_SH.indexOf("\n}", start)) + expect(fn).toMatch(/installed-version\.\$\$/) + expect(fn).toMatch(/mv -f/) + }) + + test("attributes --binary installs as local, not curl", () => { + // That branch sets specific_version="local"; folding it into the curl metric would + // misreport both source and version. + expect(INSTALL_SH).toMatch(/write_install_marker "local"/) + expect(INSTALL_SH).toMatch(/write_install_marker "curl"/) + }) + + test("marker failures cannot abort the install", () => { + // A read-only or absent $HOME must cost the event, never the install. + const start = INSTALL_SH.indexOf("write_install_marker() {") + const fn = INSTALL_SH.slice(start, INSTALL_SH.indexOf("\n}", start)) + expect(fn).toMatch(/mkdir -p "\$data_dir" 2>\/dev\/null \|\| return 0/) + expect(fn.match(/\|\| return 0/g)?.length).toBeGreaterThanOrEqual(3) + }) +}) + +// altimate_change — execution coverage for the bash marker writer. +// +// Every assertion above reads source text, which cannot catch a syntax error, a bad +// variable scope, or a wrong path inside write_install_marker(). These run the real +// installer end to end via its `--binary` path (no network, no GitHub) against a +// throwaway HOME, and assert the files the CLI actually reads. +describe("install — marker writer, executed", () => { + const INSTALL_SH_PATH = join(REPO_ROOT, "install") + // Bash-only; skip on Windows runners rather than reporting a false pass. + const shtest = process.platform === "win32" ? test.skip : test + + /** Runs `./install --binary ` with an isolated HOME/XDG and returns the marker dir. */ + function runInstaller(env: Record): { code: number; stderr: string; home: string } { + const home = mkdtempSync(join(os.tmpdir(), "install-exec-home-")) + const fakeBin = join(home, "altimate") + writeFileSync(fakeBin, "#!/bin/sh\necho 1.2.3\n", { mode: 0o755 }) + const res = spawnSync("bash", [INSTALL_SH_PATH, "--binary", fakeBin, "--no-modify-path"], { + encoding: "utf-8", + env: { ...process.env, HOME: home, ...env }, + }) + return { code: res.status ?? -1, stderr: res.stderr ?? "", home } + } + + shtest("writes both marker files under the default data dir", () => { + const { code, home, stderr } = runInstaller({ XDG_DATA_HOME: "" }) + try { + expect(code).toBe(0) + const dir = join(home, ".local", "share", "altimate-code") + // A non-empty version is required — the CLI deletes an empty marker unread. + expect(readFileSync(join(dir, ".installed-version"), "utf-8").trim().length).toBeGreaterThan(0) + // "local", not "curl": runInstaller uses --binary, which is deliberately attributed + // separately so a dev/air-gapped install cannot inflate the curl metric. + expect(readFileSync(join(dir, ".install-source"), "utf-8").trim()).toBe("local") + expect(stderr).not.toMatch(/syntax error|command not found/) + } finally { + rmSync(home, { recursive: true, force: true }) + } + }) + + shtest("honours $XDG_DATA_HOME", () => { + const xdg = mkdtempSync(join(os.tmpdir(), "install-exec-xdg-")) + const { code, home } = runInstaller({ XDG_DATA_HOME: xdg }) + try { + expect(code).toBe(0) + expect(existsSync(join(xdg, "altimate-code", ".installed-version"))).toBe(true) + // Must NOT also land in the home-relative fallback. + expect(existsSync(join(home, ".local", "share", "altimate-code", ".installed-version"))).toBe(false) + } finally { + rmSync(xdg, { recursive: true, force: true }) + rmSync(home, { recursive: true, force: true }) + } + }) + + shtest("an unwritable data dir does not fail the install, and writes no marker", () => { + // Parent is a regular file, so mkdir -p cannot succeed under it. + const blocked = mkdtempSync(join(os.tmpdir(), "install-exec-blocked-")) + const asFile = join(blocked, "not-a-dir") + writeFileSync(asFile, "x") + const { code, home } = runInstaller({ XDG_DATA_HOME: join(asFile, "nested") }) + try { + expect(code).toBe(0) + // Exit 0 alone would pass with the marker writer deleted entirely. Asserting + // no marker exists proves the fixture actually hit the failure path. + expect(existsSync(join(asFile, "nested", "altimate-code", ".installed-version"))).toBe(false) + } finally { + rmSync(blocked, { recursive: true, force: true }) + rmSync(home, { recursive: true, force: true }) + } + }) + + shtest("leaves no temp file behind from the atomic trigger publish", () => { + // The trigger is written to `.installed-version.$$` then mv'd. A leftover temp + // file would be read by nobody and would accumulate one per install. + const xdg = mkdtempSync(join(os.tmpdir(), "install-exec-tmp-")) + const { code, home } = runInstaller({ XDG_DATA_HOME: xdg }) + try { + expect(code).toBe(0) + const entries = readdirSync(join(xdg, "altimate-code")) + expect(entries.sort()).toEqual([".install-source", ".installed-version"]) + } finally { + rmSync(xdg, { recursive: true, force: true }) + rmSync(home, { recursive: true, force: true }) + } + }) +}) + +describe("install.ps1 — post-install marker", () => { + const markerBlock = INSTALL_PS1.slice( + INSTALL_PS1.indexOf("Post-install marker"), + INSTALL_PS1.indexOf("PATH (user scope"), + ) + // Comments in this block deliberately name %LOCALAPPDATA% to explain why it is wrong, + // so the "never LOCALAPPDATA" assertion has to look at code only. + const markerCode = markerBlock + .split("\n") + .filter((l) => !l.trim().startsWith("#")) + .join("\n") + + test("writes the marker and attributes itself as powershell", () => { + expect(markerCode).toMatch(/\.installed-version/) + expect(markerCode).toMatch(/"powershell"/) + }) + + test("uses the XDG/.local\\share path, never LOCALAPPDATA", () => { + // The CLI reads the data dir through Node's os.homedir() and never consults + // %LOCALAPPDATA%, so a marker written there would be silently ignored at read time. + expect(markerCode).toMatch(/XDG_DATA_HOME/) + // [IO.Path]::Combine(USERPROFILE, ".local", "share") — Combine rather than a literal + // path so PSDrive resolution cannot throw, matching welcome.ts's /.local/share. + expect(markerCode).toMatch(/USERPROFILE/) + expect(markerCode).toMatch(/"\.local", ?"share"/) + expect(markerCode).not.toMatch(/LOCALAPPDATA/) + }) + + test("falls back to a non-empty version and cannot abort the install", () => { + expect(markerCode).toMatch(/"unknown"/) + expect(markerCode).toMatch(/} catch \{/) + }) + + test("computes its paths INSIDE the try, not above it", () => { + // $ErrorActionPreference is "Stop", so a null $env:USERPROFILE or an XDG_DATA_HOME + // naming a bad PSDrive throws. Above the try that terminates the installer after the + // binary is placed but before the PATH write — installed, but not on PATH. The old + // "} catch {" assertion passed with the assignments outside, so pin the order. + const tryIdx = markerCode.indexOf("try {") + const rootIdx = markerCode.indexOf("$dataRoot =") + const dirIdx = markerCode.indexOf("$dataDir =") + expect(tryIdx).toBeGreaterThan(0) + expect(rootIdx).toBeGreaterThan(tryIdx) + expect(dirIdx).toBeGreaterThan(tryIdx) + }) + + test("publishes the companion before the trigger", () => { + // .installed-version is the reader's trigger; writing it first would let a CLI + // starting in between report "unknown", or observe a truncated (empty) file and + // drop the install entirely. + const source = markerCode.indexOf('".install-source"') + const version = markerCode.indexOf('".installed-version"') + expect(source).toBeGreaterThan(0) + expect(version).toBeGreaterThan(source) + }) + + test("publishes the trigger atomically", () => { + expect(markerCode).toMatch(/installed-version\.tmp/) + expect(markerCode).toMatch(/Move-Item -Force/) + }) + + test("writes without a BOM", () => { + // The documented entrypoint is `powershell -c "irm ... | iex"` — Windows PowerShell 5.1, + // where `-Encoding utf8` prepends a UTF-8 BOM. install-source is matched against a fixed + // allowlist, so a BOM would silently degrade every PowerShell install to "unknown". + expect(markerCode).not.toMatch(/-Encoding utf8/) + expect(markerCode.match(/-Encoding ascii/g)).toHaveLength(2) + }) +}) + +describe("is_upgrade ordering", () => { + test("index.ts calls the welcome banner BEFORE Telemetry.init()", () => { + // The primary protection for is_upgrade. The banner probes whether + // ~/.altimate/machine-id exists and init() mints it, so the banner must run first. + // Swapping these silently flips every install to is_upgrade: true. + // Code only: the comment above the banner call names Telemetry.init() to explain the + // ordering, so a naive indexOf over the raw source finds the comment, not the call. + const src = readFileSync(join(REPO_ROOT, "packages/opencode/src/index.ts"), "utf-8") + .split("\n") + .filter((l) => !l.trim().startsWith("//")) + .join("\n") + const banner = src.indexOf("showWelcomeBannerIfNeeded()") + const init = src.indexOf("Telemetry.init()") + expect(banner).toBeGreaterThan(0) + expect(init).toBeGreaterThan(0) + expect(banner).toBeLessThan(init) + }) + + afterEach(() => mock.restore()) + + test("an unawaited Telemetry.init() has not minted a machine-id when the banner runs", async () => { + // Defence in depth behind the ordering test above. index.ts now runs the banner before + // init(), so is_upgrade no longer depends on this — but init() is also called from other + // entrypoints, and any caller that emits an install-shaped event before awaiting it still + // relies on the mint happening after doInit's first await (Config.get). This pins that + // behaviour so a refactor making the mint synchronous is caught here rather than showing + // up as every install reporting is_upgrade: true. + const origCs = process.env.APPLICATIONINSIGHTS_CONNECTION_STRING + const origDisabled = process.env.ALTIMATE_TELEMETRY_DISABLED + // Both gates, not just one: doInit() returns before minting if EITHER is set, which + // would make the "machine-id exists after await" assertion fail on a machine or CI + // runner that exports OPENCODE_DISABLE_TELEMETRY. + const origOpencodeDisabled = process.env.OPENCODE_DISABLE_TELEMETRY + const tmpHome = mkdtempSync(join(os.tmpdir(), "install-telemetry-home-")) + spyOn(os, "homedir").mockImplementation(() => tmpHome) + spyOn(global, "fetch").mockImplementation((async () => new Response("", { status: 200 })) as any) + + try { + // The baked-in sink is refused under a test runner, and doInit returns before minting + // when it has no connection string — which would make this test vacuously pass. + delete process.env.ALTIMATE_TELEMETRY_DISABLED + delete process.env.OPENCODE_DISABLE_TELEMETRY + process.env.APPLICATIONINSIGHTS_CONNECTION_STRING = + "InstrumentationKey=k;IngestionEndpoint=https://example.invalid" + // init() is `initPromise ??= doInit()`; shutdown() is the only seam that clears it, so + // an earlier init in this process would otherwise be handed back already resolved. + await Telemetry.shutdown() + + const pending = Telemetry.init() + const machineIdPath = path.join(tmpHome, ".altimate", "machine-id") + + // The instant that matters — the same turn of the event loop in which index.ts calls + // showWelcomeBannerIfNeeded(). + expect(existsSync(machineIdPath)).toBe(false) + + await pending + // Proves the assertion above is not vacuous: this path really does mint, just later. + expect(existsSync(machineIdPath)).toBe(true) + } finally { + await Telemetry.shutdown() + if (origCs !== undefined) process.env.APPLICATIONINSIGHTS_CONNECTION_STRING = origCs + else delete process.env.APPLICATIONINSIGHTS_CONNECTION_STRING + if (origDisabled !== undefined) process.env.ALTIMATE_TELEMETRY_DISABLED = origDisabled + else delete process.env.ALTIMATE_TELEMETRY_DISABLED + if (origOpencodeDisabled !== undefined) process.env.OPENCODE_DISABLE_TELEMETRY = origOpencodeDisabled + else delete process.env.OPENCODE_DISABLE_TELEMETRY + rmSync(tmpHome, { recursive: true, force: true }) + } + }) +}) diff --git a/packages/opencode/test/install/postinstall.test.ts b/packages/opencode/test/install/postinstall.test.ts index 9a027abd95..dab9eb3d2a 100644 --- a/packages/opencode/test/install/postinstall.test.ts +++ b/packages/opencode/test/install/postinstall.test.ts @@ -107,6 +107,44 @@ describe("postinstall.mjs", () => { const markerPath = path.join(dataDir, "altimate-code", ".installed-version") expect(fs.existsSync(markerPath)).toBe(true) expect(fs.readFileSync(markerPath, "utf-8")).toBe("2.5.0") + // altimate_change — the curl and PowerShell installers write the same marker, so + // first_launch can only separate npm volume from theirs via this companion file. + const sourcePath = path.join(dataDir, "altimate-code", ".install-source") + expect(fs.readFileSync(sourcePath, "utf-8")).toBe("npm") + }) + + // altimate_change — publish order, asserted on real output rather than source text. + // `.installed-version` is the CLI's trigger: it returns early unless that file + // exists, then consumes `.install-source`. Writing the trigger first leaves two + // windows — a reader in between reports install_method "unknown", and because + // writeFileSync truncates first, a reader can observe an EMPTY `.installed-version` + // and delete it unread, losing the install. npm is the only channel that was + // counted before this feature, so order matters most here. + test("writes the companion before the trigger", () => { + const { dir, cleanup: c } = installTmpdir() + cleanup = c + + createMainPackageDir(dir, { version: "3.1.0" }) + createBinaryPackage(dir) + + const dataDir = path.join(dir, "xdg-data") + expect(runPostinstall(dir, { XDG_DATA_HOME: dataDir }).exitCode).toBe(0) + + const markerDir = path.join(dataDir, "altimate-code") + const source = fs.statSync(path.join(markerDir, ".install-source")).mtimeMs + const trigger = fs.statSync(path.join(markerDir, ".installed-version")).mtimeMs + // mtime resolution can tie on fast filesystems; the trigger must never be older. + expect(trigger).toBeGreaterThanOrEqual(source) + + // Source-level guard, since equal mtimes make the check above weak on its own. + const src = fs.readFileSync(POSTINSTALL_SCRIPT, "utf-8") + expect(src.indexOf('".install-source"')).toBeLessThan(src.indexOf('".installed-version"')) + + // Trigger is published via temp+rename, so no temp file may survive. A plain + // truncating write would let a reader observe an empty trigger and delete it + // unread, losing the install. + expect(src).toMatch(/renameSync/) + expect(fs.readdirSync(markerDir).sort()).toEqual([".install-source", ".installed-version"]) }) test("upgrade marker strips v prefix from version", () => { diff --git a/packages/opencode/test/telemetry/telemetry.test.ts b/packages/opencode/test/telemetry/telemetry.test.ts index a9490888e8..437f559bc9 100644 --- a/packages/opencode/test/telemetry/telemetry.test.ts +++ b/packages/opencode/test/telemetry/telemetry.test.ts @@ -1629,6 +1629,7 @@ describe("telemetry.memory", () => { session_id: "", version: "0.5.9", is_upgrade: false, + install_method: "curl", }) }).not.toThrow() }) diff --git a/test/windows/install.Tests.ps1 b/test/windows/install.Tests.ps1 index 118f7cbcd1..f1e8ddce19 100644 --- a/test/windows/install.Tests.ps1 +++ b/test/windows/install.Tests.ps1 @@ -1,10 +1,18 @@ # Pester behavioral tests for install.ps1 (the Windows standalone installer). # -# These run the real script as a subprocess on Windows PowerShell so they -# exercise actual behavior - not just substring matching. They deliberately -# stop the script early (via -Help or an unknown -Version) so no 268 MB binary -# is ever downloaded, while still covering the risky branches: argument -# parsing, the WOW64 architecture fix, and unknown-version rejection. +# Two layers, because no single one reaches everything: +# +# 1. SUBPROCESS tests run the real script and exercise actual behavior - not just +# substring matching. They deliberately stop it early (via -Help or an unknown +# -Version) so no 268 MB binary is ever downloaded, covering argument parsing, +# the WOW64 architecture fix, and unknown-version rejection. Because they stop +# early they never reach anything past version resolution. +# +# 2. AST-EXTRACTED tests parse install.ps1, pull a single function out of the tree +# and dot-source it, so code the subprocess tests can never reach is still +# executed. Used for Test-Checksum and Write-InstallMarker. These must set +# $ErrorActionPreference = "Stop" themselves to match the installer's own +# semantics - see the note in the Write-InstallMarker BeforeAll. # # Run locally on Windows: Invoke-Pester ./test/windows/install.Tests.ps1 # CI runs this on windows-latest (see .github/workflows/ci.yml). @@ -184,3 +192,131 @@ Describe "install.ps1 Test-Checksum" { Remove-Item $tmp -Force } } + +# --------------------------------------------------------------------------- +# Write-InstallMarker (install telemetry — AI-8448) +# --------------------------------------------------------------------------- +# The subprocess tests above stop the installer via -Help / unknown -Version, so +# they never reach the marker block. It is AST-extracted and executed here instead, +# the same way Test-Checksum is, against a temp profile. This is the only runtime +# coverage the PowerShell writer has. +Describe "install.ps1 Write-InstallMarker" { + BeforeAll { + $src = Get-Content -Raw $script:InstallScript + $tokens = $null; $errors = $null + $ast = [System.Management.Automation.Language.Parser]::ParseInput($src, [ref]$tokens, [ref]$errors) + $def = $ast.Find({ + param($n) + $n -is [System.Management.Automation.Language.FunctionDefinitionAst] -and $n.Name -eq "Write-InstallMarker" + }, $true) + if (-not $def) { throw "Write-InstallMarker not found in install.ps1" } + . ([ScriptBlock]::Create($def.Extent.Text)) + + # MUST match the installer's own error semantics. install.ps1:28 sets + # $ErrorActionPreference = "Stop" at script scope, and that is the entire reason + # Write-InstallMarker needs its try/catch: under "Stop" a failing cmdlet is + # TERMINATING. Dot-sourcing the function out of the AST lands it in a session + # where pwsh's default "Continue" applies, under which a New-Item failure merely + # writes to the error stream — so "Should -Not -Throw" would pass with the + # try/catch deleted, and the test could not fail. + $ErrorActionPreference = "Stop" + $script:ErrorActionPreference = "Stop" + + # Mirrors welcome.ts::getDataDir(): $XDG_DATA_HOME, else /.local/share. + function Get-MarkerDir { + param([string]$Root) + [IO.Path]::Combine($Root, "altimate-code") + } + } + + BeforeEach { + $script:Sandbox = Join-Path ([IO.Path]::GetTempPath()) ("marker-" + [guid]::NewGuid().ToString("N")) + New-Item -ItemType Directory -Force -Path $script:Sandbox | Out-Null + $script:OldXdg = $env:XDG_DATA_HOME + $script:OldProfile = $env:USERPROFILE + } + + AfterEach { + $env:XDG_DATA_HOME = $script:OldXdg + $env:USERPROFILE = $script:OldProfile + Remove-Item -Recurse -Force $script:Sandbox -ErrorAction SilentlyContinue + } + + It "writes both marker files with byte-exact content and no BOM" { + $env:XDG_DATA_HOME = $script:Sandbox + Write-InstallMarker -Version "1.2.3" + + $dir = Get-MarkerDir $script:Sandbox + # Byte-level: a BOM would make install_method fail the CLI's allowlist match. + $verBytes = [IO.File]::ReadAllBytes([IO.Path]::Combine($dir, ".installed-version")) + $srcBytes = [IO.File]::ReadAllBytes([IO.Path]::Combine($dir, ".install-source")) + [System.Text.Encoding]::ASCII.GetString($verBytes) | Should -BeExactly "1.2.3" + [System.Text.Encoding]::ASCII.GetString($srcBytes) | Should -BeExactly "powershell" + $verBytes[0] | Should -Not -Be 0xEF + $srcBytes[0] | Should -Not -Be 0xEF + } + + It "strips a leading v and falls back to 'unknown' for an unresolved version" { + $env:XDG_DATA_HOME = $script:Sandbox + Write-InstallMarker -Version "v9.9.9" + $dir = Get-MarkerDir $script:Sandbox + Get-Content -Raw ([IO.Path]::Combine($dir, ".installed-version")) | Should -BeExactly "9.9.9" + + # Empty version must NOT produce an empty marker — the CLI deletes those unread. + Write-InstallMarker -Version "" + Get-Content -Raw ([IO.Path]::Combine($dir, ".installed-version")) | Should -BeExactly "unknown" + } + + It "falls back to /.local/share when XDG_DATA_HOME is unset" { + $env:XDG_DATA_HOME = "" + $env:USERPROFILE = $script:Sandbox + Write-InstallMarker -Version "1.0.0" + $dir = [IO.Path]::Combine($script:Sandbox, ".local", "share", "altimate-code") + Test-Path ([IO.Path]::Combine($dir, ".installed-version")) | Should -BeTrue + } + + It "does not throw when USERPROFILE is empty and XDG_DATA_HOME is unset" { + # The regression guard for path computation inside the try. With + # $ErrorActionPreference = "Stop", computing this outside the try raised a + # terminating error that aborted the installer AFTER the binary was placed but + # BEFORE the PATH write - leaving an installed binary that is not on PATH. + # + # Runs inside the sandbox via Push-Location. [IO.Path]::Combine("", ".local", + # "share") does NOT throw - it returns the RELATIVE path ".local\share" - so the + # marker is created under the current working directory. Without Push-Location + # that is the git checkout root, and every CI run would litter it. + $env:XDG_DATA_HOME = "" + $env:USERPROFILE = "" + Push-Location $script:Sandbox + try { + { Write-InstallMarker -Version "1.0.0" } | Should -Not -Throw + # Documents where an empty profile actually lands: relative to cwd, not $HOME. + Test-Path ([IO.Path]::Combine(".local", "share", "altimate-code", ".installed-version")) | + Should -BeTrue + } finally { + Pop-Location + } + } + + It "does not throw when the data dir cannot be created, and writes no marker" { + # Runs under $ErrorActionPreference = "Stop" (set in BeforeAll), so New-Item's + # failure here is terminating and the try/catch is what makes this pass. Deleting + # the try/catch must fail this case — that is what makes it a real guard. + $blocker = Join-Path $script:Sandbox "blocker" + Set-Content -Path $blocker -Value "x" -NoNewline + $env:XDG_DATA_HOME = [IO.Path]::Combine($blocker, "nested") + + { Write-InstallMarker -Version "1.0.0" } | Should -Not -Throw + + # Proves the fixture actually hit the intended failure path rather than quietly + # succeeding somewhere else: no marker may exist under the blocked root. + Test-Path ([IO.Path]::Combine($blocker, "nested", "altimate-code", ".installed-version")) | + Should -BeFalse + } + + It "runs under the installer's Stop semantics" { + # Guards the BeforeAll above: if this drifts back to "Continue", the two + # "does not throw" cases silently stop being able to fail. + $ErrorActionPreference | Should -Be "Stop" + } +}