Skip to content

feat(cli): non-blocking "new version available" notice (24h-cached npm check, opt-out, CI-safe)#181

Merged
zeshi-du merged 1 commit into
TestSprite:mainfrom
Andy00L:feat/update-notice
Jul 5, 2026
Merged

feat(cli): non-blocking "new version available" notice (24h-cached npm check, opt-out, CI-safe)#181
zeshi-du merged 1 commit into
TestSprite:mainfrom
Andy00L:feat/update-notice

Conversation

@Andy00L

@Andy00L Andy00L commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Users on stale CLIs silently miss fixes: nothing ever tells them a newer
release exists. This adds the standard gh/npm-style advisory: at most once per
24 hours (cache in ~/.testsprite/update-check.json), the CLI asks the npm
registry for the latest dist-tag and prints ONE stderr line when it is
strictly newer than the running version. Hand-rolled (~150 lines, zero new
dependencies), hard 1.5s fetch timeout, and total failure-tolerance: any error
anywhere (network, fs, clock, even the stderr sink) is swallowed — the notice
can never break, delay, or change the exit code of a real command.

All three triage conditions are met:

  1. TESTSPRITE_NO_UPDATE_NOTIFIER (any non-empty value) fully silences it;
  2. DOCUMENTATION.md documents the outbound check explicitly (package name
    only, at most once per 24h, the only outbound call besides the API);
  3. every gate applies: skipped in CI, when stderr is not a TTY, under
    --output json, under --dry-run, and for shell-eval'd output paths.

Related issue

Fixes #122

Type of change

  • New feature (non-breaking change that adds functionality)

Checklist

  • PR targets the main branch.
  • Commits follow Conventional Commits.
  • npm run lint and npm run format:check pass.
  • npm run typecheck passes.
  • npm test passes and coverage stays at or above the 80% gate.
  • New behavior is covered by unit tests (mock-based; no network).
  • No secrets, API keys, internal endpoints, or personal data are included.

Notes for reviewers

Every effect is injected (env, fetch, clock, fs, TTY, stderr) — the 15 unit
tests run with zero real network/fs except one real-fs cache round-trip in a
tmpdir. Semver comparison is numeric with prerelease-sorts-older semantics;
unparseable versions compare equal so garbage can never produce a false
notice. A failed probe still stamps the cache, so a dead registry is retried
at most once per 24h window. The update-notifier npm package was considered
and rejected (dependency tree); this stays dep-free.

Summary by CodeRabbit

  • New Features
    • Added a lightweight “new version available” notice for interactive CLI runs.
  • Bug Fixes
    • Update checks now skip non-interactive runs, JSON output, dry-runs, and completion workflows.
    • Notices are rate-limited (up to once per 24 hours), cached, and fail silently when the check can’t complete.
  • Documentation
    • Expanded configuration docs with update-notice behavior, timing, rate limits, and privacy/safety details.
  • Tests
    • Added comprehensive automated coverage for update-check gating, version parsing, and notification/cache behavior.

@coderabbitai

coderabbitai Bot commented Jul 5, 2026

Copy link
Copy Markdown

Review Change Stack

Walkthrough

Adds a non-blocking update notice: the CLI now checks npm for a newer version on a 24-hour cache, prints one stderr advisory when needed, skips the check in the documented cases, and exposes an env var to disable it. The new behavior is wired into preAction, tested, and documented.

Changes

Update notification feature

Layer / File(s) Summary
Update-check module core: types, cache, gating
src/lib/update-check.ts
Adds the update-check constants, cache and dependency types, cache read/write helpers, and the gating logic for opt-out, CI, TTY, and cache freshness.
Registry fetch, semver comparison, and notify flow
src/lib/update-check.ts
Adds the npm registry probe, tolerant semver comparison, and the fire-and-forget notify flow that stamps cache and writes one stderr notice when a newer version is found.
CLI preAction wiring
src/index.ts
Imports maybeNotifyUpdate, reuses commandPath, and conditionally runs the update check in the CLI preAction hook.
Tests and documentation for update notice
src/lib/update-check.test.ts, DOCUMENTATION.md
Adds tests for gating, fetch parsing, version comparison, and notify/cache behavior, and documents TESTSPRITE_NO_UPDATE_NOTIFIER plus the update notice rules.

Estimated code review effort: 3 (Moderate) | ~25 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: a non-blocking, cached update notice with opt-out and CI-safe gating.
Linked Issues check ✅ Passed The implementation matches #122 with 24h caching, stderr-only notice, opt-out, CI/TTY/json/dry-run/completion gating, and swallowed errors.
Out of Scope Changes check ✅ Passed The docs and tests support the update-notice feature, and no unrelated code changes are apparent.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
DOCUMENTATION.md (1)

443-446: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Doc omits the completion command skip.

The preAction hook in src/index.ts also skips the update check for the completion command (its stdout is shell-evaluated), per the PR objectives, but this section only lists CI/non-TTY/--output json/--dry-run/opt-out as skip conditions.

📝 Suggested doc addition
 check is skipped in CI, when stderr is not a TTY, under `--output json` /
-`--dry-run`, and entirely when `TESTSPRITE_NO_UPDATE_NOTIFIER` is set. Any
+`--dry-run`, for `completion` (its stdout is shell-evaluated), and entirely
+when `TESTSPRITE_NO_UPDATE_NOTIFIER` is set. Any
 failure is silent: the notice can never break or delay a command. This is the
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@DOCUMENTATION.md` around lines 443 - 446, The update-check documentation is
missing one skip condition: the preAction hook in src/index.ts also bypasses the
notice for the completion command because its stdout is shell-evaluated. Update
the section describing when the notifier is skipped so it includes completion
alongside the existing CI, non-TTY, --output json, --dry-run, and
TESTSPRITE_NO_UPDATE_NOTIFIER cases, using the same terminology as the preAction
logic.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@DOCUMENTATION.md`:
- Around line 443-446: The update-check documentation is missing one skip
condition: the preAction hook in src/index.ts also bypasses the notice for the
completion command because its stdout is shell-evaluated. Update the section
describing when the notifier is skipped so it includes completion alongside the
existing CI, non-TTY, --output json, --dry-run, and
TESTSPRITE_NO_UPDATE_NOTIFIER cases, using the same terminology as the preAction
logic.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: e9df77cd-cdc1-4ec9-93c3-752b2a0c0e9a

📥 Commits

Reviewing files that changed from the base of the PR and between e53257d and df46387.

📒 Files selected for processing (4)
  • DOCUMENTATION.md
  • src/index.ts
  • src/lib/update-check.test.ts
  • src/lib/update-check.ts

@zeshi-du

zeshi-du commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Approved — the implementation meets the issue #122 conditions exactly (presence-style opt-out that fully silences before any fs/network work, CI/TTY/json/dry-run gates, 24h cache, single stderr line, never-throw). The DOCUMENTATION.md "Update notice" section is exactly the disclosure we asked for; we'll add a README pointer to it on our side. Conflicting only because #12's NO_COLOR row landed in the same env-var table — rebase onto main and this merges.

@Andy00L Andy00L force-pushed the feat/update-notice branch from df46387 to 1742a90 Compare July 5, 2026 20:02

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
src/lib/update-check.ts (1)

266-272: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

latestKnown is written but never consumed.

The cache persists latestKnown (Line 268, schema at Line 71), yet no code path reads it back to render a notice. Because shouldCheckForUpdate gates purely on lastCheckMs, the advisory only ever fires on the single invocation that performs the probe; subsequent runs within the TTL are gated out even when a newer version is known. On a failed probe the field is also dropped. This is consistent with the documented "at most once per 24h" behavior, so it's not a defect — but the stored field is currently dead data. Consider either surfacing the cached latestKnown on gated runs (gh/npm update-notifier style) or dropping the field to keep the cache shape honest.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/update-check.ts` around lines 266 - 272, The update cache currently
persists latestKnown in writeUpdateCheckCache, but that value is never read back
when shouldCheckForUpdate gates the probe. Update the update-check flow in
update-check.ts so the cached latestKnown is consumed on later runs to render
the notice even when the network check is skipped, or remove latestKnown from
the cache/schema if you intend to keep the current once-per-24h behavior. Make
the change in the code path around writeUpdateCheckCache and the subsequent
semver check so the cache shape matches actual usage.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@src/lib/update-check.ts`:
- Around line 266-272: The update cache currently persists latestKnown in
writeUpdateCheckCache, but that value is never read back when
shouldCheckForUpdate gates the probe. Update the update-check flow in
update-check.ts so the cached latestKnown is consumed on later runs to render
the notice even when the network check is skipped, or remove latestKnown from
the cache/schema if you intend to keep the current once-per-24h behavior. Make
the change in the code path around writeUpdateCheckCache and the subsequent
semver check so the cache shape matches actual usage.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 5a58b1f9-0e54-482c-b3fc-bf3e547f980c

📥 Commits

Reviewing files that changed from the base of the PR and between df46387 and 1742a90.

📒 Files selected for processing (4)
  • DOCUMENTATION.md
  • src/index.ts
  • src/lib/update-check.test.ts
  • src/lib/update-check.ts
✅ Files skipped from review due to trivial changes (1)
  • DOCUMENTATION.md
🚧 Files skipped from review as they are similar to previous changes (2)
  • src/index.ts
  • src/lib/update-check.test.ts

@Andy00L

Andy00L commented Jul 5, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto main. Resolved the env-var table conflict by keeping both rows (NO_COLOR and TESTSPRITE_NO_UPDATE_NOTIFIER). No code changes; ready to merge.

@zeshi-du zeshi-du merged commit c6f946e into TestSprite:main Jul 5, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Hackathon] Non-blocking "new version available" notice (24h-cached npm check, opt-out, CI-safe)

2 participants