Skip to content

fix: bound pagination and guard password file reads#61

Closed
merlinsantiago982-cmd wants to merge 17 commits into
TestSprite:mainfrom
merlinsantiago982-cmd:fix/pagination-password-file-guards
Closed

fix: bound pagination and guard password file reads#61
merlinsantiago982-cmd wants to merge 17 commits into
TestSprite:mainfrom
merlinsantiago982-cmd:fix/pagination-password-file-guards

Conversation

@merlinsantiago982-cmd

@merlinsantiago982-cmd merlinsantiago982-cmd commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Summary

  • stop auto-pagination when the server repeats a cursor or exceeds a hard page budget
  • guard --password-file reads with stat-first validation, regular-file checks, and a 64 KiB size cap
  • add tests for cursor cycles, runaway pagination, valid password files, directory rejection, and oversized password files

Why

A malicious or broken server could keep returning non-null cursors indefinitely, causing the CLI to loop and grow memory. --password-file also read arbitrary paths without checking the target type or size before sending the content as the project password.

Testing

  • npm test -- src/lib/pagination.test.ts src/commands/project.test.ts
  • npm run typecheck
  • npm run format:check
  • npm run lint
  • npm run build

Fixes #58

Summary by CodeRabbit

  • New Features

    • Added safer handling for password files when creating or updating projects, including file validation and a size limit.
    • Added a limit for automatic pagination to prevent runaway requests.
  • Bug Fixes

    • Rejects invalid password-file paths, oversized password files, and repeated pagination cursors before making network calls.
    • Prevents infinite or excessive auto-pagination by stopping at a configured maximum page count.

zeshi-du and others added 17 commits June 11, 2026 04:49
The Lint & Format job runs prettier --check over workflow YAML; the
aligned trailing comment failed it on every push.
Updated the README with a new link and added a video description.
- prettier-clean the README video block (fixes CI format:check on main)
- bump version to 0.1.1
- CHANGELOG: add [0.1.1] docs-only entry

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
NPM_TOKEN secret was never configured, so tag-triggered releases failed
with ENEEDAUTH. Auth now uses npm trusted publishing (configured on
npmjs.com for this repo + release.yaml):

- drop NODE_AUTH_TOKEN env (empty token would shadow OIDC auth)
- upgrade npm to >= 11.5.1 (trusted publishing requirement; Node 22 bundles 10.x)
- bump checkout/setup-node to v5 ahead of the June 16 Node 20 runner cutoff
ci(release): switch npm publish to OIDC trusted publishing
- Onboarding consolidated into `testsprite setup` (formerly `init`); the
  granular auth commands remain as hidden, deprecated aliases.
- CLI reports its version in the User-Agent header.
- README: the launch video no longer renders as a bare URL on npm.
create-batch --run launched its first concurrencyLimit triggers in
parallel, but the steady-state loop awaited each subsequent job to
fully finish (trigger + full --wait poll) before launching the next
one. Effective concurrency dropped to 1 after the initial wave
regardless of --max-concurrency.

Switch to the launch-then-race pattern already used by the other
three fan-outs in this file: launch up to the limit, relaunch on
each completion via startNext(), never await a whole job inline.

Add a regression test using equal-delay trigger responses so the
first wave settles in the same microtask batch, which is the exact
condition that exposed the bug.
…urrency

fix(test): prevent batch-run scheduler from serializing after first wave
…estSprite#7)

test code get --out <path> opened (truncated) the destination file
before the network request. If the GET then failed, or hit the
"no code generated yet" branch which writes nothing, the user's
pre-existing --out file was left emptied with no way to recover it.

Write to a sibling temp file instead and rename it onto the real
path only after a successful, complete write. Mirrors the atomic
rename contract bundle.ts already uses for multi-file bundles.

Add regression tests for both failure modes: a failing fetch and
the no-code-yet branch. Both reproduce the truncation on the old
code and pass on the fix.
…ite#9)

Batch-rerun chunks (>50 testIds) were dispatched concurrently via
Promise.all. The backend's producer/teardown closure dedup happens
per-request, not across requests, so two concurrent chunks sharing a
project's producer could each independently decide to trigger it,
double-running the producer or teardown.

Dispatch chunks sequentially in both the initial and deferred-retry
paths, closing the race at the source. Also dedupe the aggregated
accepted[] by testId and merge closure.byProject across chunks as a
defensive second layer, warning on stderr if a duplicate trigger is
detected.

Fixed a pre-existing test whose fixture relied on the old
double-counting behavior (same accepted entry returned from every
retry call).
…RROR (TestSprite#19)

A malformed API endpoint produced an opaque or misleading failure instead
of a clear config error:

  --endpoint-url "not a url"   -> `Error: Invalid URL` (exit 1, a raw
                                  `new URL()` throw with no guidance)
  --endpoint-url "localhost:3000" (missing scheme, parses as scheme
                                  "localhost:") and "ftp://x" (wrong scheme)
                               -> `fetch failed` / Service unavailable,
                                  emitted only after a full retry+backoff
                                  cycle — looks like a network outage, not a
                                  config typo.

Add `assertValidEndpointUrl` in client-factory.ts and run it in both the
real and dry-run paths of `makeHttpClient`, on the resolved endpoint (so it
covers --endpoint-url, TESTSPRITE_API_URL, and the credentials file). A
malformed value now throws a typed VALIDATION_ERROR (exit 5) with an
actionable message.

Crucially, and unlike the `--target-url` SSRF guard, this does NOT reject
localhost or private hosts — the API endpoint legitimately points at a
self-hosted, local-dev, or mock backend. Only syntactically invalid values
(unparseable, or a non-http(s) scheme) are rejected, so existing
self-hosted/CI configs and the test suite's localhost mock backend are
unaffected.

Adds unit coverage for assertValidEndpointUrl and the two makeHttpClient
paths, plus subprocess regressions.
runFailureGet now resolves and validates --out via resolveBundleDir and assertOutDirParentExists before calling GET /tests/{id}/failure, matching runArtifactGet and runCodeGet fast-fail behavior.

Adds regression tests asserting zero fetch calls on empty --out and missing parent dir paths.
…on (TestSprite#21)

A profile name (`--profile` / `TESTSPRITE_PROFILE`) is written verbatim as
an INI section header (`[name]`) in `~/.testsprite/credentials`, but was
never validated. A name containing the characters that break that grammar
silently corrupted the file:

  --profile "prod]"   -> serialises to `[prod]]`, which the section regex
                         cannot match, so the api_key/api_url lines that
                         follow are DROPPED on read. `setup` reports success
                         while the credential never persists.
  --profile $'a\nb'   -> the newline splits the header across two lines.
  --profile "  x  "   -> does not round-trip (the parser trims section
                         names, so it reads back as `x`).

Add `assertValidProfileName` in credentials.ts (a conservative allowlist:
letters, digits, dot, underscore, hyphen — covering `default`, `prod`,
`ci-staging`, `team.qa`) and call it from every profile-keyed entry point
(`readProfile`, `writeProfile`, `deleteProfile`). A malformed name now
throws a typed VALIDATION_ERROR (exit 5) before any file write, instead of
silently corrupting or failing to persist credentials.

Adds unit coverage for the guard and the three entry points, plus a
subprocess regression.

Co-authored-by: Zeshi Du <duke.zeshi@gmail.com>
…t json (TestSprite#22)

When a subcommand fired a parse error (unknown command, missing required
argument, invalid option), Commander's outputError callback wrote plain
text to stderr immediately and the catch block exited 5 with no further
output. A machine consumer that always parses stderr as JSON received an
unexpected plain-text string and crashed its JSON.parse.

Root cause: configureOutput was only applied to the root program, not to
subcommands. Each subcommand retained the default outputError that calls
write(str) directly. applyExitOverrideDeep now also propagates
configureOutput to every leaf so the message is buffered instead of
written.

In the CommanderError catch block, a resolved output mode is used to
either write a VALIDATION_ERROR JSON envelope or the buffered plain-text
message. An argv scan fallback handles the edge case where --output json
appears after the bad argument and was not yet parsed when the error fired.

The renderCommanderError helper is extracted to src/lib/render-error.ts
(alongside the existing rephraseUnknownOption helper) so it is unit-testable
without a subprocess. Eight unit tests cover JSON/text output, null fallback,
message trimming, and rephrased global-flag embedding. Four subprocess
regression tests in the [fix-5] block cover missing-arg, unknown subcommand,
argv-fallback, and text-mode no-regression paths.

Co-authored-by: zeshi-du <zeshi@testsprite.com>
@coderabbitai

coderabbitai Bot commented Jun 30, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 2153754e-f02f-4df4-9d92-bdc19ef58a08

📥 Commits

Reviewing files that changed from the base of the PR and between 15e95de and 75448b3.

📒 Files selected for processing (4)
  • src/commands/project.test.ts
  • src/commands/project.ts
  • src/lib/pagination.test.ts
  • src/lib/pagination.ts

📝 Walkthrough

Walkthrough

Adds two client-side safety mechanisms: paginate() in src/lib/pagination.ts now enforces a MAX_AUTO_PAGES (1000) cap and detects repeated nextToken cursors, throwing structured ApiError instances on violation. The --password-file option in src/commands/project.ts is guarded by readPasswordFileGuarded, which validates the path, enforces MAX_PASSWORD_FILE_BYTES (64 KiB), and returns structured errors. Tests cover all new failure paths.

Changes

Pagination Safety Guards

Layer / File(s) Summary
paginate() safety state and error helper
src/lib/pagination.ts
Imports ApiError, exports MAX_AUTO_PAGES=1000, initializes pagesFetched and seenNextTokens in the loop, throws on cap exceeded or repeated cursor via new paginationSafetyError helper.
Rejection tests
src/lib/pagination.test.ts
Adds tests asserting paginate rejects with UNAVAILABLE/repeated_next_token on cursor cycles and UNAVAILABLE/max_pages_exceeded when MAX_AUTO_PAGES is hit, with call-count assertions.

Password-File Validation

Layer / File(s) Summary
readPasswordFileGuarded helper
src/commands/project.ts
Adds statSync/resolve imports, exports MAX_PASSWORD_FILE_BYTES (64 KiB), implements readPasswordFileGuarded with absolute-path resolution, regular-file check, size cap, and passwordFileError producing typed ApiError envelopes. Wires into runCreate and runUpdate.
Password-file tests
src/commands/project.test.ts
Adds tests for successful read/trim, directory-path VALIDATION_ERROR, and oversized-file PAYLOAD_TOO_LARGE, all asserting no network call on failure.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐇 Hop hop, no infinite loop today,
The cursor won't spin us away.
A 64 KiB cap on passwords tight,
And pages stop at one thousand, right.
This bunny guards the CLI with care! 🛡️

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 20.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the two main hardening changes: pagination limits and guarded password-file reads.
Linked Issues check ✅ Passed The PR implements the requested pagination loop protections and password-file validation limits for create and update, matching #58.
Out of Scope Changes check ✅ Passed The changes stay within the security-hardening scope, with only supporting tests and constants added.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

zeshi-du
zeshi-du previously approved these changes Jul 2, 2026

@zeshi-du zeshi-du left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed and verified — merging. Thanks for the contribution!

@zeshi-du

zeshi-du commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Thanks — the pagination safety cap + loop detection and the guarded password-file reader (regular-file check, 64 KB cap, clear ENOENT error) are all improvements we want. This PR now conflicts with today's merge wave: #54 restructured the password-resolution block in runUpdate, #40 landed fractional-flag validation in pagination.ts, and #14/#17 touched the import blocks. Please rebase on latest main — the substance of your change is pre-approved, so a clean rebase should merge quickly. Note #27/#106 touch the same lines; your guarded reader supersedes #27's try/catch, so don't be surprised if the region looks different again after they land.

@zeshi-du

zeshi-du commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Thanks for the patience here — an update on scope. The pagination-bounding half of this PR just landed via #48 (same design you proposed: page safety cap + repeated-cursor detection; that PR rebased first while this one has been conflicting since 2026-07-02). The password-file read guard (stat + regular-file check + size cap) is still very much wanted — it's the remaining open half of issue #58. If you rebase this PR down to just the password-file guard (drop the pagination.ts changes), it merges quickly. 🙏

@merlinsantiago982-cmd merlinsantiago982-cmd dismissed zeshi-du’s stale review July 9, 2026 00:59

The merge-base changed after approval.

@jangjos-128

Copy link
Copy Markdown
Contributor

Hi @merlinsantiago982-cmd — so sorry for the disruption here. 🙏 While we were publishing a new CLI release, a hiccup in our process unintentionally closed this PR. To be clear: your contribution wasn't rejected, and nothing on your end caused this.

Your work is completely safe — your branch and every commit are intact and untouched, and main is already back to normal on our side.

Because of how the closure happened, the cleanest path is a fresh PR from your existing branch (rather than reopening this one) — and opening it yourself keeps it under your name, with full credit for your work:

👉 Open a new PR from your branch

It should merge cleanly. So your work doesn't get lost, if we don't hear back in the next ~3 days we'll go ahead and open it for you — but opening it yourself keeps it under your name, and either way we're always happy to hand it back to you.

Thank you for contributing to TestSprite, and again, we're sorry for the hassle. We really appreciate this work and want to see it merged. 🙌

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unbounded Pagination and Arbitrary File Read via --password-file

8 participants