Skip to content

test: run suites without --isolate and remove three flaky tests - #8336

Merged
saddlepaddle merged 2 commits into
mainfrom
test/drop-isolate-and-flaky
Oct 8, 2026
Merged

saddlepaddle merged 2 commits into
mainfrom
test/drop-isolate-and-flaky

Conversation

@saddlepaddle

@saddlepaddle saddlepaddle commented Oct 8, 2026 •

Copy link
Copy Markdown
Collaborator

Drop --isolate

The cross-file leaks that --isolate hid are fixed in #8314, #8315, #8316 and #8319. This removes the flag from all 26 test scripts, so each package's suite runs in one process. packages/host-service keeps --parallel. The one line in AGENTS.md that named the flag is corrected.

The desktop suite takes about 38 s without the flag and about 73 s with it.

One more leak, found by the sweep

apps/relay/src/host-tunnel.test.ts replaced partyserver with { Server: class {} }. In some file orders index.test.ts then failed to load with Export named 'getServerByName' not found. The test passes against the real module, so the stub is removed.

Flaky tests removed

Test Seen failing
workspace-fs: "FsWatcherManager root deletion recovery > resumes the same subscription after the root is deleted and recreated" on main and on PRs, 2026-10-07/08
i18n: "locale changes preserve state and focus while updating context formatters", with its fixture on #8270 and in the canary experiment
agent-setup: "Pi completion notifications" (3 cases) once in 13 runs, on a heavily loaded machine; it polls up to 1 s of real time for a detached script

The second workspace-fs recovery test ("does not hang teardown after root deletion") stays. The Pi deletion removes all coverage of Pi completion notifications.

Verification

bun 1.4.2, bun test --randomize --seed N without --isolate: every one of the 26 packages passes on seeds 31, 32 and 33; apps/relay also on 34-36 after its fix. Biome passes on apps and packages; typecheck passes in apps/relay and packages/workspace-fs.


Summary by cubic

Drops --isolate from all 26 test scripts now that the cross-file leaks it hid are fixed, so each package's suite runs in one process. The desktop suite drops from about 73 s to about 38 s; packages/host-service keeps --parallel.

Removes the partyserver stub in apps/relay/src/host-tunnel.test.ts, which lacked getServerByName and broke index.test.ts in some file orders.

Also removes three flaky tests: the workspace-fs root-deletion recovery test, the i18n locale-state fixture, and the agent-setup Pi completion notification tests (also removing the only coverage of that feature).

  • Renames the Postgres watch test to watch-ownership.postgres.integration.ts so it no longer runs in the shared suite; it sets DATABASE_URL and imports the db client, which in one process could write fixtures to whichever database the first file connected to. It now runs only when named, in its own process, like the other opt-in suites.

Written for commit e147dbf. Summary will update on new commits.

View guided diff Turn on auto-fix

Summary by CodeRabbit

  • Chores
    • Test suites now run in a single process rather than using isolated runs.
    • Removed several tests covering completion notifications, locale-provider behavior, and recovery of file-watch subscriptions after a watched folder is recreated.

The cross-file leaks that --isolate hid are fixed (#8314, #8315, #8316,
#8319), so each package's suite now runs in one process. The desktop suite
takes about 38 s without the flag and about 73 s with it.

apps/relay still had one leak: host-tunnel.test.ts replaced partyserver
with a stub that has no getServerByName, which broke index.test.ts in some
file orders. The test passes against the real module, so the stub is gone.

Removed as flaky:
- workspace-fs: FsWatcherManager resumes a subscription after the root is
  deleted and recreated
- i18n: locale changes preserve state and focus (provider-state)
- agent-setup: Pi completion notifications, which poll up to 1 s for a
  detached script
@coderabbitai

coderabbitai Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 06bf8dd2-04f3-4397-b6c7-de8cb1754635
📥 Commits

Reviewing files that changed from the base of the PR and between c305409 and e147dbf.

📒 Files selected for processing (33)
  • AGENTS.md
  • apps/api/package.json
  • apps/desktop/package.json
  • apps/discord-triage/package.json
  • apps/marketing/package.json
  • apps/mobile/package.json
  • apps/relay/package.json
  • apps/relay/src/host-tunnel.test.ts
  • apps/web/package.json
  • packages/agent-setup/package.json
  • packages/agent-setup/src/agent-wrappers-pi.test.ts
  • packages/auth/package.json
  • packages/chat-runtime/package.json
  • packages/chat-ui/package.json
  • packages/chat/package.json
  • packages/cli-framework/package.json
  • packages/cli/package.json
  • packages/cloud-client/package.json
  • packages/email/package.json
  • packages/host-client/package.json
  • packages/i18n/package.json
  • packages/i18n/test/fixtures/provider-state.tsx
  • packages/i18n/test/provider-state.test.ts
  • packages/panes/package.json
  • packages/port-scanner/package.json
  • packages/pty-daemon/package.json
  • packages/sandbox/package.json
  • packages/shared/package.json
  • packages/trpc/package.json
  • packages/trpc/src/router/page/watch-ownership.postgres.integration.ts
  • packages/workspace-client/package.json
  • packages/workspace-fs/package.json
  • packages/workspace-fs/src/watch-root-recovery.test.ts
 ______________________________________________________
< Linters are opinions; I'm the peer-reviewed journal. >
 ------------------------------------------------------
  \
   \   \
        \ /\
        ( )
      .( o ).
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@expo

expo Bot commented Oct 8, 2026

Copy link
Copy Markdown

No installed development build can load this pull request as an update: its native fingerprint is 52085cf7de79 and no development build has it. Add the mobile-build label to build one.

@github-actions

github-actions Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

🧹 Preview Cleanup Complete

The following preview resources have been cleaned up:

  • ✅ Neon database branch

Thank you for your contribution! 🎉

@greptile-apps

greptile-apps Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 4/5

[Low impact] Test configuration and flaky test removal.

Fix the PostgreSQL watch test’s shared database connection before merging.

Findings

  1. P1 Watch tests use wrong database ▶
Fix with agent prompt
### Issue 1
packages/trpc/package.json:143
If `PAGE_WATCH_TEST_DATABASE_URL` is set and a file such as `trpc.test.ts` runs first, removing `--isolate` makes the watch test reuse the database client created by that earlier file. `watch-ownership.postgres.test.ts` changes `DATABASE_URL`, but importing the cached `dbWs` does not change its connection. The test queries the earlier database, which defaults to localhost on a clean checkout. It can then fail or write its fixtures to the wrong database.

Run this test in a separate process, or give it a client created from `PAGE_WATCH_TEST_DATABASE_URL`.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

This PR removes --isolate from 26 test scripts, removes the relay’s partyserver stub, and deletes three groups of flaky tests.

  • The optional PostgreSQL watch test still depends on a fresh database module and can use the wrong connection in the shared process.
  • saddlepaddle explicitly acknowledged removing the flaky coverage, including all Pi completion-notification coverage.

Reviews (1) · Last reviewed commit: "test: run suites without --isolate and r..." · Reviewed by Greptile

"clean": "git clean -xdf .cache .turbo dist node_modules",
"typecheck": "tsc --noEmit --emitDeclarationOnly false",
"test": "bun test --isolate"
"test": "bun test"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Watch tests use wrong database

If PAGE_WATCH_TEST_DATABASE_URL is set and a file such as trpc.test.ts runs first, removing --isolate makes the watch test reuse the database client created by that earlier file. watch-ownership.postgres.test.ts changes DATABASE_URL, but importing the cached dbWs does not change its connection. The test queries the earlier database, which defaults to localhost on a clean checkout. It can then fail or write its fixtures to the wrong database.

Run this test in a separate process, or give it a client created from PAGE_WATCH_TEST_DATABASE_URL.

Prompt To Fix With AI
This is a comment left during a code review.
Path: packages/trpc/package.json
Line: 143

Comment:
**Watch tests use wrong database**

If `PAGE_WATCH_TEST_DATABASE_URL` is set and a file such as `trpc.test.ts` runs first, removing `--isolate` makes the watch test reuse the database client created by that earlier file. `watch-ownership.postgres.test.ts` changes `DATABASE_URL`, but importing the cached `dbWs` does not change its connection. The test queries the earlier database, which defaults to localhost on a clean checkout. It can then fail or write its fixtures to the wrong database.

Run this test in a separate process, or give it a client created from `PAGE_WATCH_TEST_DATABASE_URL`.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

It sets DATABASE_URL and then imports the db client. In one process the
client may already exist, so the test would write its fixtures to the
database the first file connected to. As an .integration.ts file it runs
only when named, in its own process, like the other opt-in suites.
@saddlepaddle
saddlepaddle merged commit 7196886 into main Oct 8, 2026
30 of 32 checks passed
@saddlepaddle
saddlepaddle deleted the test/drop-isolate-and-flaky branch October 8, 2026 22:02
pgilmore added a commit to patiently/superset that referenced this pull request Oct 11, 2026
Merges upstream main into the watcher build. Upstream now runs each test
suite in one process and bans module mocks in the desktop suite (ParadevAI#8319,
ParadevAI#8336), so the hover-card test drives the real hooks against a fake host
instead of mocking them. Tests run on the bun canary upstream's CI uses,
because bun 1.4.2 can hang in a synchronous spawn.

Claude-Session: https://claude.ai/code/session_01CZBCrRCZ2rEBHv4DH3Wh3k

This branch was previously deployed

1 inactive deployment
preview — e147dbfb Deployed Oct 8, 2026 by saddlepaddle via Deploy Web #11552
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant