Repository navigation
test: stop unit tests waiting out real 5 s fallbacks (~285 s per run) and fix two flaky tests - #37362
Merged
Merged
Conversation
Contributor
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 9 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
…aky tests The ConPTY prelude mock delivered its DA1 query on a microtask, which reached only the physical owner's early onData listener (added in #36422); the startup watch subscribes after an await, never saw it, and every Windows-path spawn waited out CONPTY_STARTUP_DECISION_LIMIT_MS (5 s). Deliver it on a macrotask like real output. Also give two fixtures the readiness their production peers provide, and fix the counter-file race in structured-agent-shell-environment and the async sign-in marker race in claude-status-sign-in.
nwparker
force-pushed
the
test/prune-expensive-low-value-tests
branch
from
November 2, 2026 06:34
061cbdb to
2264809
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cuts about 285 s of test-execution time from every full unit run by removing real 5 s waits that protected nothing. Also fixes the two tests that recently failed intermittently on main. No test declarations are deleted and no workflow gating changes.
Where unit test time goes
Measured from the per-file phase artifacts of green main run 41392747240 (12 shards, 13,535 files):
Imports take about two thirds of worker time and grow with the number of files, not the number of declarations. So removing individual cheap tests saves little. Most of the execution time that can be recovered sits in tests that wait out a real production timeout. I ranked the 90 slowest files, ran them locally with
--reporter=jsonto get per-test durations, and traced every timer of 1.5 s or longer in the suspicious files.The largest finding was a test-harness bug.
createConptyPreludeSendersent the mock ConPTY DA1 query on a microtask, so only listeners subscribed in that same microtask received it. #36422 added an earlieronDatasubscriber (the physical owner's initial-output observer), and the startup watch subscribes after anawait. The watch never saw the prelude, so every Windows-path spawn in these suites waited the fullCONPTY_STARTUP_DECISION_LIMIT_MS(5 s), timed out toundecided, and passed anyway. Sending the prelude on a macrotask (captured before tests fake timers) matches real node-pty output, which reaches every listener subscribed during spawn. Because of the bug, these tests had been exercising the fallback path instead of the input-reader path they describe.Changes
src/main/daemon/pty-subprocess-test-harness.ts(affects 9pty-subprocess-*suites)structured-agent-session-outgrown-child.test.tsrunSetupanswers right away for an account that is already prepared. Unprepared accounts still wait on the test-controlled setup promiseprepareClaudeProfileForLaunchworktree-creation-chat-handoff.test.tsxregisterNativeChatPane, as the real pane doesHANDOFF_READY_TIMEOUT_MSfallback instead of covering the ready pathstructured-agent-shell-environment.test.ts(flaky)/profile/Bis served, a background run 4 truncates the counter (echo $n > file) while the test reads it, so it read0. The assertion was redundant anyway, because only run 3 and later print/profile/B, which the next line assertsclaude-status-sign-in.test.tsx(flaky)getByTextbecamefindByTextfor the sign-in marker. In the remote-server case the marker is now awaited before asserting there is no Sign in buttonopenAccountswaits for. The old remote-server negative check could also pass before the marker loaded (a negative case that never reached its rejection path)Total for the affected files: 293.4 s → 8.4 s (about 285 s, roughly 5.9% of unit test-execution time per full run). All 291 tests in the affected files pass locally; the counts are unchanged.
Deliberately kept despite cost
codex-interrupt-order,codex-goal-stop-settings,codex-stopped-send-order,codex-turn-end-settlementturn/completed, so Stop waits outCODEX_STOP_GRACE_MS(5 s). Faking timers across sqlite and lockfile timers is risky, and an injectable grace would be a test-only production seamremote-runtime-request-connectionast-grep-ignore)account-add-process,remote-node-runtime-host-download,server-claude-helper-*,wsl-distro-holder-script, codex app-server teardown/supervisioninput-order-projectionN=100,journal-*-scaling,acp-history-scaling)input-order-projection, about 14 s) is up to the ownersuse-ai.operation-recovery"cold saved recovery (false)"acquireAgentLaunchChoices, because the runtime fixture lacks host capabilities. A fix needs a runtime catalog fixture; left for a follow-upjournal-checkpoint-scalingChecks
pnpm test <affected files> --maxWorkers=2: 291 passed, 0 failed. The two flaky files were also run 3 times.pnpm exec oxlintandpnpm exec oxfmt --checkon the changed files: clean.pnpm check:flaky-test-pattern-ratchet: OK. No baseline references changed, and no files were deleted, so the shard timings need no edit.pnpm tc:node: the only error is the pre-existingagent-model-catalog-scaling.test.tsimport that is failing on main right now.The unit shard balancer still uses the October 25 timing baseline, so it will overweight the
pty-subprocess-*files until its next refresh.