Repository navigation
bun run test exits 0 when tests fail — batch.test.ts kills the runner with process.exit(0) #2435
Description
What happens
bun run test can exit 0 with a large number of failing tests, and print no final tally.
Cause
browse/test/batch.test.ts:47 schedules:
setTimeout(() => process.exit(0), 500);When that timer fires it terminates the whole bun test process with status 0 — the run is truncated and every failure recorded up to that point is discarded. Because the test script is bun test … && (bun run slop:diff …), the && tail then runs, which is itself a signal that the exit code was 0.
Environment
bun1.3.13- Ubuntu 24.04.4 LTS (kernel 6.8.0-136)
main@a325940
Repro
bun install
bun run test > out.log 2>&1; echo "EXIT=$?"
grep -c '^(fail)' out.log # non-zero
grep -E 'Ran [0-9]+ tests' out.log # no tally — the run did not finish
Observed here: EXIT=0 with 82 (fail) lines and no tally block. browse/test/batch.test.ts is the last file to emit output in every truncated run.
Why it's worth fixing
The suite currently cannot fail CI, so a regression landing anywhere before that timer fires goes green.
The most portable symptom, if it helps anyone else hitting this: a runner that prints no final tally did not finish — the missing summary is a more reliable tell than the exit code, because the exit code is the thing being falsified.
Note on scoping while investigating
Worth flagging because it cost a couple of runs to notice: bun test test/ is a substring filter rather than a directory scope, so it still pulls in browse/test/**, and --ignore 'browse/**' did not exclude them either. Two consecutive "isolated" runs reproduced an identical signature, which briefly read as evidence against the diagnosis — it was the same unscoped run twice. An explicit file list was the only reliable way to isolate the non-browse suite:
bun test $(ls test/*.test.ts | grep -vE 'skill-e2e-|skill-llm-eval|skill-routing-e2e|codex-e2e|gemini-e2e')
That form gives a truthful exit 1 and a real tally.
No fix attached
Happy to send a PR if you'd like a particular approach — either letting the test await shutdown rather than exiting, or moving the process.exit(0) behind an env guard so it only fires in the standalone context it was written for.
Activity
test: stop one test file exiting the shared runner with status 0
tranthanhnhatkhoa commentedon Aug 8, 2026
Still reproduces on the tagged release v1.60.2.0 (
960c3a8d, 2026-08-07) — five days after this was filed, and in the release whose changelog headline is free-suite health.browse/test/batch.test.ts:47still has thesetTimeout(() => process.exit(0), 500)named above.Environment: bun 1.3.14, Ubuntu 24.04.4 LTS (kernel 6.8.0-106),
main@960c3a8d.One data point that may help localize it: the truncation point varies between machines. This run got 4199 passes before dying; the original report saw 82 failures. So the timer is racing the suite rather than cutting at a fixed place, which fits a wall-clock
setTimeoutagainst variable test durations — and means the number of masked failures is machine- and load-dependent, not a stable "~6%" as #2421 suggests.Worth noting for anyone triaging: the 32 failures here are not caused by the exit-code bug and are not new — I A/B'd the affected files against
2beb636f(2026-07-14) and got an identical 59 pass / 30 fail, test for test. They appear to be pre-existing environment-sensitivity in the config/state-resolution suites (gstack-team-init,gstack-config,gstack-session-update,gstack-gbrain-detect), plausibly related to #2413. The exit-code bug is what stops anyone noticing them.v1.64.0.0 fix wave: full tracker audit — 90 fixes, 52 issues closed, …
garrytan commentedon Aug 17, 2026
Fixed on main in v1.64.0.0 — batch.test.ts process.exit(0) removed; test/no-suicide-exit.test.ts is the tripwire; bun run test routes through the strict sharded runner. Closing.