Repository navigation
tests: the suite is not hermetic — network, real home config, wall-clock sleeps, unreset module caches (T-01, T-06..T-09) #235
Description
From the 2026-09-01 codebase review (plans/codebase-review-2026-09-01.md), findings T-01 and T-06 – T-09.
The suite is strong — 3199 passing, 85% coverage, mutation evidence behind the CI guards — but not hermetic:
- one test reaches the npm registry;
- one reads the developer's real home configuration;
- 65 tests wait on real wall-clock sleeps;
- no fixture resets the module-level caches, so cross-test pollution is possible and ordering-dependent failures are hard to attribute.
This is not theoretical here. Three separate CI incidents in the last week came from exactly this class: #226 (a 0.2 s wall-clock assertion failing by 2 ms), the #200 SSE hang (order-dependent, and the local repro needed the module rather than the full suite), and the standing /tmp/package.json quirk that makes ~107 npm-identity tests fail on any host that happens to have that file — documented in tests/conftest.py, but documentation is not isolation.
Related and already open: #226 (the timing assertion). That one should probably be fixed as part of this rather than separately, since the fix is the same idea — assert the property, not the wall clock.
Suggested order, cheapest first: reset module caches in an autouse fixture; replace real sleeps with virtual/eventual assertions; point home-config reads at a tmp_path; mark or stub the network test. Effort M in total, but each item is independently landable.
Activity
ViperJuice commentedon Sep 15, 2026
Partial progress from PKGID (#246):
tests/conftest.pynow has an autouse_no_live_npm_registryguard. Any test whose package-identity lookup opens registry.npmjs.org fails at teardown and names the URL. There is also afake_npm_registryfixture. Three tests had silently started calling the real registry once registration began resolving packages; they now use the fixture. This covers the package-identity path only (pmcp.manifest.package_identity), not other network use such as the npm resolver or version checker. Wall-clock sleeps, real-home reads and module-cache resets are untouched, so this stays open.docs(plan): reconcile panel CR on slice A — guard re-spec, sibling cw…
chore(plan): rev 2 of the slice C1 timing plan after panel review (see …
docs(plan): state the temp-layout assumption the ancestor guard depen…
docs(plan): reconcile panel CR on slice B — falsifiable closed-loop A…
chore(plan): module-state resets between tests — slice B of #235 (#265)
chore(plan): deterministic test timing, slice C1 of #235 (see #235, see
chore(plan): test cwd/filesystem isolation — slice A of #235 (see #261 …
fix(test): end the session with pytest.exit when the temp-root chain …
test: isolate each test's working directory and guard the temp-root c…
test: reset process-global module state between tests (see #235) (#267)
test: make the two stalled-handler assertions capable of failing
test: remove the two wall-clock bounds this slice's own helper tests …
docs(changelog): correct the count of removed wall-clock assertions
test: deterministic timing — slice C1 of #235, fixes the #226 flake (#…
6 remaining items
docs(tests): refresh two module docstrings the constants replaced
test: wait for the property instead of a fixed sleep (slice C2a of #235…
test: convert the B-ack retry C2b missed, and correct an overstated s…
docs: correct the _wait_until rationale and complete the leftover list
test: fold the suite's private pollers onto the shared helper (slice …
docs(plan): fix four panel nits, incl. a message that reports stale s…
docs(plan): wall-clock-window conversion — slice C2c of #235, the las…
test: replace the wall-clock windows with property proofs (slice C2c of
docs(plan): C3 revision 2 — a dead driver must fail the awaited call,…
docs(plan): inject a request clock so the idle-timeout tests stop wai…
test: pin the remaining request-clock roles the review board found un…
ViperJuice commentedon Sep 26, 2026
Slice C3 (#292) review board: 0 blocking. Two deferred, pre-existing items are tracked in #293: wall-time consumers of the request clock outside
ClientManager, and suites that wait out pytest-timeout when the idle timeout is broken.test: drive the idle-timeout tests with an injected request clock (sl…
ViperJuice commentedon Sep 26, 2026
Every slice is merged, so I'm closing this. Each of the four items in the issue maps to merged work:
HOMEredirect + cwd isolation, temp-root guard)_no_live_npm_registryguard +fake_npm_registry; see the comment above)Verified at main 9ca081e on CI (3.10/3.11/3.12 green on the PR head that was merged). Locally, the full suite gives 4071 passed, 3 skipped, 25 deselected, with the idle-timeout tests at ≤ 10 ms each (40/40 flake loop).
Two pre-existing items that C3's review deferred as out of scope are tracked in #293.