Core test suite is flaking under parallel shards — two unrelated tests failed docs-only PRs within an hour #834

Activity

1. rbuergi commented on Aug 6, 2026

   

   rbuergi

   on Aug 6, 2026

   ContributorAuthor

   More actions

   Root cause found for lead 2 — a genuine test defect, not chance

   CrossPartitionRealUserQueryTests.RunUnpinned issues:

   .Query<MeshNode>(new MeshQueryRequest
   {
       Query = "nodeType:Markdown",
       Limit  = 10,
       UserId = userId!,
   });

   …unscoped across every partition and capped at 10 rows — and UnpinnedStructuredQuery_AsSystem_EmitsInitial then asserts its own seeded row is in the result:

   change.Items.Should().Contain(n => n.Path == $"{nsA}/doc1", "System sees every partition's rows");

   The seeded doc1 is only guaranteed to appear while fewer than 10 other Markdown nodes exist. Against a shared PostgreSQL with sibling tests creating Markdown nodes concurrently, it gets crowded out of the page — which is precisely why the failure is intermittent, why it appears under parallel-shard load, and why it lands on PRs that changed nothing.

   The System sees every partition's rows message is also slightly misleading: as written the test proves "System sees ≥1 row from partition A within the first 10 global results", which is a weaker and load-dependent claim.

   Fix (test-side, no product change): make the assertion independent of unrelated rows — scope the query to the seeded namespaces (or a marker nodeType/id unique to the test run), and keep Limit only if the test is genuinely about paging. The sibling …AsAmbientRealUser_EmitsInitial and the anonymous/other-user cases share RunUnpinned and carry the same latent instability, so fixing the helper fixes all of them.

   Worth stating for the record: this one is not a first-frame-completeness race, so the GetQuery contract (gate the first emission until every sub-query has produced its Initial) is not implicated here. Lead 1 (#829's compile-timeout: a 10s assertion inside a [Fact(Timeout = 20000)] that performs a real Roslyn compile) is still open and is a budget mismatch rather than a race.

   🤖 Generated with Claude Code
2. rbuergi

   added a commit that references this issue on Aug 6, 2026

   Merge pull request #836 from Systemorph/fix/flaky-crosspartition-limit

   ...

   f0975d5
3. rbuergi commented on Aug 7, 2026

   

   rbuergi

   on Aug 7, 2026

   ContributorAuthor

   More actions

   Both documented defects are fixed on main:

   1. Paged-out assertion — 459b440 + 7a9b159/7f989fb41: RunUnpinned now takes a namespaceScope; the row-assert test (UnpinnedStructuredQuery_AsSystem_EmitsInitial) is scoped to its seeded namespace and folds the visibility check into a Match predicate, while the deliberately-global tests assert liveness only (converge, don't hang) and no longer claim a specific row can survive an unscoped limit:10 page.
   2. Halved compile budget — 459b440: FileSystem_Organizations_GetsHubConfiguration_FromCompiledAssembly now declares [Fact(Timeout = 60_000)] with explicit 30 s inner waits, replacing the 10 s default budget inside a 20 s Fact.

   Closing. The broader one-time-flake pattern remains tracked in #826.

   🤖 Generated with Claude Code
4. rbuergi

   closed this as completedon Aug 7, 2026