You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Network-dependent tests (NuGetAssemblyResolver, MemexTemplateGenerator) present as phantom CI flakes; wall-clock deadlines misreport host/runner stalls #679
Found by a flake sweep across 60 of 62 test projects.
Two families of tests hard-fail when nuget.org is unreachable
MeshWeaver.Graph.Test.NuGetAssemblyResolverTest (3 tests) — FatalProtocolException: Unable to load the service index for https://api.nuget.org/v3/index.json
MemexTemplate.MemexTemplateGeneratorTest (7 tests) — Expected value to be 0 because generator failed → NU1900 … nuget.org
During the sweep these produced 10 failures that were purely environmental — a window where nuget.org was unreachable from this machine. On a clean network they pass.
Why file it
These will present as a phantom flake in CI whenever nuget.org has a bad minute: a red build, a failure message that points at NuGet internals rather than at "the network was down", and an engineer spending time attributing it. That is precisely the pattern this campaign has been unwinding — five "flakes" that were real defects, and the cost of the noise was that nobody trusted the signal.
Unlike the real defects, the right answer here is probably not a fix but a deliberate decision:
Skip loudly when the feed is unreachable — a skipped test with "nuget.org unreachable" is honest; a red one is misleading.
Pin to a local/offline feed so the tests exercise the resolver logic without live network.
Accept and document — leave them network-dependent, but make the failure message say so, so triage takes seconds rather than minutes.
Whichever is chosen, the goal is that a network blip is never indistinguishable from a code defect.
Related hazard found in the same sweep
MonolithMeshTestBase's watchdog and several tests' polling deadlines (e.g. CommentDoneReactivityTest.WaitForReadOnlyMarkdown, 30 s wall-clock with 1 s polls) measure wall-clock. During the sweep the host machine slept (233 sleep events in pmset -g log), producing impossible durations — 5523 s and 7288 s against a 720 s cap — and a batch of spurious WATCHDOG-HARD failures. Every one passed on re-run under caffeinate.
On CI the equivalent is a runner stall: any suspension converts directly into a red build with a timeout message that describes the symptom and not the cause. Worth considering whether these deadlines should measure monotonic elapsed CPU/progress rather than wall-clock, or at least report suspected suspension when a deadline is exceeded by an implausible margin.
Activity
Merge pull request #885 from Systemorph/fix/679-network-test-skips