Repository navigation
The FutuRe exit=139 is one background-GC fault, three times — and the dump reads fine on macOS - #1004
Conversation
…p again The 2026-08-06 dump analysis concluded that MeshWeaver.FutuRe.Test's exit=139 is a CoreCLR background-GC fault — a swept object whose MethodTable pointer is exactly NULL — and explicitly not the collectible-ALC use-after-unload shape that #613 had been chasing. One sighting is a hypothesis; #613 was closed and reopened on exactly that ambiguity. Run 31309378803 on main @ 8a5c8bf reproduced it, and it resolves to the same frame, the same instruction and the same registers: WKS::gc_heap::background_sweep, mov ecx,[rax] with RAX=0, si_addr=0x0, TRAPNO=14/ERR=0x4/CR2=0x0. That tree contains both #967 and #996, so the teardown work neither caused nor cured it — and the trace log shows all 22 disposals clean, no quiesce leak, no dirty teardown, with the crash landing ~476ms into a FRESH fixture. It is not a teardown crash; the issue title has been misleading the search. Also records the route that produced it: the whole analysis runs on macOS in pure Python over the ELF core plus one build-id lookup against Microsoft's public symbol server. The doc previously opened with 'you cannot open a Linux dump on macOS', which is true only of the DAC-backed managed commands and sent the last investigation to a slow emulated container for a fact the core carries in plain sight. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR updates the native-crash debugging documentation to make macOS-first triage faster and to record evidence that the MeshWeaver.FutuRe.Test exit=139 crash is the same CoreCLR background-GC fault on two separate dates/runs.
Changes:
- Adds a macOS-first workflow to identify the faulting frame and dereferenced address without using Docker/DAC tooling.
- Reframes the container-based approach as needed only for managed/DAC commands.
- Documents a second independent reproduction (2026-08-09) to confirm the fault signature matches the earlier incident.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Adding the second 2026-08-09 dump — the one #613 was reopened on — makes it three sightings of the same fault, and it corrects something I got wrong an hour ago. I had written that the crash 'is not a teardown crash at all' because two of two landed inside a fresh fixture. dotnet-3050 lands 96ms into a Mesh.Dispose(). The faulting instruction is byte-identical in all three: RVA 0x5cb1e1, background_sweep+0xa61, mov ecx,[rax] with RAX=0, CR2=0, ERR=0x4. So the mutator phase varies while the fault does not — which is what a concurrent background sweep predicts, and the opposite of a teardown-ordering defect, which reproduces at one phase by construction. Reading the phase of a single sighting is how this issue got framed as a teardown race for weeks; the RVA is the thing to compare, and on 10.0.10 it is one number. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…iew) The section promises 'pure Python, no elfutils' and then names <BUILD_ID> as an input without saying where it comes from — which is exactly the step that sends a reader back to eu-readelf and defeats the point. It is the same section-header walk the rest of the recipe already does, so it is a few lines, not a dependency. Also indents the fence under item 4 so the numbered list does not break, and says out loud that the shipped libcoreclr.so has nine STT_FUNC symbols: that lookup returning nothing reads as 'the technique does not work' rather than 'this file is stripped, fetch the .debug'. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Test Results (shard 0)685 tests 685 ✅ 4m 56s ⏱️ Results for commit 608b0cb. ♻️ This comment has been updated with latest results. |
Test Results (shard 3) 11 files 11 suites 4m 58s ⏱️ Results for commit 608b0cb. ♻️ This comment has been updated with latest results. |
Test Results (shard 5)1 724 tests 1 532 ✅ 5m 50s ⏱️ Results for commit 608b0cb. ♻️ This comment has been updated with latest results. |
Test Results (shard 2)1 898 tests 1 896 ✅ 6m 20s ⏱️ Results for commit 608b0cb. ♻️ This comment has been updated with latest results. |
Test Results (shard 4)1 208 tests 1 198 ✅ 8m 0s ⏱️ Results for commit 608b0cb. ♻️ This comment has been updated with latest results. |
Test Results (shard 1)1 188 tests 1 188 ✅ 7m 15s ⏱️ Results for commit 608b0cb. ♻️ This comment has been updated with latest results. |
Test Results 65 files 65 suites 37m 22s ⏱️ Results for commit 608b0cb. ♻️ This comment has been updated with latest results. |
MeshWeaver.FutuRe.Test exit=139was root-caused on 2026-08-06 as a CoreCLR background-GC fault —WKS::gc_heap::background_sweep()reading a swept object whose MethodTable pointer is exactly NULL — and explicitly not the collectible-ALC use-after-unload shape that #613 spent weeks chasing. One sighting makes that a hypothesis. #613 was closed on 2026-08-09 05:48Z and reopened 70 minutes later precisely because nobody could tell whether the next crash was the same thing.It is. Both
exit=139s of 2026-08-09 — including the one the issue was reopened on — resolve to the same fault at instruction granularity. On runtime10.0.10the whole comparison collapses to one number, RVA0x5cb1e1:31083356138)31299547995)31309378803)background_sweep()0x5cb1e1(+0xa61)0x5cb1e1(+0xa61)mov ecx,[rax],rax == 08b 08=mov ecx,[rax],RAX=0x08b 08,RAX=0x0TRAPNO/ERR/CR20x4/0x00x4/0x00x4/0x0si_signo/si_code/si_addrSEGV_MAPERR) /0x00x0Mesh.Dispose()The mutator phase varies while the faulting instruction does not — and that is the finding. A background-GC sweep runs concurrently with whatever the mutator is doing, so an identical fault at three different phases is what this diagnosis predicts; a teardown-ordering defect is the opposite, reproducing at one phase by construction (which is how
NodeAlcUnloadTeardownOrderingTestcould pin the exit=134). Reading the phase of a single sighting is how this crash spent weeks framed as a teardown race — I made the same mistake mid-investigation off two of three dumps and the third corrected it, so the caveat is now written down next to the table.Two things this settles, both of which had been re-litigated
The teardown work neither caused nor cured it. The crashing tree contains #967 (node ALCs unload at the end of teardown) and #996 (pending Rx timers no longer root disposed hubs). The crashing process's own
_meshweaver-test-trace.logrecords all 22DISPOSE_DONElines asteardown clean — all pooled I/O joined, async dispose queue drained, zeroDISPOSE_QUIESCE_LEAK, zeroDISPOSE_DIRTY_TEARDOWN; the straggler log holds two first-chance records, both from a different project's process.The crash is not where the title says. Two of the three land inside a fresh test — in the 11:05Z case the last trace record is
INIT_MEMfor a newly constructed fixture whose predecessor had finished tearing down cleanly 470 ms earlier, under roughly one gen2 collection per second. "Teardown SIGSEGV in …" describes one sighting of three.I checked the trace's completeness before reading anything into those zeros, per the
FAULT-BUDGETrule the doc already states: all four suppression windows in that file belong to a later project's pid, so the crashing process's record is whole. That is what makes "zero quiesce leaks" evidence rather than an absence.The method is the other half of the value
The doc opened with "🚨 You cannot open a Linux dump on macOS", which is true only of the DAC-backed managed commands (
clrstack,verifyheap) — and it sent the last investigation into a slow emulated container for a fact the core carries in plain sight. The 185 MB dump download was abandoned at ~30 KB/s during the exit=134 work; that is the cost this framing imposes.The whole analysis above ran on a Mac in about ten minutes, in pure Python over the ELF core plus one build-id lookup against Microsoft's public symbol server:
NT_SIGINFOfor the kernel's verdict,NT_FILEfor the load base, a scan of thePT_LOADsegments for the faultingucontext(TRAPNO == 14,RIPinsidelibcoreclr) sinceNT_PRSTATUSrecordscreatedump's own signal handler, then RVA →.symtabof the separately-downloadedcoreclr.debug. The shippedlibcoreclr.sois stripped to nine exported symbols, so resolving against it fails and looks like the technique doesn't work — the debug file has ~31 800 and the hit is exact. That recipe is now the first section, with the container kept for what genuinely needs it.Scope
Documentation only — no code change, and deliberately so. The fault is runtime-internal bookkeeping; the doc's existing "do not 'fix' this by turning off concurrent GC" guidance stands. What MeshWeaver contributes is the workload that provokes it (per-
[Fact]mesh build + teardown with collectible NodeType assemblies), which is not something to suppress.Testing
MeshWeaver.Documentationbuilds clean under-c Release -warnaserror.DocumentationLinkIntegrityTest— 1/1 passed.🤖 Generated with Claude Code