Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions .github/workflows/dotnet.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,36 @@ jobs:
CI_JOB_NAME: dotnet-ci
run: ./build/run-with-cancellation-relay.sh ci

# GH-4098. This job wedged once and was cancelled at its cap, and every piece of evidence went
# with it: a cancelled job's logs are not truncated, they are DISCARDED (BlobNotFound), leaving
# only per-step timestamps in the API. The supervisor already writes a partial ledger inside the
# runner's grace window when it is cancelled (build/SupervisedTests.cs registerCancellationCapture,
# build/StallCapture.cs) -- naming the tests that were still in flight -- but this workflow never
# uploaded it, so the one artifact that survives a discarded log was being thrown away with the
# runner. tests.yml has had this since GH-3787; dotnet.yml is the workflow that actually wedged.
#
# `always()` and not `failure()`: a cancelled job is neither success nor failure, and cancellation
# is precisely the case this exists for.
- name: Upload retry ledger
if: always()
uses: actions/upload-artifact@v7
with:
name: test-ledger-dotnet-ci
path: artifacts/test-ledger/
if-no-files-found: ignore
retention-days: 30

# Only reachable when the runner itself survived. If the OOM killer took a *test* process rather
# than the runner service, the kill is recorded here and nowhere else. Same `always()` reasoning
# as the upload above.
- name: Kernel OOM report
if: always()
run: |
echo "::group::dmesg (OOM / kill events)"
sudo dmesg | grep -iE "out of memory|oom-kill|killed process" || echo "no OOM events in dmesg"
echo "::endgroup::"
free -m

# The CI target now also runs MessageRoutingTests, which boots a real
# RabbitMQ broker (started inline by the build target via docker compose).
# Tear it down on exit so the runner is clean for downstream steps and
Expand Down
6 changes: 5 additions & 1 deletion .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -138,8 +138,12 @@ jobs:

# Only reachable when the runner itself survived. If the OOM killer took a *test* process
# rather than the runner service, the kill is recorded here and nowhere else.
#
# GH-4098: `always()`, not `failure()`. A job cancelled at the 20-minute cap is neither success
# nor failure, so `failure()` skipped this step on exactly the wedges it was written for --
# #4083's CIMarten cancellations among them.
- name: Kernel OOM report
if: failure()
if: always()
run: |
echo "::group::dmesg (OOM / kill events)"
sudo dmesg | grep -iE "out of memory|oom-kill|killed process" || echo "no OOM events in dmesg"
Expand Down
Loading