[abruption]

Summary

Refs #161 and #163. Draft RC3 diagnostic/mitigation candidate, not a root-cause fix. Head 5e626e9 is for #164 preparation only. Keep #161 open.

• Preserve every failed setup attempt with allowlisted stage, reason, elapsed time, HTTP status and WebSocket close code/source. A second-attempt success is degraded, not a clean pass.
• Retry once only for pre-upgrade control/admission timeouts. WebSocket upgrade, attach, peer TLS, probe, auth and post-submit failures are not retried. Never replay an application message.
• Add opt-in metadata-only receiver and Relay lifecycle events. Distinguish client-first from receiver-first rooms, pairing, attach notification, closure and selected timeout reasons. Attach sent, WebSocket 101 and receiver waiting age do not prove delivery or connection health.
• Add scaled stale-room, idle expiry, delayed admission, close-direction and timeout-reason fixtures. Keep diagnostic output free of room names, identity, credentials and payload.

Evidence and limits

Prior RC2 public health and Relay stalls support an intermittent CF-to-origin transfer hypothesis, but the fault owner and causal link to each failed message remain unproven. The earlier soak stopped near minute 172 without completion; it is not a passing four-hour run. No production instrumentation or RC3 artifact has been deployed.

Local Python 3.14.7 full suite: 507 OK, 11 skips. Focused Relay diagnostics: 26 OK. Documentation/i18n and diff checks pass. CI for 5e626e9 passed all 24 checks, including release gate. An independent read-only review of prior head 4e99df2 found no substantive correctness or privacy regression and recommended clarifying peer_closed and stream_close.closeCode. Those two changes are in 5e626e9; the exact diff 4e99df2..5e626e9 was then reviewed read-only: no correctness or privacy regression and no merge-blocking defect. Full suites on Python 3.13.7 and 3.14.7 each ran 507 OK with 11 skips; focused diagnostics ran 30 times under CPU load without failure. Do not merge or publish solely because unit tests or CI pass.

Diagnostic interpretation cautions: peer_closed now requires an explicit Relay-initiated counterpart close; an independent remote 1001 is remote_going_away. stream_close.closeCode is the received code (1006 can indicate no close frame). Near-simultaneous remote closes can rarely make a peer_closed label ambiguous because the Relay flag races with an already-closing counterpart. The early-returning paired leg may lack its own stream_close event while the opposite leg records one. None is accepted as root-cause proof.

#164 still needs exact RC3 artifact review, separately approved deployment, real multi-host ACK, and an uninterrupted four-hour validation.

[abruption]

RC3 diagnostic review — follow-up to #161 / #163 / #164

Independent Claude read-only review (ISSUE161-RC3-REVIEW-20260923) and local code inspection found a release-gating diagnostic gap in this Draft candidate. This is not evidence that the proposed retry fixes the outage.

1. In app.py, diagnostics[kind] = ... overwrites the prior attempt. If the first Relay attempt attaches to a stale receiver and times out during peer handshake, a subsequent attempt can find no room and end as relay_attach / connection_failed / attempts:2; that final label would hide the first-stage failure. Preserve an allowlisted per-attempt history (stage, reason, elapsed, close code where available). A successful second attempt should be counted as degraded in RC3 validation, not a clean pass.
2. wire.py closes on attach failure and transport_errors.failure() reduces connection closures to connection_failed. Where safely available, preserve the WebSocket close code so client wait expiry can be distinguished from role/auth closures; do not log raw exception text or peer identifiers. Distinguish peer TLS handshake deadline from later probe response time by bounded elapsed/stage data.
3. Receiver warning output treats normal 60 s idle expiry like a generic connection failure and rate-limits it to one event/minute. It does not measure room-open/close, attach receipt, peer-TLS result or reconnect gap. Add metadata-only lifecycle evidence or make that a separate gated change under investigate(relay): bound idle receiver reconnect gaps and stale-room attaches #163; do not infer a stable receiver from client-only route diagnostics.
4. Add deterministic fixtures for idle expiry, delayed admission (>10 s), a paired but unresponsive receiver leg, and a subsequent attempt while its only room is absent. Current timeout/retry/native-effect tests do not cover that chain. Check the RC3 harness outer invoke deadline against worst-case setup plus one retry; the old 35 s cap may erase the diagnostic JSON.

The observed relay_attach / connection_failed / attempts:2 is consistent with multiple mechanisms; it does not prove a stale receiver. Keep #161 open. Please do not merge/publish this PR as a root-cause fix solely because CI passes or a retry succeeds. Any production instrumentation or direct-origin A/B still needs separately scoped approval.