Skip to content

fix(relay): shorten receiver reconnect gaps and count role_busy refusals - #168

Merged
abruption merged 2 commits into
mainfrom
fix/163-receiver-reconnect
Sep 23, 2026
Merged

abruption merged 2 commits into
mainfrom
fix/163-receiver-reconnect

Conversation

@abruption

@abruption abruption commented Sep 23, 2026 •

Copy link
Copy Markdown
Owner

Summary

Refs #163, #161. Builds on #162 (merged): candidate (a) relies on the closeSource diagnostics introduced there.

These changes reduce reconnect gaps that amplify a stalled Relay path. They do not repair the underlying KR↔Cloudflare transport stall, and they are not evidence that #161 is resolved.

  • (a) Idle-expiry backoff: when the Relay closes a waiting room normally (1000, received, stage relay_attach) after it stayed open for at least one second, the receiver reconnects after 0.5 s instead of escalating toward 5 s. Rooms closed within one second, locally sent or abnormal closes, and other failures keep the existing exponential backoff (hot-loop guard).
  • (b) Keepalive: Relay legs use ping_interval=10, ping_timeout=10 (previously the websockets default 20/20 on endpoints), matching the Relay. A stalled leg is detected in about 20 s instead of about 40 s.
  • Metrics: additive receiverRoleBusy / clientRoleBusy counters and an opt-in role_busy diagnostic event (role only; no identity or room name). These measure how often a fresh leg is refused while the Relay still holds an older one, to decide whether the deferred candidate (c2) is warranted.
  • Docs: all four paired-device editions and the i18n hash.

Offline evidence (loopback only)

Isolated harness from #163 (127.0.0.1, temporary state, static fake accounts, pinned source hashes); predictions were fixed before each run. Builds were local experiment branches on top of 5e626e9. 10 repetitions per case:

Experiment base a b a+b
Reconnect gaps after idle expiry (1.5 s rooms) 0.53→1.0→2.0→4.0→5.0→5.0 s 0.52–0.55 s each as base 0.52–0.55 s each
Four idle expiries, then an 8 s admission timeout; client arrives meanwhile 0/10 succeed (fails at 10.0 s) 10/10 succeed (8.53–8.57 s) 0/10 10/10 (8.53–8.57 s)
Stalled Relay→receiver leg: receiver reconnect / client arriving at +24 s 42.5 s / 0/10 42.5 s / 0/10 22.5 s / 10/10 22.5 s / 10/10

The existing #163 scenarios (S0–S4) passed on a, b and a+b. The earlier CPU-load acceptance runs of the harness itself were 260/260.

Candidate (c) — reviewed, deferred

  • c1: ping the waiting receiver before pairing. Not adopted.
    • It did not turn a sustained stall into success: the failure moved from peer TLS at 8 s to relay_attach at 12 s.
    • During a 3 s transient stall it closed a leg that would have recovered.
    • It adds a round trip to every pairing.
  • c2: a newer receiver leg replaces an unpaired older one.
    • It fixed a zombie-leg case (success in 0.03 s vs failure).
    • It caused replacement churn with two live receivers sharing an identity: 17–18 replacements in 30 s and about 40% more connections.
    • It is deferred until the role_busy counters above show that the case matters in live runs. A ping-guarded variant would then need its own validation.

Verification

  • python -m unittest discover -s tests -t .: 513 tests, OK (11 skipped) on Python 3.14.7 and 3.13.7.
  • New tests/relay/test_receiver_reconnect.py:
    • idle-expiry backoff, the short-room guard, sent/abnormal closes, and reset after failures
    • 10/10 keepalive kwargs
    • per-role role_busy counting and events without identity
  • git diff --check clean.

Not included

No version bump, release, deployment or production change. RC3 inclusion and the live five-host validation (#164) remain separate decisions.

@abruption abruption added this to the v1.0.0-rc.3 milestone Sep 23, 2026
- Do not escalate the receiver backoff after a normal Relay close of a waiting room that stayed open for at least one second; shorter rooms and other failures keep the exponential backoff (hot-loop guard).
- Use a 10-second WebSocket ping interval and timeout on Relay legs so a stalled leg is detected in about 20 s instead of about 40 s.
- Count role_busy refusals per role (receiverRoleBusy, clientRoleBusy) and emit an opt-in role_busy diagnostic event.
- Document the behavior in all paired-device translations and refresh the i18n hash.

Refs #163, #161.
@abruption
abruption force-pushed the fix/163-receiver-reconnect branch from afedd3c to bc1a87c Compare September 23, 2026 07:55
@abruption
abruption changed the base branch from fix/161-relay-stability to main September 23, 2026 07:55
@abruption
abruption marked this pull request as ready for review September 23, 2026 07:59
@abruption
abruption merged commit f2633d0 into main Sep 23, 2026
47 checks passed
@abruption
abruption deleted the fix/163-receiver-reconnect branch September 27, 2026 01:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant