Skip to content

fix(host-service): re-register the host when the relay refuses it - #8907

Open
RohithReddy20 wants to merge 4 commits into
ParadevAI:mainfrom
RohithReddy20:fix/host-reregister-4403
Open

RohithReddy20 wants to merge 4 commits into
ParadevAI:mainfrom
RohithReddy20:fix/host-reregister-4403

Conversation

@RohithReddy20

@RohithReddy20 RohithReddy20 commented Oct 26, 2026 •

Copy link
Copy Markdown

What & why

Fixes #8445.

connectRelay called host.ensure once per process, and a 4403 forbidden close from the relay was retried like any other drop. A host whose access row disappeared kept reconnecting every 5 s and never recovered until restart.

  • On a 4403 close the tunnel client stops the retry cycle, runs host.ensure again (same 30 s to 5 min backoff as boot), then reconnects. A refusal is recorded until the next host.ensure result replaces it.
  • Repeated refusals back off 30 s, 60 s, up to 5 min. The counter resets on the first control message, not on open, since the relay opens before it refuses.
  • Each host.ensure attempt has a 30 s deadline so a stalled request can't block reconnects.

How I tested it

  • Ran the host-service against a local stub relay and API, then dropped the host's row. On main it got 4403 on every reconnect for a minute and never called host.ensure again. With this branch it re-registered on the first 4403 and reconnected.
  • Against a stub relay that refuses three times, this branch waited 0 s, 30 s, then 60 s between attempts before connecting.
  • bun test src/tunnel/ in packages/host-service: new tests for the 4403 recovery against a local relay, the backoff sequence, and the ensure deadline. Each fails on main.
  • Full packages/host-service suite: 2644 pass, 0 fail.
  • lint, typecheck clean.

Checklist

  • PR title follows conventional commits (type(scope): subject)
  • bun run lint and bun run typecheck pass (CI fails on lint warnings too)
  • "Allow edits from maintainers" is checked on fork PRs

Summary by CodeRabbit

  • Bug Fixes
    • Improved recovery when a relay rejects a connection, with retries that back off over time.
    • Registration attempts now time out and can be cancelled during shutdown, helping prevent connections from getting stuck.
    • Relay resolution and tunnel setup no longer repeat as part of registration retries.

The relay checks the host's access row on every control connect. When
that row disappears while the host-service is running (host deleted,
member removed and re-invited), every reconnect is refused with 4403.
`host.ensure` only ran once per process, and the tunnel client treated
4403 like any transport drop, so partysocket kept retrying every ~5s
for days with no way to recover short of a manual restart. Meanwhile
`superset status` still reported the host as registered.

The tunnel client now stops partysocket's retry cycle on 4403 and hands
the close reason to a new `onForbidden` hook, with a single in-flight
guard so repeated closes do not stack. The watchdog defers while that
recovery runs. `connectRelay` wires the hook to the same registration
loop the boot path uses: it records the failure so health.check tells
the truth, calls `host.ensure` again with the existing backoff, and the
client reconnects once registration lands. The machine id and routing
key do not change, so the existing tunnel is reused.

A regression test runs a local relay that closes the first control
connect with 4403 and asserts the hook runs once before the second
connect arrives.

Fixes ParadevAI#8445
The relay maps an access check it cannot complete (API unreachable from
the relay) to the same 4403 as a missing registration. With immediate
re-registration, a host that can still reach the API would re-run
host.ensure and reconnect in a zero-delay loop. Consecutive 4403 closes
without an intervening open now wait 30s, 60s, ... up to 5 min before
the next connect; the first refusal after an open still recovers at
once, so a lost registration row is repaired immediately.
The relay completes the WebSocket upgrade before it closes a refused
control connect with 4403, so `open` fired on every refusal and reset
the refusal counter. Repeated refusals then reconnected, and re-ran
host.ensure, with no delay. Reset the counter on the first inbound
message instead: a refused socket never receives one.

The re-registration hook pauses reconnects and the stuck-control
watchdog until it settles, but host.ensure had no deadline. The API
client sets no timeout, and the only signal fired on shutdown, so one
stalled request kept the tunnel down. Give each attempt a 30 s deadline
so a stalled request is cancelled and retried on the existing backoff.

Tests: the backoff test now opens each refused socket before closing
it, and only a received message resets the delay. A new connect test
checks that a stalled host.ensure is cancelled at its deadline.
@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@coderabbitai

coderabbitai Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: f162c223-260f-46ce-af3e-66f22250af9c
Commits

Reviewing files that changed from the base of the PR and between 0f79020 and c377332.

Files selected for processing: 1
  • packages/host-service/src/tunnel/tunnel-client.ts
Files skipped from review as they are similar to previous changes: 1
  • packages/host-service/src/tunnel/tunnel-client.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 8 remain after this review.

Walkthrough

Host registration now retries independently of relay setup. When the relay closes a control connection with code 4403, the tunnel client can re-register the host before reconnecting, with delays for repeated refusals.

Changes: Host recovery

Layer / File(s) Summary
Registration and relay setup
packages/host-service/src/tunnel/connect.ts, packages/host-service/src/tunnel/connect.test.ts
registerHost retries registration with a 30-second per-attempt timeout and supports caller cancellation. connectRelay resolves the relay and creates the tunnel after registration succeeds. Its forbidden-close callback attempts registration again. The test covers cancellation during a pending registration attempt.
Forbidden-close recovery
packages/host-service/src/tunnel/tunnel-client.ts, packages/host-service/src/tunnel/tunnel-client.test.ts, packages/host-service/src/tunnel/tunnel-client.integration.test.ts
When configured with onForbidden, the client handles code 4403 closes through the callback before reconnecting. Repeated forbidden closes use delays that start immediately and then grow from 30 seconds to a five-minute cap. Incoming control messages reset the retry count. Unit and real-socket tests cover reconnection and callback behavior.

Priority: Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Bug fix · Severity of issue fixed: Medium

Sequence Diagram(s)

sequenceDiagram
  participant Relay
  participant TunnelClient
  participant registerHost
  participant HostAPI
  Relay->>TunnelClient: Close control connection with code 4403
  TunnelClient->>registerHost: Invoke onForbidden with close reason
  registerHost->>HostAPI: Call host.ensure.mutate
  HostAPI-->>registerHost: Return registration result
  registerHost-->>TunnelClient: Complete recovery callback
  TunnelClient->>Relay: Reconnect control connection
Loading

Possibly related PRs

  • ParadevAI/Paradev#8900: Changes the same tunnel recovery flow with forbidden-close handling, re-registration, and refusal backoff.
Merge Risk: Minimal · up to c3773

The reviewed recovery path addresses relay refusals without an established merge-blocking issue.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 22.22% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 9 functions across 5 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Title check Passed The title "fix(host-service): re-register the host when the relay refuses it" follows conventional commit format and names the primary change. The changeset adds a registerHost function and an onForbi…
Description check Passed The description follows the template's main sections. It covers What & why with a linked issue (fixes #8445), a How I tested it section with concrete steps and results, and a checklist. The checklist …
Linked Issues check Passed The changes satisfy the coding requirements in [#8445]. TunnelClient intercepts relay close code 4403, invokes recovery, and delays reconnects with the required 0, 30, 60-second sequence up to five …
Out of Scope Changes check Passed The reviewed changes stay within [#8445]. The production changes implement host re-registration after relay refusal, retry backoff, timeout handling, and the related tunnel behavior. The added tests v…
Full details: Title check

Explanation

The title "fix(host-service): re-register the host when the relay refuses it" follows conventional commit format and names the primary change. The changeset adds a registerHost function and an onForbidden recovery path that re-runs host.ensure after a 4403 forbidden close from the relay. The title is specific and matches the objectives of the PR.

Full details: Description check

Explanation

The description follows the template's main sections. It covers What & why with a linked issue (fixes #8445), a How I tested it section with concrete steps and results, and a checklist. The checklist items are marked as complete. The description does not include screenshots, which the template prefers for user-visible changes. This host-service change is not a UI change, so the gap is minor. The description also does not state whether the 4403 check uses the not_registered reason from the linked issue, and it does not say how failed re-registration is reported through health.check.

Full details: Linked Issues check

Explanation

The changes satisfy the coding requirements in [#8445]. TunnelClient intercepts relay close code 4403, invokes recovery, and delays reconnects with the required 0, 30, 60-second sequence up to five minutes. Recovery calls registerHost, which reruns host.ensure, uses a 30-second attempt deadline, retries with 30-second-to-five-minute backoff, and records failures for health reporting. The client resets the refusal counter after an inbound control message. Tests cover the recovery ordering, backoff, reset, and registration deadline.

Full details: Out of Scope Changes check

Explanation

The reviewed changes stay within [#8445]. The production changes implement host re-registration after relay refusal, retry backoff, timeout handling, and the related tunnel behavior. The added tests verify these behaviors. No unrelated production change is established by the supplied evidence.

Finishing Touches: 💡 1
🛠️ Fix failing CI checks: 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@RohithReddy20
RohithReddy20 marked this pull request as draft October 26, 2026 17:52
@greptile-apps

greptile-apps Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The changes since the previous review appear safe to merge.

Summary

The PR re-registers hosts after the relay refuses a connection.

  • Hosts re-register before reconnecting after a relay refusal.
  • Registration requests stop waiting after 30 seconds.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Relay closes with 4403] --> B[Pause reconnects]
  B --> C[Record registration failure]
  C --> D[Retry host.ensure until success or shutdown]
  D --> E[Wait before reconnecting after repeated refusals]
  E --> F[Reconnect to relay]
  F --> G[First control message resets refusal count]
Loading

Reviews (2) · Last reviewed commit: "fix(host-service): trim the relay refusa..." · Reviewed by Greptile

@RohithReddy20
RohithReddy20 marked this pull request as ready for review October 26, 2026 18:34
@RohithReddy20

Copy link
Copy Markdown
Author

@coderabbitai review

@RohithReddy20

Copy link
Copy Markdown
Author

@greptileai review

@coderabbitai

coderabbitai Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor
⚠️ Action not completed

Already reviewed the last commit. Use @coderabbitai full review to rerun a review of the entire changeset.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@RohithReddy20

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA

github-actions Bot added a commit that referenced this pull request Oct 26, 2026
Kitenite added a commit that referenced this pull request Nov 8, 2026
* chore: seed CLA signature file

* @tanxarx has signed the CLA in #8881

* @ozeitis has signed the CLA in #8888

* @shubhankiit has signed the CLA in #8894

* @chrisfentiman has signed the CLA in #8910

* @RohithReddy20 has signed the CLA in #8907

* @larcane97 has signed the CLA in #8922

* @moizghumann has signed the CLA in #8935

* @andyst-dev has signed the CLA in #7930

* @svenpeeters has signed the CLA in #8943

* @kenil-kanani has signed the CLA in #8940

* @akshatcoder-hash has signed the CLA in #8963

* @NilotpalK has signed the CLA in #8995

* @serhii-kucherenko has signed the CLA in #9000

* @tonydzi has signed the CLA in #9002

* @adaam2 has signed the CLA in #9038

* @tsavo-at-pieces has signed the CLA in #9059

* @ganeshmshetty has signed the CLA in #9130

* @aaronrosenthal has signed the CLA in #9135

* @alexnault has signed the CLA in #9138

* @kunalgoesbyken has signed the CLA in #9235

* @Joaco-gavernet has signed the CLA in #9236

* @theblondealex has signed the CLA in #8240

* @talarczykco has signed the CLA in #7274

* @throw99 has signed the CLA in #7906

* @lacherogwu has signed the CLA in #9310

* @KGKallasmaa has signed the CLA in #9316

* @Niltonsf has signed the CLA in #9331

* @Anmol1578 has signed the CLA in #9351

* @RyoKusnadi has signed the CLA in #9317

* @thedotmack has signed the CLA in #9414

* @abudker has signed the CLA in #9439

* @sanchitmonga22 has signed the CLA in #9443

* @Bruno-rose has signed the CLA in #9446

* @Ashad001 has signed the CLA in #9487

* @vivek-pothina has signed the CLA in #8359

* @colingreig has signed the CLA in #9489

* @anmol0b has signed the CLA in #9492

* @resteqs has signed the CLA in #9506

* @pgilmore has signed the CLA in #7878

* fix(desktop): name the DMG from the display name

The DMG file and its volume title now use DESKTOP_DISPLAY_NAME, so a
Paradev build ships Paradev-<version>-<arch>.dmg. Unset, names stay
Superset. The release keeps the Superset-<arch>.dmg alias beside the new
one, since the README, docs and download button link to it. The .app
bundle and the -mac.zip update files keep the Superset name.

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

host-service never re-registers after the relay answers 4403 not_registered, so a running host stays invisible until restarted

1 participant