Skip to content

Reopen the pinned SSH master in the background so finished runs reconcile (OR-598) - #778

Closed
devin-ai-integration[bot] wants to merge 2 commits into
mainfrom
devin/1792839406-ssh-reconnect-master
Closed

devin-ai-integration[bot] wants to merge 2 commits into
mainfrom
devin/1792839406-ssh-reconnect-master

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Fixes OR-598.

Summary

  • Root cause: on Unix, the SSH service reaches unmanaged hosts only through the run's pinned ControlMaster socket (-S <socket>, ProxyCommand=false, BatchMode=yes). If that master dies (network drop, sleep, ControlPersist expiry), nothing reopens it until a manual orx compute connect. Meanwhile watch_ssh_job keeps the run at running even after the host is back and the remote exit_code exists.
  • Fix (src/jobs/ssh/service/transport.rs): after state::validate_configuration, transport::connect runs reopen_master for unmanaged targets: ssh -oBatchMode=yes -oClearAllForwardings=yes -T <resolved.options> -- <dest> true. It uses only the pinned options and snapshot config, so the master comes back at the same ControlPath. It never prompts and is capped at 30s. Hosts that need a password or MFA still fail and show the existing "Use Connect" message. A live master just multiplexes true.
  • Unit test reopen_never_prompts_and_uses_the_pinned_route covers the arguments.

Test plan

  • cargo build --locked, cargo fmt --check, cargo clippy --locked -- -D warnings, node ui/scripts/check-i18n.mjs, node ui/scripts/check-styles.mjs, pnpm typecheck
  • cargo test --locked: 1270 passed. 3 local::projects tests fail locally only because of this VM's global git identity; they are unrelated to this diff. CI is authoritative.
  • Reproduced on main (Linux, Docker sshd host lab, key auth). Steps: orx exp run --backend ssh --host lab --no-container, disconnect the container network, let the remote job write exit_code 0, then reconnect. The run stayed Running for more than 4 minutes ("Monitoring unavailable … Reconnect with orx compute connect") and only finished after a manual Connect.
  • Same flow on this branch (cdc8b1f): the dashboard Claude chat replied live, and after the network came back Run 3 went to Done by itself, with no Connect.

Bug reproduction on main

Post-fix functionality check

Link to Devin session: https://app.devin.ai/sessions/8fd5cca120e446cd8817fe62783a6b58
Open in Devin Desktop: https://app.devin.ai/desktop/session/8fd5cca120e446cd8817fe62783a6b58?variant=devin

…le (OR-598)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@greptile-apps

greptile-apps Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; both previous findings are addressed and no new actionable issues were found.

Summary

The PR reopens an unmanaged host's saved SSH connection before starting the remote helper.

  • SSH monitoring can reopen lost connections to unmanaged hosts.

Reviews (2) · Last reviewed commit: "Reopen the pinned SSH master after valid..." · Reviewed by Greptile

Comment thread src/jobs/ssh.rs Resolved
Comment thread src/jobs/ssh.rs Resolved
@devin-ai-integration devin-ai-integration Bot changed the title Reopen dropped SSH masters in the background so finished runs reconcile (OR-598) Reopen the pinned SSH master in the background so finished runs reconcile (OR-598) Oct 24, 2026
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@greptileai review

@paperclover

Copy link
Copy Markdown
Contributor

comment written by Claude Opus 5.5

superseded by #862.

@paperclover paperclover closed this Nov 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting review Devin bugfix PR is open and needs human review before merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants