Skip to content

Treat a non-numeric SSH exit_code receipt as missing so dead runs finalize - #792

Closed
devin-ai-integration[bot] wants to merge 1 commit into
mainfrom
devin/1792998318-ssh-empty-exit-receipt
Closed

devin-ai-integration[bot] wants to merge 1 commit into
mainfrom
devin/1792998318-ssh-empty-exit-receipt

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Fixes OR-618.

Summary

  • When the remote disk is full, the launcher's printf ... > exit_code still creates exit_code but cannot write to it, so the file is empty. host_job_script (and the container host_state) checked [ -f exit_code ] and printed EXIT . parse_job_state rejected that, and watch_ssh_job read the rejection as an outage ("Monitoring unavailable: unexpected inspect output: EXIT"). So the run stayed running/Cancelling forever, and repeating orx exp cancel did nothing.
  • Both inspect scripts now accept the receipt only if it is a number (grep -Eqx '[0-9]+'). Anything else goes on to the existing pid liveness check. A dead launcher then reports DEAD: the run is marked cancelled if a cancel was requested, otherwise error with "process died without an exit code". The invalid receipt stays on disk untouched as evidence.
  • Extended host_inspect_reports_a_vanished_run_dir_as_missing to cover an empty receipt with a dead pid.

Test plan

  • cargo fmt --check, cargo clippy --locked -- -D warnings, cargo test --locked jobs::ssh; CI green
  • Reproduced on main: SSH backend to localhost, --remote-root set to a 64 MB tmpfs. The run fills the disk and exits, leaving a 0-byte exit_code and no processes. orx exp cancel was accepted twice, but the run stayed running/Cancelling.
  • Same flow on this branch, with a live Claude Code dashboard chat (CLAUDE_OK) in the same slot:
    • Same command: 0-byte exit_code left in place, no processes, and the run finalizes on its own as failed ("process died without an exit code (killed?)").
    • Same disk-full setup, but the process kept alive (dd ... || true; sleep 600) and then stopped with orx exp cancel: 0-byte exit_code, supervisor finished (cancelled), and the dashboard shows Cancelled.
  • Not covered end to end: the container (--image) SSH path. Only the shared script change applies there.

Bug reproduction on main

Post-fix functionality

Automation improvements

  • The Codex preflight failed because CODEX_AUTH_JSON holds a refresh token that has already been used. Refreshing that secret from a fresh ChatGPT Codex login would let Codex-dependent bugs be reproduced.

Link to Devin session: https://app.devin.ai/sessions/bf679bd9a4624330abbceb026ca2296d
Open in Devin Desktop: https://app.devin.ai/desktop/session/bf679bd9a4624330abbceb026ca2296d?variant=devin

…alize

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@greptile-apps

greptile-apps Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] This PR appears safe to merge.

Summary

SSH inspection now treats an invalid exit_code receipt as missing and checks whether the launcher is alive.

  • SSH runs can finish when their exit receipt is empty.

Reviews (1) · Last reviewed commit: "Treat a non-numeric SSH exit_code receip..." · Reviewed by Greptile

@paperclover

Copy link
Copy Markdown
Contributor

comment written by Claude Opus 5.5

superseded by #862.

@paperclover paperclover closed this Nov 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting review Devin bugfix PR is open and needs human review before merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants