Skip to content

fix(agents): defer the sleeping-agent sweep until a direct-SSH workspace merges its snapshot - #12704

Closed
robertnisipeanu wants to merge 2 commits into
stablyai:mainfrom
robertnisipeanu:fix/sleeping-agent-remote-hydration-gate
Closed

robertnisipeanu wants to merge 2 commits into
stablyai:mainfrom
robertnisipeanu:fix/sleeping-agent-remote-hydration-gate

Conversation

@robertnisipeanu

Copy link
Copy Markdown
Contributor

Summary

Fixes #12701.

For direct-SSH workspaces the worktree tab lists arrive with the remote snapshot merge, after connect; the sleeping-agent sweep runs earlier (its only gate, workspaceSessionReady, means local hydration) and judges every record against tab lists that don't exist yet — cold-resuming live remote sessions and letting the background wake's canonical-record dedupe delete records permanently. This PR sequences the sweep behind the merge:

  • sleeping-agent-remote-hydration-gate.ts — shouldDeferSleepingAgentResumeForRemoteHydration: defer only when the worktree's resolved execution host is an SSH target that is connected/connecting, not in remoteWorkspaceHydratedTargetIds, and whose first pull hasn't concluded (sync phase absent or pulling). Local worktrees, disconnected targets, and settled pulls pass through unchanged, so offline sessions keep today's behavior.
  • Both sweep entry points check the gate: resumeSleepingAgentSessionsForWorktree (covers all worktree-activation callers) and wakeSleepingAgentsForWorktreeInBackground (whose canonical-record dedupe is destructive and must not run against a pre-merge tab list). Deferred background wakes are buffered, not dropped.
  • Every pull conclusion settles the deferral: the snapshot apply re-runs the sweep for the visible worktree and replays buffered wakes; concludePullWithoutApply (shared by the previously silent no-apply-token exits of both syncAfterConnect and applyUnsolicitedSnapshot), the offline branch, the revision-0 branch, and both workspace-hydration-timeout branches all reach a terminal phase and flush the wake buffer. A buffered wake never outlives the deferral that parked it; replayed wakes whose target is still mid-pull re-buffer without looping.

Validation

  • pnpm exec vitest run --config config/vitest.config.ts src/renderer/src/lib/resume-sleeping-agent-session-ssh-hydration-gate.test.ts src/renderer/src/lib/wake-sleeping-agents-remote-hydration.test.ts src/renderer/src/lib/wake-sleeping-agents-in-background.test.ts src/renderer/src/hooks/remote-workspace-target-sync.test.ts
  • pnpm run typecheck:web

New tests cover: deferral while a connected target is mid-pull; sweep proceeds when hydrated / disconnected / pull concluded offline; local worktrees never deferred; buffered wakes replay once on flush and re-buffer while still deferred; no-apply-token and hydration-timeout pulls conclude with a terminal error phase.

🤖 Generated with Claude Code

@coderabbitai

coderabbitai Bot commented Aug 10, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The change adds SSH remote-hydration gates for sleeping-agent resume and activation. The background wake dispatcher tracks deferred intents and remote pull lifecycles. Remote target synchronization reports terminal outcomes and settles active pulls during errors and shutdown. IPC handling centralizes direct-SSH teardown ordering. Tests cover hydration, coalescing, topology changes, pull failures, cleanup, fanout performance, and SSH restart restoration. Reliability-gate records add commands, assertions, performance evidence, and OpenSSH results.

Merge Risk: 🔵 Low · up to cd01a

Direct-SSH sleeping-agent wakes may remain buffered when the remote workspace bridge is unavailable, delaying expected resumes, and the reliability-gate record has incomplete assertion traceability. These are bounded follow-up issues that should have explicit owner awareness before or shortly after merge.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description explains the fix, linked issue, and validation, but it omits most required template sections, including ELI5, Visual Proof, Review, Notes, Checklist, and Author. Complete the repository template by adding the missing sections and provide Visual Proof or state N/A with a reason.
Docstring Coverage ⚠️ Warning Docstring coverage is 14.81% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely describes deferring sleeping-agent sweeps until direct-SSH workspace snapshot hydration completes.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🧹 Nitpick comments (3)
src/renderer/src/lib/wake-sleeping-agents-remote-hydration.test.ts (1)

120-129: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Derive the loop bound from the production constant.

The literal 256 duplicates MAX_SETTLED_REMOTE_PULL_TARGETS in src/renderer/src/lib/wake-sleeping-agents-in-background.ts. If the constant changes, this test still passes for the wrong reason or fails without an obvious cause. Export the constant and import it here.

♻️ Proposed refactor

In src/renderer/src/lib/wake-sleeping-agents-in-background.ts:

-const MAX_SETTLED_REMOTE_PULL_TARGETS = 256
+export const MAX_SETTLED_REMOTE_PULL_TARGETS = 256

In this file:

-import { createBackgroundSleepingAgentWakeDispatcher } from './wake-sleeping-agents-in-background'
+import {
+  createBackgroundSleepingAgentWakeDispatcher,
+  MAX_SETTLED_REMOTE_PULL_TARGETS
+} from './wake-sleeping-agents-in-background'
-    for (let index = 0; index <= 256; index += 1) {
+    for (let index = 0; index <= MAX_SETTLED_REMOTE_PULL_TARGETS; index += 1) {
src/renderer/src/lib/wake-sleeping-agents-store-fanout.test.ts (1)

22-41: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Reset the module registry before vi.doMock, and dispose the dispatcher in afterEach.

Two hygiene gaps:

  1. vi.doMock('@/store', ...) runs without a preceding vi.resetModules(). src/renderer/src/hooks/ipc-events-test-harness.ts calls vi.resetModules() first, which is the pattern that guarantees the dynamic import at Line 36 receives the mock. Default per-file isolation hides the difference today, but the mock silently becomes a no-op if this file later gains a second test or a static import of the same module.
  2. dispatcher.dispose() runs only at Line 64. If any assertion above throws, the dispatcher stays registered as the global wake-intent owner through installSleepingAgentWakeIntentOwner. src/renderer/src/lib/wake-sleeping-agents-remote-hydration.test.ts uses an afterEach disposer for this reason.

Also applies to: 64-64

src/renderer/src/hooks/direct-ssh-hydration-teardown.ts (1)

4-10: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Record why the dispose order matters.

The order is load-bearing. targetSync.stop() settles in-flight pull attempts, which calls back into remotePullSettled. Disposing the dispatcher first makes those callbacks no-ops, so teardown cannot replay a deferred wake. A future reader can swap the two lines without noticing. A single-line comment prevents that, and matches the "Why:" convention used across src/renderer/src/hooks/useIpcEvents.ts.

♻️ Proposed change
 ): void {
+  // Why: stop() settles in-flight pulls; disposing first keeps those callbacks from replaying a wake.
   wakeDispatcher.dispose()
   targetSync?.stop()
 }

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 7f9a12cf-38bc-4157-8ba4-4a2be9869e0d

📥 Commits

Reviewing files that changed from the base of the PR and between a63df91 and 29a60e84ebe21f1840c33dc5888e561ddeb8b8fc.

📒 Files selected for processing (19)
  • config/reliability-gates.jsonc
  • src/renderer/src/hooks/direct-ssh-hydration-teardown.ts
  • src/renderer/src/hooks/ipc-events-test-harness.ts
  • src/renderer/src/hooks/remote-workspace-patch-status.ts
  • src/renderer/src/hooks/remote-workspace-target-sync.test.ts
  • src/renderer/src/hooks/remote-workspace-target-sync.ts
  • src/renderer/src/hooks/use-ipc-events-direct-ssh-teardown.test.ts
  • src/renderer/src/hooks/useIpcEvents.ts
  • src/renderer/src/lib/resume-sleeping-agent-session-remote-compat.test.ts
  • src/renderer/src/lib/resume-sleeping-agent-session-ssh-hydration-gate.test.ts
  • src/renderer/src/lib/resume-sleeping-agent-session.ts
  • src/renderer/src/lib/sleeping-agent-hydration-topology-subscription.ts
  • src/renderer/src/lib/sleeping-agent-remote-hydration-gate.ts
  • src/renderer/src/lib/sleeping-agent-wake-intent.ts
  • src/renderer/src/lib/wake-sleeping-agents-in-background.test.ts
  • src/renderer/src/lib/wake-sleeping-agents-in-background.ts
  • src/renderer/src/lib/wake-sleeping-agents-remote-hydration.test.ts
  • src/renderer/src/lib/wake-sleeping-agents-store-fanout.test.ts
  • tests/e2e/ssh-cold-activation-restore.spec.ts

Comment thread config/reliability-gates.jsonc Resolved
Comment on lines +691 to +694
const remoteWorkspaceApi = window.api.remoteWorkspace
const backgroundSleepingAgentWakeDispatcher = createBackgroundSleepingAgentWakeDispatcher({
remoteHydrationEnabled: remoteWorkspaceApi !== undefined
})

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Use one predicate for remoteWorkspaceApi; a null value enables the gate but never starts a pull.

Line 693 tests remoteWorkspaceApi !== undefined. Line 725 tests if (remoteWorkspaceApi). The two disagree when window.api.remoteWorkspace is null.

In that case:

  • remoteHydrationEnabled becomes true, so defer in the wake dispatcher is active.
  • remoteWorkspaceTargetSync stays null, so remotePullLifecycle never fires started or settled.

getSleepingAgentRemoteHydrationTargetId still returns a target id for a connected SSH host whose sync phase is undefined. defer then returns true and the intent parks in deferredByWorktreeId. No pull settlement can ever release it. The sleeping agents for that worktree never wake until an unrelated topology change clears the target.

Align both checks on truthiness.

🐛 Proposed fix
     const remoteWorkspaceApi = window.api.remoteWorkspace
     const backgroundSleepingAgentWakeDispatcher = createBackgroundSleepingAgentWakeDispatcher({
-      remoteHydrationEnabled: remoteWorkspaceApi !== undefined
+      remoteHydrationEnabled: Boolean(remoteWorkspaceApi)
     })
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const remoteWorkspaceApi = window.api.remoteWorkspace
const backgroundSleepingAgentWakeDispatcher = createBackgroundSleepingAgentWakeDispatcher({
remoteHydrationEnabled: remoteWorkspaceApi !== undefined
})
const remoteWorkspaceApi = window.api.remoteWorkspace
const backgroundSleepingAgentWakeDispatcher = createBackgroundSleepingAgentWakeDispatcher({
remoteHydrationEnabled: Boolean(remoteWorkspaceApi)
})

Comment on lines +18 to +20
export function deferSleepingAgentActivationWake(worktreeId: string): boolean {
return activeOwner?.deferActivation(worktreeId) ?? true
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Do not report deferral without an installed owner.

When activeOwner is null, Line 19 returns true. resumeSleepingAgentSessionsForWorktree then exits, but no component stores the worktree ID for replay. A resume that occurs before the dispatcher installs can leave the sleeping session unresumed.

Return false when no owner exists, or store ownerless intents until installation. Add a test for this lifecycle.

Comment on lines +45 to +58
const startedAt = performance.now()
for (let index = 0; index < 20_000; index += 1) {
const previous = state
state = {
...state,
agentStatusByPaneKey: { [`pane-${index}`]: { state: 'working' } },
terminalLayoutsByTabId: { [`tab-${index}`]: null }
} as unknown as AppState
subscriberRef.current?.(state, previous)
}
const durationMs = performance.now() - startedAt

expect(recomputeTargetTopology).toHaveBeenCalledOnce()
expect(durationMs).toBeLessThan(250)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Remove the wall-clock threshold; keep the call-count assertion.

expect(durationMs).toBeLessThan(250) measures wall-clock time for 20,000 synchronous notifications. On a contended CI runner, or on a slower macOS or Windows agent, this exceeds 250 ms and the test fails without a code regression.

expect(recomputeTargetTopology).toHaveBeenCalledOnce() already proves the fanout guarantee: unrelated writes do not recompute topology. The timer adds flake and no extra signal. If a performance budget must stay in CI, move it to a dedicated benchmark rather than a unit test.

💚 Proposed fix
-    const startedAt = performance.now()
     for (let index = 0; index < 20_000; index += 1) {
       const previous = state
       state = {
         ...state,
         agentStatusByPaneKey: { [`pane-${index}`]: { state: 'working' } },
         terminalLayoutsByTabId: { [`tab-${index}`]: null }
       } as unknown as AppState
       subscriberRef.current?.(state, previous)
     }
-    const durationMs = performance.now() - startedAt
 
     expect(recomputeTargetTopology).toHaveBeenCalledOnce()
-    expect(durationMs).toBeLessThan(250)
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const startedAt = performance.now()
for (let index = 0; index < 20_000; index += 1) {
const previous = state
state = {
...state,
agentStatusByPaneKey: { [`pane-${index}`]: { state: 'working' } },
terminalLayoutsByTabId: { [`tab-${index}`]: null }
} as unknown as AppState
subscriberRef.current?.(state, previous)
}
const durationMs = performance.now() - startedAt
expect(recomputeTargetTopology).toHaveBeenCalledOnce()
expect(durationMs).toBeLessThan(250)
for (let index = 0; index < 20_000; index += 1) {
const previous = state
state = {
...state,
agentStatusByPaneKey: { [`pane-${index}`]: { state: 'working' } },
terminalLayoutsByTabId: { [`tab-${index}`]: null }
} as unknown as AppState
subscriberRef.current?.(state, previous)
}
expect(recomputeTargetTopology).toHaveBeenCalledOnce()

…ace merges its snapshot

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@robertnisipeanu
robertnisipeanu force-pushed the fix/sleeping-agent-remote-hydration-gate branch from 29a60e8 to 1d7dfdd Compare August 14, 2026 23:15
@coderabbitai

coderabbitai Bot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

Resolves config/reliability-gates.jsonc by keeping upstream's renamed
test-file references and re-applying this PR's sleeping-agent hydration
gate entries (commands, testFiles) on top.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AqNqAZQTwmAkwGcHj5TYXk
@coderabbitai

coderabbitai Bot commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 4968ff1f-a361-4727-913a-604dc850916b

📥 Commits

Reviewing files that changed from the base of the PR and between 3ffab9a and cd01a77.

📒 Files selected for processing (27)
  • config/reliability-gates.jsonc
  • config/scripts/wsl-git-shell-benchmark.mjs
  • src/main/codex/codex-trust-grant-host.test.ts
  • src/main/local-worktree-filesystem-wsl-banner.wsl.test.ts
  • src/main/local-worktree-filesystem.test.ts
  • src/main/providers/local-pty-provider-windows-shell-launch.test.ts
  • src/renderer/src/components/settings/CliSkillRuntimeSetup.test.tsx
  • src/renderer/src/components/settings/CliSkillRuntimeSetup.tsx
  • src/renderer/src/hooks/direct-ssh-hydration-teardown.ts
  • src/renderer/src/hooks/ipc-events-test-harness.ts
  • src/renderer/src/hooks/remote-workspace-patch-status.ts
  • src/renderer/src/hooks/remote-workspace-target-sync.test.ts
  • src/renderer/src/hooks/remote-workspace-target-sync.ts
  • src/renderer/src/hooks/use-ipc-events-direct-ssh-teardown.test.ts
  • src/renderer/src/hooks/useIpcEvents.ts
  • src/renderer/src/lib/resume-sleeping-agent-session-remote-compat.test.ts
  • src/renderer/src/lib/resume-sleeping-agent-session-ssh-hydration-gate.test.ts
  • src/renderer/src/lib/resume-sleeping-agent-session.ts
  • src/renderer/src/lib/sleeping-agent-hydration-topology-subscription.ts
  • src/renderer/src/lib/sleeping-agent-remote-hydration-gate.ts
  • src/renderer/src/lib/sleeping-agent-wake-intent.ts
  • src/renderer/src/lib/wake-sleeping-agents-in-background.test.ts
  • src/renderer/src/lib/wake-sleeping-agents-in-background.ts
  • src/renderer/src/lib/wake-sleeping-agents-remote-hydration.test.ts
  • src/renderer/src/lib/wake-sleeping-agents-store-fanout.test.ts
  • src/shared/wsl-exec-mode-separator.test.ts
  • tests/e2e/ssh-cold-activation-restore.spec.ts
💤 Files with no reviewable changes (1)
  • src/main/codex/codex-trust-grant-host.test.ts
🚧 Files skipped from review as they are similar to previous changes (17)
  • src/renderer/src/lib/wake-sleeping-agents-in-background.test.ts
  • src/renderer/src/lib/wake-sleeping-agents-store-fanout.test.ts
  • src/renderer/src/hooks/remote-workspace-patch-status.ts
  • src/renderer/src/hooks/ipc-events-test-harness.ts
  • src/renderer/src/lib/sleeping-agent-wake-intent.ts
  • src/renderer/src/lib/wake-sleeping-agents-remote-hydration.test.ts
  • src/renderer/src/lib/sleeping-agent-remote-hydration-gate.ts
  • src/renderer/src/lib/resume-sleeping-agent-session-remote-compat.test.ts
  • src/renderer/src/lib/sleeping-agent-hydration-topology-subscription.ts
  • src/renderer/src/lib/resume-sleeping-agent-session.ts
  • src/renderer/src/hooks/use-ipc-events-direct-ssh-teardown.test.ts
  • src/renderer/src/hooks/direct-ssh-hydration-teardown.ts
  • src/renderer/src/lib/wake-sleeping-agents-in-background.ts
  • tests/e2e/ssh-cold-activation-restore.spec.ts
  • src/renderer/src/hooks/remote-workspace-target-sync.test.ts
  • src/renderer/src/lib/resume-sleeping-agent-session-ssh-hydration-gate.test.ts
  • src/renderer/src/hooks/remote-workspace-target-sync.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 8 remain after this review.

Comment on lines +4372 to 4380
"src/renderer/src/lib/resume-sleeping-agent-session-ssh-hydration-gate.test.ts",
"src/renderer/src/lib/resume-sleeping-agent-session-remote-compat.test.ts",
"src/renderer/src/lib/wake-sleeping-agents-in-background.test.ts",
"src/renderer/src/lib/wake-sleeping-agents-remote-hydration.test.ts",
"src/renderer/src/hooks/remote-workspace-target-sync.test.ts",
"src/renderer/src/hooks/use-ipc-events-direct-ssh-teardown.test.ts",
"src/renderer/src/lib/wake-sleeping-agents-store-fanout.test.ts",
"tests/e2e/ssh-cold-activation-restore.spec.ts"
],

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Add missing assertionRefs entries and expand the under-documented SSH restart entry.

Two test files appear in testFiles (Lines 4374-4375) and in the commands array (Line 4355), but assertionRefs has no entry for either of them:

  • src/renderer/src/lib/resume-sleeping-agent-session-remote-compat.test.ts
  • src/renderer/src/lib/wake-sleeping-agents-in-background.test.ts

Separately, the assertionRefs entry for tests/e2e/ssh-cold-activation-restore.spec.ts (Lines 4479-4484) lists only one assertion string, but the matching evidenceRuns summary (Line 4503) describes three distinct outcomes: "six-terminal remount, authenticated PTY-owner continuity, and preservation of one live provider-session record with zero resume claims." Add separate assertion entries for the remount and PTY-owner continuity behaviors, or narrow the evidence summary to match the single documented assertion.

A prior review already flagged this exact under-documentation pattern for this same test file and it was marked addressed in a previous commit. The gap has reappeared here, now compounded by two entirely undocumented test files. This weakens the traceability of the reliability-gate evidence for this record.

Also applies to: 4443-4484

@nwparker nwparker removed the P1 label Aug 19, 2026
@OrcaWin

OrcaWin commented Oct 11, 2026

Copy link
Copy Markdown
Collaborator

Thank you @robertnisipeanu for tracking this down. Current main now holds sleeping-agent resume until the SSH host has actually answered with its workspace snapshot (#16750, follow-ups #16956 and #23213), which covers the race this PR sequenced. Closing as fixed; please reopen if you still see a live remote session get cold-resumed.

@OrcaWin OrcaWin closed this Oct 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SSH: sleeping-agent sweep races the remote workspace snapshot merge — every restart cold-resumes live remote sessions and destroys their records

4 participants