Skip to content

fix(goal-loop): wait for background work the agent will be woken by - #1224

Merged
Juliusolsson05 merged 4 commits into
mainfrom
fix/goal-loop-background-tasks
Sep 25, 2026
Merged

Juliusolsson05 merged 4 commits into
mainfrom
fix/goal-loop-background-tasks

Conversation

@Juliusolsson05

@Juliusolsson05 Juliusolsson05 commented Sep 25, 2026 •

Copy link
Copy Markdown
Owner

Problem (#1138, class C1)

The goal loop continued the moment Claude's turn ended, even while the agent had async subagents or run_in_background shells that the CLI will wake it for with a task-notification. Each continuation in that gap got "still waiting" and cost one continuation (observed three in a row, 2026-09-22).

Evidence (staged decomposition; details in the #1138 comment)

  • Claude Code 2.1.280+ puts background_tasks on the Stop hook payload. Its own schema: lets hooks distinguish 'session is done' from 'session is paused waiting for background work to wake it'.
  • Captured unedited from 2.1.282 and saved as testing/fixtures/goal-loop-stop-hooks/claude-2.1.282.json:
    • [] when idle;
    • a running shell;
    • a running subagent;
    • a subagent's leftover shell after it finished.
  • Corpus: 807 transcripts. 97.4 % of 1,964 background launches pair with a delivered notification, and 33 % of main turn ends had background work pending.
  • Codex: child final answers carry trigger_turn:false, so a finishing child never wakes the parent. No change is needed there.

Change

  • BuiltInMcpHttpHost forwards background_tasks from a Stop. A missing or malformed field means unknown and isn't forwarded, so an older CLI behaves exactly as today.
  • GoalLoopService holds (background-work) while the last allowed Stop reported a running or pending shell, subagent, workflow or MCP task. The notification turn's own Stop replaces the set, and an empty list releases the hold.
  • Monitors and unknown kinds are not held on, since a persistent monitor never ends.
  • Background work is exempt from the 30-minute silence pause, like a running tool. Otherwise the loop would be paused when the notification turn arrives. The 2-hour absolute limit still applies.
  • The set is cleared on a stalled-hold pause (a Resume means continue anyway) and when the session's process dies.

Tests (fail-first, real payloads)

  • The service replays the captured Stops:
    • holds on a shell;
    • holds on a subagent;
    • continues when the field is absent;
    • doesn't hold on a monitor;
    • isn't paused for silence but is by the absolute limit.
  • The real MCP host over real HTTP carries the payload through to the loop.

Mutation-checked, each individually: host drops the field, silence not exempt, hold on monitors. Each turns a test red.

Verification

npx tsc -b clean; src/main/goalLoop passes (4 files, 69 tests).
Not verified here: a live overnight loop. The 11 % case (a notification already queued at Stop) isn't in background_tasks; the CLI wakes the agent immediately there, so the existing turn-open gate handles it.

Fixes #1138

🤖 Generated with Claude Code

Review disposition (A Pi, B Grok, C Pi; 2 rounds, plus steering)

Finding Disposition
A1/B1 MAJOR: the Monitor tool arrives as shell (verified in the 2.1.282 binary); a never-ending shell parked the loop for 2 h, paused it, and re-parked after every Resume Fixed (ac45391a): each report holds for GOAL_LOOP_BACKGROUND_WORK_LIMIT_MS (45 min), then DELIVERS; the next Stop opens a fresh window. Corpus: 92–94% of background tasks notify within 45 min
A2: a stale latch stretched a silent turn-open hold to 2 h Fixed: the gate and the silence exemption both use the expiring backgroundWorkHolds()
A3/C2: an unreadable Stop body released the hold Fixed: unknown keeps the latch. The host treats a non-empty list with ANY unreadable entry as unknown (steering q5, 474ae837)
A4/B/C test gaps: workflow, MCP task, pending, leftover-shell fixture, process death, host keep-on-unknown Fixed; each has a test and a killed mutant
C6: no trailing newline in the fixture Fixed
Round 2 (A, C): a chain of refreshed reports can still reach the 2 h pause Accepted by design (visible, resumable backstop). The comment is corrected (52bbd30a)
Round 2 (C): an unattended loop beside a never-ending shell spends its cap in about 19 h Accepted: cap-pause is visible and the cap can be raised

Round 2: A, B and C all MERGE-READY. The steering note q5 was applied after round 2, with a test and a mutation check.

A goal loop continued the moment Claude's turn ended, even when the agent
had async subagents or run_in_background shells running that the CLI will
wake it for with a task-notification. Each continuation in that gap got
'still waiting' and cost one continuation (observed three in a row on
2026-09-22).

Claude Code 2.1.280+ reports this itself: the Stop hook payload carries
background_tasks, which its schema describes as the way to tell 'done' from
'paused waiting for background work'. Captured unedited from 2.1.282
(testing/fixtures/goal-loop-stop-hooks/): [] when idle, a running 'shell',
a running 'subagent', and a subagent's leftover shell after it finished.

- The hook host forwards background_tasks from a Stop (it parsed the body
  and dropped the field). Absent or malformed means unknown and is not
  forwarded, so an older CLI keeps today's behaviour.
- The loop holds its continuation ('background-work') while the last
  allowed Stop reported a running/pending shell, subagent, workflow or MCP
  task; the notification turn's own Stop replaces the set, and an empty
  list releases it. Monitors and unknown task kinds are not held on (a
  persistent monitor never ends).
- Background work is exempt from the 30-minute SILENCE pause, like a
  running tool, because a paused loop would miss the notification turn;
  the absolute 2-hour limit still applies. Cleared on a stalled-hold pause
  (Resume means continue anyway) and when the session's process dies.

Codex needs no change: child final answers carry trigger_turn:false, so a
child finishing never wakes the parent (400 rollouts checked).

Tests replay the captured payloads through the service and, over real
HTTP, through the real MCP host. Mutation-checked: host dropping the
field, silence not exempt, holding on monitors.

Fixes #1138

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Juliusolsson05 added a commit that referenced this pull request Sep 25, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review of #1224, verified against the 2.1.282 binary: the Monitor tool is
a local_bash task and reaches the Stop payload as type "shell", so a
persistent monitor (or a dev server started in the background) parked an
unattended loop for the 2-hour limit, paused it, and re-parked it after
every Resume.

Each Stop's report now holds for GOAL_LOOP_BACKGROUND_WORK_LIMIT_MS
(45 min) and then expires, so the continuation lands as it did before
#1138 and the next Stop starts a fresh window. The silence exemption uses
the same expiring answer, so a stale report no longer stretches a silent
turn-open hold to 2 hours.

A Stop without a readable report keeps the latch (only a parse failure
reaches it; older CLIs never latch), and a list whose entries are all
unreadable is unknown, not empty.

Tests pin workflow, MCP task, pending, the recorded leftover-shell Stop,
process death, the persistent-monitor cycle and the stale-report pause.
All seven mutations are killed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Juliusolsson05 added a commit that referenced this pull request Sep 25, 2026
…round-1 fixes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Juliusolsson05 and others added 2 commits September 25, 2026 00:36
…ain comment

Round-2 review of #1224: an unparseable Stop body and a list with no
readable entry must keep the hold through the real host (mutating the
host to coerce the list to [] now fails). The pauseStalledHold clear is
reachable by a chain of refreshed reports, so its comment says so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Steering note q5 on #1224: dropping one unreadable entry could turn
"a workflow is still running" into "only a monitor is running" and
release the hold. Any unreadable entry now makes the report unknown,
which keeps the last report inside its 45-minute window. The host-path
test posts a readable monitor plus an unreadable workflow after a held
shell; restoring the all-unreadable-only guard fails it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Juliusolsson05
Juliusolsson05 merged commit 714335e into main Sep 25, 2026
2 checks passed
@Juliusolsson05
Juliusolsson05 deleted the fix/goal-loop-background-tasks branch September 25, 2026 07:56
Juliusolsson05 added a commit that referenced this pull request Sep 25, 2026
…question, #1228/#1229 round 2

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(goal-loop): continuations fire while the agent is waiting on its own background tasks

1 participant