Server ingest treats every turn_complete as a finished turn #100265

Activity

1. tatoalo

   self-assigned this

   on Sep 14, 2026
2. tatoalo commented on Sep 17, 2026

   

   tatoalo

   on Sep 17, 2026

   Last edited by tatoalo

   ContributorAuthor

   More actions

   1. The Temporal signal must keep firing. signal_agent_turn_completed() (products/tasks/backend/models.py:2559) does not report a completion — it signals agent_state_changed(False), which marks the agent idle, re-arms the short idle window (workflow.py:1489) and clears the in-flight-turn timeout extension (workflow.py:873). All of that is correct for an idle resume. Only notify_task_run_turn_completed() — the "<title>" finished push plus _project_completed_activity — is wrong. So no new callback kind is needed, and the contract change is one optional field.

   2. The live path is not the one the issue names. Sandboxes write to agent-proxy in production, so the branch that fires today is products/tasks/backend/agent_proxy_callback.py:142 (the awaiting_input kind), which the issue does not mention. event_ingest.py is the fallback reached when TASKS_AGENT_PROXY_INGEST_URL is unset. Both need the fix; the callback is the one that stops the bleeding.

   The desktop consumers are all fine, including the two #100208 did not touch. packages/core/src/sessions/sessionService.ts:3566 gates notifyTurnCompleted on stopReason === "end_turn", so it already excludes idle_resume. The broken set is exactly the four server-side sites.

   Still live in production

   • The producer ships: published @posthog/agent@2.4.187 (current latest) contains IDLE_RESUME_STOP_REASON and settleIdleResume.
   • The trigger fires: sandbox TTL rotation. workflow.py:2850 writes same_run_resume_idle: True unconditionally, provision_sandbox.py:563 turns that into POSTHOG_RESUME_IDLE=1, and the agent-server then takes the idle branch and broadcasts turn_complete with stopReason: idle_resume.
   • Rotation is idle-gated by construction — _sandbox_rotation_block_reason (workflow.py:794) refuses to rotate while self._agent_active — so every successful rotation is exactly this case. posthog_tasks_sandbox_deadline_total{outcome="rotated"}, whose help text already scopes it to interactive cloud runs, records them daily.
   • The 30-second turn_completed cooldown does not absorb it: it is keyed per run, and a rotation lands long after the run's last real turn. _project_completed_activity runs before the cooldown check regardless, so the activity row flips either way.

   PR 1 — the fix

   Add a predicate beside the existing ones rather than changing them. is_turn_complete must keep returning True for an idle resume: ee/hogai/sandbox/executor.py:355 uses it to close Max's SSE loop, and agent-proxy uses it to set agentActive(false), which is right.

   • ee/hogai/sandbox/types.py — IDLE_RESUME_STOP_REASON = "idle_resume" and is_idle_resume_turn_complete(event), reading notification.params.stopReason. Mirrors packages/shared/src/session-events.ts:94.
   • services/agent-proxy/src/lib/side-effects.ts — isIdleResumeTurnComplete, the byte-identical twin.

   Wire contract: one optional boolean on the existing awaiting_input payload.

   • side-effects.ts:290 — still setAgentActive(false), still fires awaiting_input, now with turn_completed: !isIdleResumeTurnComplete(event).
   • AgentProxyCallbackRequestSerializer (presentation/serializers.py:4578) — add turn_completed = BooleanField(required=False, default=True).
   • agent_proxy_callback.py:142 — keep signal_agent_turn_completed(); gate notify_task_run_turn_completed(task_run) on the flag.
   • event_ingest.py:396 — the fallback holds the raw event, so it calls the predicate directly and passes the same flag into _dispatch_turn_completed_sync.

   A field rather than a new kind value because the two services deploy independently. Older Django ignores an unknown field (DRF Serializer does not reject extras); older agent-proxy omits it and gets default=True. Both orders degrade to current behavior. A new kind would return 400 against an un-deployed Django instead.

   Add a counter — TURN_COMPLETED_SUPPRESSED_TOTAL{reason="idle_resume"} in products/tasks/backend/metrics.py, incremented at the single Django decision point, which covers both ingest paths since both converge there. Without it the fix can only be inferred.

   Tests extend the existing files: test_agent_proxy_callback.py (flag false means the signal fires and the notify does not), test_event_ingest.py (same through the raw event), and services/agent-proxy/tests/hono/ingest-handler.test.ts for the callback body.

   PR 2 — stop the drift

   The predicate lives in four places and drifted because #100208 fixed the desktop copies and nothing failed when the server copies were left behind. The comments assert they are "byte-identical to ee/hogai/sandbox/types.py" and nothing checks it.

   Add a golden contract fixture — a JSON list of {event, expect: {turn_complete, idle_resume, pi_error, session_update}} — at ee/hogai/sandbox/turn_event_contract.json, beside the Python that is already named as the reference. Both suites load it and assert their implementation agrees: a pytest case in ee/hogai/sandbox/, a vitest case in services/agent-proxy/tests/lib/. Adding a stop reason then means adding a row, and whichever language has not caught up goes red.

   Land PR 1 first and stack PR 2 on it. PR 1 closes this issue on its own, and the fixture's expectations are only correct once the fix is in.

   Flagged, not fixed here

   A run that rotates before the agent was ever active gets agent_state_changed(False), flipping _end_of_turn_received to True and so changing what _agent_lost_mid_turn() (workflow.py:2794) reports for a workflow-origin run. That is current behavior and this plan does not change it either way, but it deserves its own issue rather than being altered quietly inside this one.

   Alternative considered

   Fix it at the producer: have settleIdleResume emit a distinct method (_posthog/idle_resume_settled) instead of _posthog/turn_complete, so every consumer matching on the method name is correct by default, including ones nobody updates. More robust in the abstract. Not recommended here because the idle-resume event is currently what propagates "agent is idle" to setAgentActive(false) and the Temporal signal, so a rename means re-plumbing that edge in all four consumers — more surface on the live path, for a case the field-based fix already handles safely in either deploy order.
3. trunk-io

   closed this as completedin #102378on Sep 18, 2026