Skip to content

Holds: ask the person "approve or skip?" so held calls get answered - #821

Merged
LoopGlitch26 merged 3 commits into
Cygnux-Labs:mainfrom
hayatexe:agent/issue-816
Nov 14, 2026
Merged

LoopGlitch26 merged 3 commits into
Cygnux-Labs:mainfrom
hayatexe:agent/issue-816

Conversation

@hayatexe

Copy link
Copy Markdown
Contributor

What and why

In the first field report (#815), 63 of 70 held calls expired unanswered: in the desktop app a held call returns at
once with the approve command, and the agent usually reworded the call or moved on instead of asking the person.

stopped() (tracekit/integrations/claude_code.py), which every harness uses for a stopped call, now tells the agent
for a held call:

ask the person: approve this call or skip it? Don't reword the call to get past the hold; wait for their answer.
Pass on the Person line and say what the call was for; a subagent puts both in its final report. Once it is
approved, make the identical call again.

It used to say "tell the person what this call was for and pass on these commands, then wait for their answer; …".
The question comes first because agent_hooks._reply cuts other harnesses' messages at 500 characters: with a
typical Why and Instead line, Codex used to get only the start of the old guidance. The Person: line with its three
commands, the subagent rule, the Instead-step sentence, "Never hide what a call runs to get past the policy" and the
blocked message are unchanged. The held-message examples in docs/coding-agents.md and
docs/quickstarts/claude-code.md match (docs/approvals.md doesn't show the message).

Fixes #816

How I ran it as a user

Rendered a TK-A032 hold (cloud upload, with an Instead step) through Claude Code's _block and Codex's
agent_hooks._reply.

Claude Code (stderr, systemMessage and the deny reason):

[tracekit] HELD, nothing ran: apr-6f0c2d1e (TK-A032), expires 09:27 UTC.
Why: uploading to cloud storage
Instead: Have a person approve the held call, or write the files locally and let a person upload them.
Person: look with `! tracekit approvals show apr-6f0c2d1e`; allow once with `! tracekit approvals approve apr-6f0c2d1e`; refuse with `! tracekit approvals reject apr-6f0c2d1e`.
Agent: ask the person: approve this call or skip it? Don't reword the call to get past the hold; wait for their answer. Pass on the Person line and say what the call was for; a subagent puts both in its final report. Once it is approved, make the identical call again. If the Instead line is a step you can take yourself, you may take it and say so. Never hide what a call runs to get past the policy: every call is signed and kept.

Codex (the deny reason, cut at 500 characters):

[tracekit] HELD, nothing ran: apr-6f0c2d1e (TK-A032), expires 09:27 UTC.
Person: look with `tracekit approvals show apr-6f0c2d1e`; allow once with `tracekit approvals approve apr-6f0c2d1e`; refuse with `tracekit approvals reject apr-6f0c2d1e`.
Why: uploading to cloud storage
Instead: Have a person approve the held call, or write the files locally and let a person upload them.
Agent: ask the person: approve this call or skip it? Don't reword the call to get past the hold; wait for their answer. …

The question and the don't-reword rule survive the cut.

Tests: test_hook_v2.py, test_constraints.py, test_grants.py, test_agent_approvers.py, test_harness_hooks_v2.py
and test_docs.py on Windows: 267 passed, 93 skipped (POSIX-only). pyflakes clean on the changed files. Linux CI is
the real check for the skipped hook tests.

Checklist

  • make check passes (lint, tests, build); make test-ts too if sdk/typescript/ changed
  • a regression test covers the change (HoldLine.test_a_held_call_asks_the_person_approve_or_skip)
  • user-visible changes are described above (release notes are written at release time; don't edit CHANGELOG.md)
  • docs updated if behaviour or a security claim changed (docs/coding-agents.md, docs/quickstarts/claude-code.md)
  • every commit is signed off (git commit -s, see CONTRIBUTING.md)

Security invariants

  • the agent still never holds a signing key or assigns sequence numbers
  • nothing an agent-controlled process supplies (isolation level, fail mode, gap/tamper records, approval identity) is newly trusted by the signer or verifier
  • gaps are still signed, never silent; a check that cannot run warns instead of passing
  • no verification code is shipped inside a bundle

Evidence format

  • this change does not touch the frozen formats

In Cygnux-Labs#815, 63 of 70 held calls expired unanswered: the agent got the
approve command at once and usually reworded the call or moved on
instead of asking. stopped()'s held-call guidance now leads with one
question, "approve this call or skip it?", and says not to reword the
call to get past the hold. The question comes first because
agent_hooks._reply cuts other harnesses' messages at 500 characters,
which already cut the old guidance short.

The Person line, the subagent rule, the Instead step, "Never hide
what a call runs" and the blocked message are unchanged.

Fixes Cygnux-Labs#816

Signed-off-by: hayatexe <145812188+hayatexe@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings November 13, 2026 13:27

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

@LoopGlitch26

Copy link
Copy Markdown
Member

Thanks. Putting the question first because _reply cuts other harnesses at 500 characters is a good catch: in Codex the Agent line comes after Person, Why and Instead, so only its start survives.

One change before merge. "Don't reword the call to get past the hold" now contradicts the Instead step for the holds where rewording is the intended path. TK-SHELL-PARSE is the most common hold (93 of our last 134 unanswered holds), and its Instead line is "Split it into plain commands; pipes, &&, if, for, case and functions are read." The #815 report says that's what worked: "Writing the command out plainly got through, as the message suggests." With this PR the agent first reads "Don't reword the call", and only at the end "If the Instead line is a step you can take yourself, you may take it". An agent that obeys the first sentence stops splitting the command and waits on a hold it could have cleared itself.

What the message needs to say: when there's an Instead line, taking its step (and saying so) is allowed and isn't "rewording to get past the hold". Rewording to dodge the rule is what's forbidden. One way: with an Instead line, "Don't reword the call to get past the hold, except to take the Instead line's step when it is yours to take: then say so"; without one, the sentence as you have it. The trailing "If the Instead line is…" sentence then goes, so it isn't said twice. Pick the wording you like, as long as it's one consistent rule and stays near the front, since the 500-character cut applies.

Please also add a test with a TK-SHELL-PARSE-style hold (an Instead line, no person-only step) that checks the message lets the agent take the Instead step, and a no-Instead hold that keeps "Don't reword the call to get past the hold".

… step (Cygnux-Labs#816 review)

"Don't reword the call to get past the hold" came before "If the
Instead line is a step you can take yourself", so an agent that obeyed
the first stopped splitting a TK-SHELL-PARSE command into plain ones,
the step that got through in Cygnux-Labs#815. With an Instead line the held text
now says: don't reword around the hold, but do take an Instead step
that's yours and say so; otherwise wait. It is short enough that both
halves survive agent_hooks._reply's 500-character cut. Without an
Instead line the rule is as before.

Signed-off-by: hayatexe <145812188+hayatexe@users.noreply.github.com>
@hayatexe

Copy link
Copy Markdown
Contributor Author

Thanks, that's a real contradiction. Fixed in 452b9e4:

  • With an Instead line the held text is one rule: "ask the person: approve or skip? Don't reword around the hold, but do take an Instead step that's yours and say so; otherwise wait for their answer." The trailing "If the Instead line is…" sentence is gone for held calls (the blocked message keeps it, unchanged).

  • Without an Instead line it's as before: "…Don't reword the call to get past the hold; wait for their answer."

  • I shortened the question to "approve or skip?" in the Instead case. My first try at your wording still got cut inside the exception in Codex. With this wording both halves survive _reply's 500 characters for the TK-SHELL-PARSE and TK-A032 examples:

    Agent: ask the person: approve or skip? Don't reword around the hold, but do take an Instead step that's yours and say …
    
  • Tests: test_a_held_call_with_an_instead_step_lets_the_agent_take_it (TK-SHELL-PARSE-style reason; checks the rule is there once and that agent_hooks._reply("codex", …) keeps both halves), and the no-Instead test now checks the full "Don't reword the call to get past the hold; wait for their answer." and no Instead wording. Both fail with the code change reverted. docs/coding-agents.md and the Claude Code quickstart match.

test_hook_v2.py, test_constraints.py, test_grants.py, test_agent_approvers.py, test_harness_hooks_v2.py, test_docs.py: 268 passed, 93 skipped (POSIX-only) on Windows; pyflakes clean.

@LoopGlitch26

Copy link
Copy Markdown
Member

Main is fixed now (#820, the lint error and the gptme audit test that failed every PR's CI). Please merge main into your branch so CI runs against it; a re-run alone reuses the old combined snapshot.

Signed-off-by: hayatexe <145812188+hayatexe@users.noreply.github.com>
@LoopGlitch26
LoopGlitch26 enabled auto-merge (squash) November 14, 2026 00:31
@LoopGlitch26
LoopGlitch26 merged commit b7c22da into Cygnux-Labs:main Nov 14, 2026
32 of 34 checks passed
@LoopGlitch26

Copy link
Copy Markdown
Member

Merged, thank you. A held call now tells the agent to ask the person "approve or skip?" instead of moving on. Holds that expired unanswered were the biggest problem in #815's week (63 of 70), so this should show up in the next report.

@hayatexe

Copy link
Copy Markdown
Contributor Author

The two Windows CI failures here (py3.9 on windows-latest and py3.13 on windows-latest) both come down to a single test, and it looks unrelated to this PR:

FAILED tests/test_agent_hooks.py::Harnesses::test_cursor_edit_tools_are_decided_as_edits
  - AssertionError: Tuples differ: (0, 'allow') != (2, 'deny')

A Cursor StrReplace edit of a .env file is expected to be denied, but on Windows it's allowed — the .env match seems to miss under Windows path separators. Every Linux and macOS job (including py3.9/py3.13) is green; it's only the two Windows runners.

This PR only touches tracekit/integrations/claude_code.py, tests/test_hook_v2.py and two docs — nothing in test_agent_hooks.py or the Cursor integration. The same single test also fails identically on #819's Windows jobs, so it appears to be a pre-existing, Windows-specific issue rather than anything introduced here. Flagging in case it's useful.

@LoopGlitch26

Copy link
Copy Markdown
Member

Thanks @hayatexe, that's the right read. It wasn't from this PR: the v1 file rules matched only / separators, so a Windows path like C:\p\.env slipped past them. #838 (for #827) fixed that on main today, so the Windows jobs are green again from a14d842 on.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Holds: ask the person "approve or skip?" so held calls get answered (63 of 70 expired in #815)

3 participants