Repository navigation
Holds: ask the person "approve or skip?" so held calls get answered - #821
Conversation
In Cygnux-Labs#815, 63 of 70 held calls expired unanswered: the agent got the approve command at once and usually reworded the call or moved on instead of asking. stopped()'s held-call guidance now leads with one question, "approve this call or skip it?", and says not to reword the call to get past the hold. The question comes first because agent_hooks._reply cuts other harnesses' messages at 500 characters, which already cut the old guidance short. The Person line, the subagent rule, the Instead step, "Never hide what a call runs" and the blocked message are unchanged. Fixes Cygnux-Labs#816 Signed-off-by: hayatexe <145812188+hayatexe@users.noreply.github.com>
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.
|
Thanks. Putting the question first because One change before merge. "Don't reword the call to get past the hold" now contradicts the Instead step for the holds where rewording is the intended path. TK-SHELL-PARSE is the most common hold (93 of our last 134 unanswered holds), and its Instead line is "Split it into plain commands; pipes, &&, if, for, case and functions are read." The #815 report says that's what worked: "Writing the command out plainly got through, as the message suggests." With this PR the agent first reads "Don't reword the call", and only at the end "If the Instead line is a step you can take yourself, you may take it". An agent that obeys the first sentence stops splitting the command and waits on a hold it could have cleared itself. What the message needs to say: when there's an Instead line, taking its step (and saying so) is allowed and isn't "rewording to get past the hold". Rewording to dodge the rule is what's forbidden. One way: with an Instead line, "Don't reword the call to get past the hold, except to take the Instead line's step when it is yours to take: then say so"; without one, the sentence as you have it. The trailing "If the Instead line is…" sentence then goes, so it isn't said twice. Pick the wording you like, as long as it's one consistent rule and stays near the front, since the 500-character cut applies. Please also add a test with a TK-SHELL-PARSE-style hold (an Instead line, no person-only step) that checks the message lets the agent take the Instead step, and a no-Instead hold that keeps "Don't reword the call to get past the hold". |
… step (Cygnux-Labs#816 review) "Don't reword the call to get past the hold" came before "If the Instead line is a step you can take yourself", so an agent that obeyed the first stopped splitting a TK-SHELL-PARSE command into plain ones, the step that got through in Cygnux-Labs#815. With an Instead line the held text now says: don't reword around the hold, but do take an Instead step that's yours and say so; otherwise wait. It is short enough that both halves survive agent_hooks._reply's 500-character cut. Without an Instead line the rule is as before. Signed-off-by: hayatexe <145812188+hayatexe@users.noreply.github.com>
|
Thanks, that's a real contradiction. Fixed in 452b9e4:
|
|
Main is fixed now (#820, the lint error and the gptme audit test that failed every PR's CI). Please merge main into your branch so CI runs against it; a re-run alone reuses the old combined snapshot. |
Signed-off-by: hayatexe <145812188+hayatexe@users.noreply.github.com>
|
Merged, thank you. A held call now tells the agent to ask the person "approve or skip?" instead of moving on. Holds that expired unanswered were the biggest problem in #815's week (63 of 70), so this should show up in the next report. |
|
The two Windows CI failures here ( A Cursor This PR only touches |
What and why
In the first field report (#815), 63 of 70 held calls expired unanswered: in the desktop app a held call returns at
once with the approve command, and the agent usually reworded the call or moved on instead of asking the person.
stopped()(tracekit/integrations/claude_code.py), which every harness uses for a stopped call, now tells the agentfor a held call:
It used to say "tell the person what this call was for and pass on these commands, then wait for their answer; …".
The question comes first because
agent_hooks._replycuts other harnesses' messages at 500 characters: with atypical Why and Instead line, Codex used to get only the start of the old guidance. The
Person:line with its threecommands, the subagent rule, the Instead-step sentence, "Never hide what a call runs to get past the policy" and the
blocked message are unchanged. The held-message examples in
docs/coding-agents.mdanddocs/quickstarts/claude-code.mdmatch (docs/approvals.mddoesn't show the message).Fixes #816
How I ran it as a user
Rendered a TK-A032 hold (cloud upload, with an Instead step) through Claude Code's
_blockand Codex'sagent_hooks._reply.Claude Code (stderr,
systemMessageand the deny reason):Codex (the deny reason, cut at 500 characters):
The question and the don't-reword rule survive the cut.
Tests:
test_hook_v2.py,test_constraints.py,test_grants.py,test_agent_approvers.py,test_harness_hooks_v2.pyand
test_docs.pyon Windows: 267 passed, 93 skipped (POSIX-only). pyflakes clean on the changed files. Linux CI isthe real check for the skipped hook tests.
Checklist
make checkpasses (lint, tests, build);make test-tstoo ifsdk/typescript/changedHoldLine.test_a_held_call_asks_the_person_approve_or_skip)CHANGELOG.md)docs/coding-agents.md,docs/quickstarts/claude-code.md)git commit -s, seeCONTRIBUTING.md)Security invariants
Evidence format