Repository navigation
Design: LangChain held ToolNode pass for several calls of one message (#687), after 1.7.0 - #756
Draft
LoopGlitch26 wants to merge 1 commit into
Draft
LoopGlitch26 wants to merge 1 commit into
LoopGlitch26 wants to merge 1 commit into
Conversation
This was referenced Nov 12, 2026
LoopGlitch26
added a commit
that referenced
this pull request
Nov 15, 2026
* LangChain: several asks in one message, each answered once (#687) Builds the design in #756. A tracekit_tool_node, or a ToolNode that tracekit.instrument() wired, now plans the calls of one model message together (the held pass): each call's state is established first (what this process did, else what the signer recorded, else decided now), the pass pauses once, before any tool runs, when any call asked, answered or not, then each call allowed or approved is consumed under its own approval and one refused is final. A call that ended is given back, not decided, consumed or run again. A tool's own interrupt() always comes after the gate's pause, so it gets the person's answer; a call whose tool paused after its consume goes on under that one consume. A RetryPolicy retry runs only the call that raised, as a new attempt. Under ainvoke a call that raises waits for its siblings, so none is cut off mid-run. The durable store is the signer (design option a): call_state, RPC contract 24 (RPC_MIN stays 15; 1.6.0 shipped 22 and main is already at 23), a read under the run token: the latest attempt's decision, whether the call's arguments are the decided ones, its approval and whether this attempt consumed it, a refused consume, and its tool.result. The signer keeps refused/ended on the call's state, rebuilt from the records on replay and kept in snapshots. A resumed node in another process reads it; an ended call comes back to the model as "ran; its result is recorded at run_seq N". A call consumed and not ended goes on only when a tool's own pause was answered since; otherwise the signer refuses it again (TK-APPROVAL-CONSUMED), as main does for a T2 executor's consume. A single call (create_agent's tasks, a ToolNode with one call) keeps the per-call gate; only a call whose tool paused after its consume goes on. An executor's tool stays the executor's. tracekit why's replay copy has its own wrapper and is never held. Tests: ManyAsks (272 answer orders), the #687 probes 16 trials each, approve-all, approve-then-retry, ToolPauses (own interrupt under and beside asks, executor beside a held pass, RetryPolicy, slow sibling), AnotherProcess (cross-process X1-X3 and a new gate per resume on the real signer), call_state on both signers and across a restart. docs/limits.md replaces the parallel-asks row with what remains. Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com> * LangChain held pass: review fixes for #851 (duplicate ids, cut-off calls, old snapshots) From the independent review of e715878: 1. A message with one call id twice paid twice on one approval (the second call took up the first's consumed plan; the second complete was refused unknown_decision, so the evidence showed one payment). Calls of a held pass that share an id are now refused, undecided, each in a message of its own: what the signer keeps of a call, and so what a pass goes on from, is by id. (Falling back to the per-call gate was tried first: under ainvoke with a slow tool it still paid twice, the duplicate's own pause read as a tool's.) 2. An approved call cut off mid-run in this process (its ainvoke cancelled by asyncio.wait_for, a BaseException out of the tool) ran again on the next resume with no new consume. A call's plan is now kept as `running` while its tool runs; a pass that finds it so has the signer judge it again, which refuses the used approval (TK-APPROVAL-CONSUMED). A call that never started still runs. 3. A store whose last snapshot predates call_state (RPC 24) answered "consumed, no result" for a pay that ran. The start now rebuilds each call's ended/refused from the run's records when a snapshot's calls lack them (pipeline.outcome, shared with replay), as the 1.7 `tool` migration does. 4. A SignerAPI of the app's own without call_state made every resumed single call raise AttributeError; it now decides again, as 1.6 did. Tests (each fails on e715878): a call id twice (invoke/ainvoke x both wirings), a call cut off by asyncio.wait_for and by a BaseException, a snapshot written before call_state, a SignerAPI without call_state. Pins (mutation-checked): a wrapper outside the gate changing a decided call, same_args for an open call, the retry of a failed recorded result, and call_state's consumed attempt check; plus a refused consume then one that goes through. Docs: limits.md rows for the cut-off and duplicate-id refusals, the authorize grant call_state needs (and a SignerAPI without it), and the LangGraph internals the held pass hooks (instrument() warns and leaves ToolNodes ungated when they change); the LangChain integration page. Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com> * LangChain held pass: two resumes of one thread run an approved call once (#851) From the re-review of 8e5f838: two resumes of one thread at the same time ran an approved call twice on one approval, and the second run was silent (1 consume, 1 result; the second complete refused unknown_decision with a client warning only). Main never double-runs. - In one process, a pass takes a call to run by moving its plan to `running` under the gate's lock (_claim); another pass of the node that finds it running (or about to run, from a plan another pass consumed) is refused: "it runs in another pass of its node: the thread was resumed twice at once". A single call's plan stays in memory while it runs. A call cut off mid-run is now marked `cut` in an except BaseException, so a pass tells it from one running in another pass (_resumed). - Across processes, the signer decides which continuation wins: call_claim (contract 24, still unreleased; a default grant) lets one pass of a node go on with a call this attempt consumed and that has no result, once per pause of its tool (the node's answered interrupts), and refuses the second. Claims are run state in signer memory (no record, not snapshotted): after a signer restart a pause may be claimed once more. Without call_claim (an authorize list without it, a SignerAPI of the app's own) the call is refused once the person answers its tool, as in 1.6. - docs/limits.md: "resume a thread from one place at a time", including two resumes that reach approval_consume at the same moment in one process, which can both be refused (approval used, decided_not_completed gap); the grants row names call_claim. Tests (each fails on 8e5f838): two resumes at once after a tool's own pause, in one process (held and single, sync and async, both wirings) and in two (a new gate each, the real signer); a resume that starts while another is about to run the call; call_claim on both signers. Plus the concurrent-consume case (never two payments). Mutations of the local claim, the signer claim and the signer's once-only check each fail them. Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com> * LangChain held pass: a call approved before a signer restart is approved again; a cut claim reads true (#851) From the re-review of 1ad4d10: - A claimed continuation that died mid-tool, then a signer restart, then a new gate's resume ran the pay again ([120, 120], one result in the evidence): claims are signer memory. The signer now remembers where its log stood when its process started, and each approval the seq its approval.consumed took (`consumed_at`, set live and on replay, kept in snapshots; absent in an older one, which counts as before the start). call_claim refuses a call whose approval was consumed before the start: "the signer restarted since this call's approval was used; ask again". A call approved after the restart is claimed as before. No event format change; the `lean:` on claims goes, as no double run remains. - A pass cut off while it claims the call (its ainvoke cancelled during call_claim, a BaseException out of it) left the plan `running`, so the next pass blamed a double resume. The claim now runs outside _gate, inside the wrappers, which mark the plan `cut`; the next pass has the signer judge it (TK-APPROVAL-CONSUMED). - The redundant single-call `running` check in _gate goes: _claim refuses it. The claim also keeps the plan in memory (_start folded in). - rpc_schema: `ERROR = _obj(` spacing. - docs/limits.md: after a signer restart, an approved call whose tool had paused the run itself must be approved again. Tests (each fails on 1ad4d10): a crash after the claim then a signer restart, and a restart while the person answers; a pass cut while it claims (sync and async, both wirings); call_claim refusing a consume from before the start (from the snapshot and the records alone) and allowing one since. Pin: a SignerAPI without call_claim refuses the continuation (mutation-checked, as are the restart rule and the cut while claiming). Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com> * LangChain held pass: claims survive a storage recovery; a pass cuts only a plan it took (#851) From the independent review of 93a34ee: - After a storage error the signer recovers without restarting and rebuilds its run state from the records (_recover -> _replay), which dropped every run's in-memory claims: a pause claimed before the outage could be claimed again and a pay ran twice. _recover now carries the claims across, as it does the provenance indexes. The replayed consume's place in the log (consumed_at) stays: without it a call consumed in this process and first claimed after the outage would be refused as if the signer had restarted; the new test pins both. - A cancelled claim in one pass could mark `cut` a plan another pass had taken and was running, so a third resume ran an allowed call again while the first still ran it. Each pass now carries a token: _claim marks the plan it takes with it, and _cut cuts only that plan (one it hadn't taken yet it then never takes). - _claim's refusal path no longer drops a single call's plan from memory (untested and not needed: a later pass reads the same refusal). - The cut-while-claiming test waits on events rather than a wait_for timeout against a sleep. - docs/limits.md: a storage recovery keeps the claims. Tests (fail on 93a34ee): a claim before a storage outage isn't claimed again after it, and a consume from before it is first claimed after it; three async resumes, the second cancelled while it claims the call the first runs, the third refused. Mutations of the claims carry, of the replayed consumed_at, and of the token check each fail them. Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com> * LangChain held pass: claims survive a recovery that fails partway (#851) From the re-review of 1de86d2: _replay assigns the rebuilt runs before its registry reads and writes. When one of those raised, _recover returned False with the runs rebuilt without their claims, and the next, successful attempt carried over none, so a pause claimed before the outage could be claimed again. _recover now puts the kept claims and provenance indexes back onto the runs in a `finally`, whether or not the replay ended. Tests: TestService.test_call_claim_survives_a_recovery_that_fails_partway (a registry read fails once after the replay; fails on 1de86d2 and with the restore moved back to the success path). ToolPauses. test_a_pass_cancelled_before_it_takes_a_call_never_takes_it pins _claim's check of its pass's cut token: without it, a claim that starts after its pass was cancelled takes the call and leaves it running for good, and the next resume is refused as a second resume. Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com> --------- Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Design only, for review. Nothing is built here, and this lands after 1.7.0. #753 keeps main's behaviour for these cases and describes it in
docs/limits.md. This PR adds onlydesign/langchain-held-pass/: the probes and tests below, which become real tests when the design is built.Problem
A
tracekit_tool_node, or aToolNodethattracekit.instrument()wired, runs every call of one model message in one node task. Three LangGraph behaviours matter:Command(resume=...)values with the task'sinterrupt()calls by the order they are reached, across passes.RetryPolicyretry. It keeps nothing of an earlier pass except that pass's resume values.invoke,asyncio.gatherunderainvoke. So the order in which they reachinterrupt()isn't fixed. When one raises, a sibling still running can be cut off whenasyncio.runends.On main (#687). With several asks in one message:
ainvokeand 4 of 16 underinvoke;TK-APPROVAL-UNBOUND, though it ran once;No asking call runs without an approval.
What #753 tried. Each step fixed one class of case and exposed the next:
interrupt(). A tool that asked the person got{'approval_id': ...}as the answer and ran without asking: fail open.interrupt()itself, the calls that had already run ran again on the next pass.RetryPolicy: a kept exception was re-raised and the tool never retried. Under async sibling cancellation it gave a double run, or an approved call recorded as refused. In another process the memory is gone.tracekit why's fork-replay copy, which then ran a real tool. This was caught bytest_why_langgraph, and never on main.Invariants
Every case in the matrix must hold all six:
interrupt()calls sit at the same positions in every pass of a task, before any tool runs. A tool's owninterrupt()always gets the value the person gave it. The gate never takes or forwards a value meant for a tool.create_agent, which runs each call as its own task, keeps today's behaviour.tracekit why's replay copy, which has its own wrapper, runs no real tool. The pass logic acts only while the node's wrapper is still the gate's.State machine (per node task × tool call id)
Pass rules, for a node with two or more calls that aren't executor calls:
interrupt()once, before any tool runs, if any call is ASKED: freshly asked or remembered, answered or not. That keeps I1.RetryPolicyretry runs an errored call again as a new attempt.asyncio.gather(..., return_exceptions=True)or shielded, so none is cut off mid-run by a sibling's pause.Durable store. This is the decision to make in review.
(a) The signer. A
call_stateRPC, under the run token, answers for a(tool_call_id, attempt):An ENDED call's result comes back from the content vault when the decision sends results (
send_result). Otherwise it comes back as a stated stand-in: "ran; its result is recorded at run_seq N". This needs an RPC contract version bump. The signer's existing guarantees still decide: a T2 executor's consume makes the agent's own resume run nothing.(b) LangGraph's store (
runtime.store), when the app configures one, for the results to give back. Without one, it falls back to (a).(c) Memory in the process. This is what Integrations: MCP big lists and input-required rounds, LangChain parallel asks, tracekit last, gateway listing errors #753 tried. It fails I2 and I3 across processes, so it's rejected.
Open questions:
interrupt()race in LangGraph itself. Should the second be refused and named, or only documented?Test matrix
Dimensions:
create_agent(control),tracekit_tool_node, aToolNodewired byinstrument()invoke,ainvokeinterrupt(), under an ask or beside asks. Raises once underRetryPolicy. Slow while a sibling pauses (async cutoff). Two tools that both pause.SignerServicetracekit whyfork replay of a held nodeEach case asserts, per call:
interrupt()received.The expected values follow from I1–I6.
The inputs are in
design/langchain-held-pass/:test_integration_langchain_at_f40ac8e293.py: Integrations: MCP big lists and input-required rounds, LangChain parallel asks, tracekit last, gateway listing errors #753's tests at f40ac8e, which pass there and fail on main:ManyAsks: 272 cases of approvals answered in varying orders;ToolPauses: a tool's owninterrupt()under and beside asks, 12 variants, plus the executor case.The reviewer's probes then found the cases these miss.
probes/, the reviewer's probes. Each takes the repo root asargv[1]:probe_tool_interrupt.py,probe_tool_interrupt2.py: a tool's own question under an ask, six variants driven to the end;probe_retry.py:RetryPolicyon a held pass with a flaky tool;probe_async_race.py: a slow sibling while a tool pauses, sync against async;probe_cross_process.py: resumed by a new gate on the same checkpointer and the real signer (X1, X2, X3 and a same-process control);probe_executor_and_mixed.py,probe_executor_ask.py: an executor's tool in a pass; pay0 approved, pay1 waiting, plusconfirm.probes/, earlier probes. Each runs with the repo onPYTHONPATH:parallel_ask_ainvoke.py,parallel_ask_invoke.py: the LangChain: run the async tool gate's signer calls in a worker thread (issue #680) #687 probes (<jitter> <trials>);many_asks_one_at_a_time.py(<n> sync|async): main's one-pause-per-resume path;allowed_sibling_runs.py(<n>): an allowed call beside N asks, which runs again on each resume on main;tool_interrupt_eight_shapes.py: builder × one or several calls × mode;fresh_gate_after_tool_pause.py: a fresh gate after a tool's own pause;approve_then_retry.py: a refused call's approval granted, then the model's retry.Each
design/input becomes a test intests/test_integration_langchain.pywhen this is built, and thedocs/limits.mdrow from #753 goes once it holds.