Skip to content

Design: LangChain held ToolNode pass for several calls of one message (#687), after 1.7.0 - #756

Draft
LoopGlitch26 wants to merge 1 commit into
mainfrom
agent/langchain-held-pass
Draft

LoopGlitch26 wants to merge 1 commit into
mainfrom
agent/langchain-held-pass

Conversation

@LoopGlitch26

Copy link
Copy Markdown
Member

Design only, for review. Nothing is built here, and this lands after 1.7.0. #753 keeps main's behaviour for these cases and describes it in docs/limits.md. This PR adds only design/langchain-held-pass/: the probes and tests below, which become real tests when the design is built.

Problem

A tracekit_tool_node, or a ToolNode that tracekit.instrument() wired, runs every call of one model message in one node task. Three LangGraph behaviours matter:

  1. LangGraph pairs Command(resume=...) values with the task's interrupt() calls by the order they are reached, across passes.
  2. It runs the node again from its start on each resume and on each RetryPolicy retry. It keeps nothing of an earlier pass except that pass's resume values.
  3. The calls run concurrently: threads under invoke, asyncio.gather under ainvoke. So the order in which they reach interrupt() isn't fixed. When one raises, a sibling still running can be cut off when asyncio.run ends.

On main (#687). With several asks in one message:

  • each resume answers one pause, and the run pauses again;
  • the approved call doesn't always get the resume value: it ran 7 of 16 times under ainvoke and 4 of 16 under invoke;
  • a call that ran on an earlier resume is gated again on the next one: it gets a new approval request and is denied TK-APPROVAL-UNBOUND, though it ran once;
  • an allowed call beside an ask runs again on each resume.

No asking call runs without an approval.

What #753 tried. Each step fixed one class of case and exposed the next:

  • Skip the gate's pause once an approval is answered. That moved resume values onto the tool's own interrupt(). A tool that asked the person got {'approval_id': ...} as the answer and ran without asking: fail open.
  • Decide all of a pass's calls first and pause once. That fixed the ordering. But when a tool then called interrupt() itself, the calls that had already run ran again on the next pass.
  • Give back calls that already ended, from memory. That broke under RetryPolicy: a kept exception was re-raised and the tool never retried. Under async sibling cancellation it gave a double run, or an approved call recorded as refused. In another process the memory is gone.
  • Hook the node functions on the instance. That leaked into tracekit why's fork-replay copy, which then ran a real tool. This was caught by test_why_langgraph, and never on main.

Invariants

Every case in the matrix must hold all six:

  • I1 Pairing. Tracekit's interrupt() calls sit at the same positions in every pass of a task, before any tool runs. A tool's own interrupt() always gets the value the person gave it. The gate never takes or forwards a value meant for a tool.
  • I2 At most once. A call runs at most once per consumed approval, or per decision for an allowed call. This holds across passes, retries, cancellations and processes.
  • I3 Truthful outcome. The model reads, and the signer records, what happened to each call: its result if it ran, a refusal only if it didn't. Each execution has one consumed decision and one completion. A refusal is recorded once.
  • I4 Fail closed. When a call's state can't be established, it doesn't run. That covers state lost in another process, and a signer that can't be reached for a call that asks.
  • I5 No new trust. Nothing the agent's process keeps decides whether a call may run. The signer does.
  • I6 Nothing else changes.
    • create_agent, which runs each call as its own task, keeps today's behaviour.
    • An executor's tool stays the executor's to decide.
    • A single call keeps main's per-call gate.
    • tracekit why's replay copy, which has its own wrapper, runs no real tool. The pass logic acts only while the node's wrapper is still the gate's.

State machine (per node task × tool call id)

NEW ──decide──▶ DENIED (final: refusal)
             ├▶ ALLOWED ─────────────────────────────┐
             └▶ ASKED(approval) ──pass pauses once──▶ on resume:
                    APPROVED ────────────────────────┤
                    REQUESTED / REJECTED / EXPIRED ─▶ REFUSED (final)
                                                     ▼
                                     CONSUMED ──▶ RUNNING ──▶ ENDED(result | error)   (final per attempt)
                                                     │  ▲
                                   tool's interrupt()▼  │resume (no second consume)
                                                PAUSED_BY_TOOL
                         RUNNING ──cut off (process exit)──▶ UNKNOWN: recorded error "cancelled",
                                                              its approval counted as used

Pass rules, for a node with two or more calls that aren't executor calls:

  • P1. First establish every call's state from the durable store. Then call interrupt() once, before any tool runs, if any call is ASKED: freshly asked or remembered, answered or not. That keeps I1.
  • P2. Run tools only for CONSUMED and PAUSED_BY_TOOL calls. An ENDED call returns its kept outcome, a result or a refusal, never a kept exception. A RetryPolicy retry runs an errored call again as a new attempt.
  • P3. Await every sibling before the pass passes on a tool's interrupt or an error. The node runs its calls itself, under asyncio.gather(..., return_exceptions=True) or shielded, so none is cut off mid-run by a sibling's pause.
  • P4. Write a call's state change (CONSUMED, PAUSED_BY_TOOL, ENDED) to the durable store before the next step can lose it.

Durable store. This is the decision to make in review.

  • (a) The signer. A call_state RPC, under the run token, answers for a (tool_call_id, attempt):

    • the decision;
    • the approval's state;
    • whether this attempt consumed it;
    • whether a result is recorded.

    An ENDED call's result comes back from the content vault when the decision sends results (send_result). Otherwise it comes back as a stated stand-in: "ran; its result is recorded at run_seq N". This needs an RPC contract version bump. The signer's existing guarantees still decide: a T2 executor's consume makes the agent's own resume run nothing.

  • (b) LangGraph's store (runtime.store), when the app configures one, for the results to give back. Without one, it falls back to (a).

  • (c) Memory in the process. This is what Integrations: MCP big lists and input-required rounds, LangChain parallel asks, tracekit last, gateway listing errors #753 tried. It fails I2 and I3 across processes, so it's rejected.

Open questions:

  • Should a retry of an ASKED call after an error ask again, or reuse the approval through approve-then-retry?
  • Two tools in one message that both call interrupt() race in LangGraph itself. Should the second be refused and named, or only documented?

Test matrix

Dimensions:

Dimension Values
Builder create_agent (control), tracekit_tool_node, a ToolNode wired by instrument()
Mode invoke, ainvoke
Calls in one message 1 ask; 2, 3 and 5 asks beside one allowed and one denied call; an executor's tool among them
Answers Each ask approved, rejected or left waiting: every combination for up to 3 asks, approve-or-wait for 5. Answered in every order, before or after a resume.
Tools Plain. Its own interrupt(), under an ask or beside asks. Raises once under RetryPolicy. Slow while a sibling pauses (async cutoff). Two tools that both pause.
Process Same process, or resumed in another process: a new gate and graph on the same checkpointer, with the real SignerService
Replay tracekit why fork replay of a held node

Each case asserts, per call:

  • how many times it ran;
  • what the model read;
  • the results, refusals and approval requests recorded;
  • every pause in order, and what each tool's interrupt() received.

The expected values follow from I1–I6.

The inputs are in design/langchain-held-pass/:

  • test_integration_langchain_at_f40ac8e293.py: Integrations: MCP big lists and input-required rounds, LangChain parallel asks, tracekit last, gateway listing errors #753's tests at f40ac8e, which pass there and fail on main:

    • ManyAsks: 272 cases of approvals answered in varying orders;
    • ToolPauses: a tool's own interrupt() under and beside asks, 12 variants, plus the executor case.

    The reviewer's probes then found the cases these miss.

  • probes/, the reviewer's probes. Each takes the repo root as argv[1]:

    • probe_tool_interrupt.py, probe_tool_interrupt2.py: a tool's own question under an ask, six variants driven to the end;
    • probe_retry.py: RetryPolicy on a held pass with a flaky tool;
    • probe_async_race.py: a slow sibling while a tool pauses, sync against async;
    • probe_cross_process.py: resumed by a new gate on the same checkpointer and the real signer (X1, X2, X3 and a same-process control);
    • probe_executor_and_mixed.py, probe_executor_ask.py: an executor's tool in a pass; pay0 approved, pay1 waiting, plus confirm.
  • probes/, earlier probes. Each runs with the repo on PYTHONPATH:

    • parallel_ask_ainvoke.py, parallel_ask_invoke.py: the LangChain: run the async tool gate's signer calls in a worker thread (issue #680) #687 probes (<jitter> <trials>);
    • many_asks_one_at_a_time.py (<n> sync|async): main's one-pause-per-resume path;
    • allowed_sibling_runs.py (<n>): an allowed call beside N asks, which runs again on each resume on main;
    • tool_interrupt_eight_shapes.py: builder × one or several calls × mode;
    • fresh_gate_after_tool_pause.py: a fresh gate after a tool's own pause;
    • approve_then_retry.py: a refused call's approval granted, then the model's retry.

Each design/ input becomes a test in tests/test_integration_langchain.py when this is built, and the docs/limits.md row from #753 goes once it holds.

… tests, no code

The design is the draft PR's description. These files are the cases it must hold: the 1.7.0 reviews' probes and the
tests from #753 at f40ac8e. They become tests in tests/ when the design is built.
LoopGlitch26 added a commit that referenced this pull request Nov 15, 2026
* LangChain: several asks in one message, each answered once (#687)

Builds the design in #756. A tracekit_tool_node, or a ToolNode that
tracekit.instrument() wired, now plans the calls of one model message
together (the held pass): each call's state is established first (what
this process did, else what the signer recorded, else decided now), the
pass pauses once, before any tool runs, when any call asked, answered or
not, then each call allowed or approved is consumed under its own
approval and one refused is final. A call that ended is given back, not
decided, consumed or run again. A tool's own interrupt() always comes
after the gate's pause, so it gets the person's answer; a call whose tool
paused after its consume goes on under that one consume. A RetryPolicy
retry runs only the call that raised, as a new attempt. Under ainvoke a
call that raises waits for its siblings, so none is cut off mid-run.

The durable store is the signer (design option a): call_state, RPC
contract 24 (RPC_MIN stays 15; 1.6.0 shipped 22 and main is already at
23), a read under the run token: the latest attempt's decision, whether
the call's arguments are the decided ones, its approval and whether this
attempt consumed it, a refused consume, and its tool.result. The signer
keeps refused/ended on the call's state, rebuilt from the records on
replay and kept in snapshots. A resumed node in another process reads it;
an ended call comes back to the model as "ran; its result is recorded at
run_seq N". A call consumed and not ended goes on only when a tool's own
pause was answered since; otherwise the signer refuses it again
(TK-APPROVAL-CONSUMED), as main does for a T2 executor's consume.

A single call (create_agent's tasks, a ToolNode with one call) keeps the
per-call gate; only a call whose tool paused after its consume goes on.
An executor's tool stays the executor's. tracekit why's replay copy has
its own wrapper and is never held.

Tests: ManyAsks (272 answer orders), the #687 probes 16 trials each,
approve-all, approve-then-retry, ToolPauses (own interrupt under and
beside asks, executor beside a held pass, RetryPolicy, slow sibling),
AnotherProcess (cross-process X1-X3 and a new gate per resume on the real
signer), call_state on both signers and across a restart. docs/limits.md
replaces the parallel-asks row with what remains.

Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com>

* LangChain held pass: review fixes for #851 (duplicate ids, cut-off calls, old snapshots)

From the independent review of e715878:

1. A message with one call id twice paid twice on one approval (the
   second call took up the first's consumed plan; the second complete was
   refused unknown_decision, so the evidence showed one payment). Calls
   of a held pass that share an id are now refused, undecided, each in a
   message of its own: what the signer keeps of a call, and so what a
   pass goes on from, is by id. (Falling back to the per-call gate was
   tried first: under ainvoke with a slow tool it still paid twice, the
   duplicate's own pause read as a tool's.)
2. An approved call cut off mid-run in this process (its ainvoke
   cancelled by asyncio.wait_for, a BaseException out of the tool) ran
   again on the next resume with no new consume. A call's plan is now
   kept as `running` while its tool runs; a pass that finds it so has the
   signer judge it again, which refuses the used approval
   (TK-APPROVAL-CONSUMED). A call that never started still runs.
3. A store whose last snapshot predates call_state (RPC 24) answered
   "consumed, no result" for a pay that ran. The start now rebuilds each
   call's ended/refused from the run's records when a snapshot's calls
   lack them (pipeline.outcome, shared with replay), as the 1.7 `tool`
   migration does.
4. A SignerAPI of the app's own without call_state made every resumed
   single call raise AttributeError; it now decides again, as 1.6 did.

Tests (each fails on e715878): a call id twice (invoke/ainvoke x both
wirings), a call cut off by asyncio.wait_for and by a BaseException, a
snapshot written before call_state, a SignerAPI without call_state. Pins
(mutation-checked): a wrapper outside the gate changing a decided call,
same_args for an open call, the retry of a failed recorded result, and
call_state's consumed attempt check; plus a refused consume then one
that goes through.

Docs: limits.md rows for the cut-off and duplicate-id refusals, the
authorize grant call_state needs (and a SignerAPI without it), and the
LangGraph internals the held pass hooks (instrument() warns and leaves
ToolNodes ungated when they change); the LangChain integration page.

Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com>

* LangChain held pass: two resumes of one thread run an approved call once (#851)

From the re-review of 8e5f838: two resumes of one thread at the same time
ran an approved call twice on one approval, and the second run was silent
(1 consume, 1 result; the second complete refused unknown_decision with a
client warning only). Main never double-runs.

- In one process, a pass takes a call to run by moving its plan to
  `running` under the gate's lock (_claim); another pass of the node that
  finds it running (or about to run, from a plan another pass consumed) is
  refused: "it runs in another pass of its node: the thread was resumed
  twice at once". A single call's plan stays in memory while it runs. A
  call cut off mid-run is now marked `cut` in an except BaseException, so
  a pass tells it from one running in another pass (_resumed).
- Across processes, the signer decides which continuation wins:
  call_claim (contract 24, still unreleased; a default grant) lets one pass
  of a node go on with a call this attempt consumed and that has no result,
  once per pause of its tool (the node's answered interrupts), and refuses
  the second. Claims are run state in signer memory (no record, not
  snapshotted): after a signer restart a pause may be claimed once more.
  Without call_claim (an authorize list without it, a SignerAPI of the
  app's own) the call is refused once the person answers its tool, as in
  1.6.
- docs/limits.md: "resume a thread from one place at a time", including
  two resumes that reach approval_consume at the same moment in one
  process, which can both be refused (approval used, decided_not_completed
  gap); the grants row names call_claim.

Tests (each fails on 8e5f838): two resumes at once after a tool's own
pause, in one process (held and single, sync and async, both wirings) and
in two (a new gate each, the real signer); a resume that starts while
another is about to run the call; call_claim on both signers. Plus the
concurrent-consume case (never two payments). Mutations of the local
claim, the signer claim and the signer's once-only check each fail them.

Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com>

* LangChain held pass: a call approved before a signer restart is approved again; a cut claim reads true (#851)

From the re-review of 1ad4d10:

- A claimed continuation that died mid-tool, then a signer restart, then
  a new gate's resume ran the pay again ([120, 120], one result in the
  evidence): claims are signer memory. The signer now remembers where its
  log stood when its process started, and each approval the seq its
  approval.consumed took (`consumed_at`, set live and on replay, kept in
  snapshots; absent in an older one, which counts as before the start).
  call_claim refuses a call whose approval was consumed before the start:
  "the signer restarted since this call's approval was used; ask again".
  A call approved after the restart is claimed as before. No event
  format change; the `lean:` on claims goes, as no double run remains.
- A pass cut off while it claims the call (its ainvoke cancelled during
  call_claim, a BaseException out of it) left the plan `running`, so the
  next pass blamed a double resume. The claim now runs outside _gate,
  inside the wrappers, which mark the plan `cut`; the next pass has the
  signer judge it (TK-APPROVAL-CONSUMED).
- The redundant single-call `running` check in _gate goes: _claim refuses
  it. The claim also keeps the plan in memory (_start folded in).
- rpc_schema: `ERROR = _obj(` spacing.
- docs/limits.md: after a signer restart, an approved call whose tool had
  paused the run itself must be approved again.

Tests (each fails on 1ad4d10): a crash after the claim then a signer
restart, and a restart while the person answers; a pass cut while it
claims (sync and async, both wirings); call_claim refusing a consume from
before the start (from the snapshot and the records alone) and allowing
one since. Pin: a SignerAPI without call_claim refuses the continuation
(mutation-checked, as are the restart rule and the cut while claiming).

Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com>

* LangChain held pass: claims survive a storage recovery; a pass cuts only a plan it took (#851)

From the independent review of 93a34ee:

- After a storage error the signer recovers without restarting and
  rebuilds its run state from the records (_recover -> _replay), which
  dropped every run's in-memory claims: a pause claimed before the outage
  could be claimed again and a pay ran twice. _recover now carries the
  claims across, as it does the provenance indexes. The replayed
  consume's place in the log (consumed_at) stays: without it a call
  consumed in this process and first claimed after the outage would be
  refused as if the signer had restarted; the new test pins both.
- A cancelled claim in one pass could mark `cut` a plan another pass had
  taken and was running, so a third resume ran an allowed call again
  while the first still ran it. Each pass now carries a token: _claim
  marks the plan it takes with it, and _cut cuts only that plan (one it
  hadn't taken yet it then never takes).
- _claim's refusal path no longer drops a single call's plan from memory
  (untested and not needed: a later pass reads the same refusal).
- The cut-while-claiming test waits on events rather than a wait_for
  timeout against a sleep.
- docs/limits.md: a storage recovery keeps the claims.

Tests (fail on 93a34ee): a claim before a storage outage isn't claimed
again after it, and a consume from before it is first claimed after it;
three async resumes, the second cancelled while it claims the call the
first runs, the third refused. Mutations of the claims carry, of the
replayed consumed_at, and of the token check each fail them.

Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com>

* LangChain held pass: claims survive a recovery that fails partway (#851)

From the re-review of 1de86d2: _replay assigns the rebuilt runs before
its registry reads and writes. When one of those raised, _recover returned
False with the runs rebuilt without their claims, and the next, successful
attempt carried over none, so a pause claimed before the outage could be
claimed again. _recover now puts the kept claims and provenance indexes
back onto the runs in a `finally`, whether or not the replay ended.

Tests: TestService.test_call_claim_survives_a_recovery_that_fails_partway
(a registry read fails once after the replay; fails on 1de86d2 and with
the restore moved back to the success path). ToolPauses.
test_a_pass_cancelled_before_it_takes_a_call_never_takes_it pins _claim's
check of its pass's cut token: without it, a claim that starts after its
pass was cancelled takes the call and leaves it running for good, and the
next resume is refused as a second resume.

Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com>

---------

Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant