Repository navigation
Description
Version
1.9.0 (PyPI); also reproduced on develop at b63f09b.
Which installation method(s) does this occur on?
Pip (1.9.0) and source (develop).
Describe the bug.
With use_native_tool_calling: true, react_agent turns the first native tool call of a reply into an
AgentAction and runs it. On develop the only check before that is finish_reason == "length" (added in #2519;
1.9.0 has no check). A reply cut off another way still runs the tool with partially parsed arguments:
- a stream that ends with no
finish_reasonand no[DONE](_type: openaiand_type: nim); - the Responses API reporting
status: "incomplete"(max_output_tokens).
Expected: the tool is not called; the agent raises "LLM output truncated", as it does for
finish_reason: "length" on develop.
Observed: the tool is called with the partial arguments and the workflow finishes normally (Done.).
Data from a local fake OpenAI-compatible server (not a real provider), default settings apart from
use_native_tool_calling: true, 3 runs per row, run one after another; all 3 agreed. The fake model asks
write_file to write a 40-line file; the arguments stop inside line 17.
| LLM type / API | how the reply ends | 1.9.0 | develop b63f09b |
|---|---|---|---|
openai, Chat Completions |
finish_reason: "length" |
tool ran, 16 of 40 lines | not run, LLM output truncated |
openai, Chat Completions |
stream ends, no finish_reason |
tool ran, 16 of 40 lines | tool ran, 16 of 40 lines |
openai, Responses |
status: incomplete |
tool ran, 16 of 40 lines | tool ran, 16 of 40 lines |
openai, Responses |
stream ends, no response.completed |
tool ran, 16 of 40 lines | tool ran, 16 of 40 lines |
nim (ChatNVIDIA 1.5.0), Chat Completions |
finish_reason: "length" |
tool ran, 16 of 40 lines | not run |
nim, Chat Completions |
stream ends, no finish_reason |
tool ran, 16 of 40 lines | tool ran, 16 of 40 lines |
| all three | TCP reset | not run | not run |
In the control case (complete reply) the tool runs once and writes all 40 lines. The default text-parsing mode
was not measured meaningfully here (the fake server always answers with a native tool call).
Root cause
packages/nvidia_nat_langchain/src/nat/plugins/langchain/agent/react_agent/agent.py on develop: agent_node
checks only response_metadata.get("finish_reason") == "length" (line 314), then passes
output_message.tool_calls[0] to the tool (lines 322-336). The reply comes from _stream_llm, which accumulates
streamed chunks; LangChain builds the tool call from the cut JSON with a partial-JSON parser, so it looks valid.
Same cause as the tool_calling_agent issue #2550.
Minimum reproducible example
One file, no API key, no network; it starts its own fake server.
pip install "nvidia-nat-langchain[openai]==1.9.0", then python repro_react_cutoff_tool_call.py.
Output on develop b63f09b:
full arguments: 320 chars of content; sent: 183 chars of JSON, cut inside line 17
chat-length tool not run raised RuntimeError: LLM output truncated (finish_reason='length'). Truncated output: ''
chat-drop TOOL RAN with 132 of 320 content chars returned 'Done.'
responses-length TOOL RAN with 132 of 320 content chars returned 'Done.'
(On 1.9.0 chat-length also runs the tool.)
repro_react_cutoff_tool_call.py
"""Minimal reproduction: NAT react_agent (use_native_tool_calling: true) runs a streamed tool call whose arguments
were cut off.
No API key, no network. A 60-line fake OpenAI-compatible server streams one write_file call whose JSON
arguments stop mid-string, then ends the stream in one of three ways:
chat-length Chat Completions, finish_reason="length"
chat-drop Chat Completions, body ends with no finish_reason / [DONE]
responses-length Responses API, response.incomplete (reason max_output_tokens)
The tool appends to a list; the script prints whether it ran and with how many characters. react_agent sends the
tool result back as text, so the second turn is recognised by the result text and answered in ReAct format.
usage: python repro_react_cutoff_tool_call.py (needs: pip install "nvidia-nat-langchain[openai]==1.9.0")
"""
import asyncio, json, os, tempfile, threading
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
os.environ.setdefault("NO_PROXY", "127.0.0.1")
FULL = json.dumps({"path": "report.txt", "content": "".join(f"line {i:02d}\n" for i in range(1, 41))})
CUT = FULL[: FULL.index("line 17") + 4] # '{"path": "report.txt", "content": "line 01\nline 02 ... line'
RAN: list[int] = []
FINAL = "Thought: I now know the final answer\nFinal Answer: Done."
class H(BaseHTTPRequestHandler):
protocol_version = "HTTP/1.1"
def log_message(self, *a): pass
def sse(self, obj, ev=None):
self.wfile.write(((f"event: {ev}\n" if ev else "") + "data: " + json.dumps(obj) + "\n\n").encode())
def do_POST(self):
mode = self.path.split("/")[1]
body = self.rfile.read(int(self.headers["Content-Length"]))
self.send_response(200); self.send_header("Content-Type", "text/event-stream")
self.send_header("Connection", "close"); self.end_headers()
if b"Wrote " in body: # second turn: answer and stop
if mode.startswith("responses"):
item = {"id": "m", "type": "message", "role": "assistant", "status": "completed",
"content": [{"type": "output_text", "text": FINAL, "annotations": []}]}
self.sse({"type": "response.output_item.added", "output_index": 0, "item": dict(item, status="in_progress", content=[])}, "response.output_item.added")
self.sse({"type": "response.output_text.delta", "item_id": "m", "output_index": 0, "content_index": 0, "delta": FINAL}, "response.output_text.delta")
self.sse({"type": "response.output_item.done", "output_index": 0, "item": item}, "response.output_item.done")
self.sse({"type": "response.completed", "response": {"id": "r2", "object": "response", "status": "completed",
"model": "m", "output": [item]}}, "response.completed")
else:
self.sse({"id": "c", "object": "chat.completion.chunk", "created": 0, "model": "m",
"choices": [{"index": 0, "delta": {"role": "assistant", "content": FINAL}, "finish_reason": "stop"}]})
self.wfile.write(b"data: [DONE]\n\n")
return
if mode.startswith("responses"):
fc = {"id": "fc", "type": "function_call", "call_id": "call_1", "name": "write_file", "arguments": "", "status": "in_progress"}
self.sse({"type": "response.created", "response": {"id": "r1", "object": "response", "status": "in_progress", "model": "m", "output": []}}, "response.created")
self.sse({"type": "response.output_item.added", "output_index": 0, "item": fc}, "response.output_item.added")
self.sse({"type": "response.function_call_arguments.delta", "item_id": "fc", "output_index": 0, "delta": CUT}, "response.function_call_arguments.delta")
self.sse({"type": "response.incomplete", "response": {"id": "r1", "object": "response", "status": "incomplete", "model": "m",
"incomplete_details": {"reason": "max_output_tokens"}, "output": [dict(fc, arguments=CUT, status="incomplete")]}}, "response.incomplete")
return
def chunk(delta, fin=None):
return {"id": "c", "object": "chat.completion.chunk", "created": 0, "model": "m", "choices": [{"index": 0, "delta": delta, "finish_reason": fin}]}
self.sse(chunk({"role": "assistant", "tool_calls": [{"index": 0, "id": "call_1", "type": "function",
"function": {"name": "write_file", "arguments": CUT}}]}))
if mode == "chat-length":
self.sse(chunk({}, "length")); self.wfile.write(b"data: [DONE]\n\n")
# chat-drop: nothing more; the body simply ends
from pydantic import BaseModel
from nat.builder.builder import Builder
from nat.builder.function_info import FunctionInfo
from nat.cli.register_workflow import register_function
from nat.data_models.function import FunctionBaseConfig
class WriteArgs(BaseModel):
path: str
content: str
class WriteFileConfig(FunctionBaseConfig, name="repro_write_file"):
pass
@register_function(config_type=WriteFileConfig)
async def repro_write_file(_c: WriteFileConfig, _b: Builder):
async def write_file(args: WriteArgs) -> str:
"""Write content to a file."""
RAN.append(len(args.content))
return f"Wrote {len(args.content)} characters to {args.path}"
yield FunctionInfo.from_fn(write_file, description="Write content to a file.")
async def run(port: int, mode: str) -> str:
from nat.utils import run_workflow
llm = {"_type": "openai", "base_url": f"http://127.0.0.1:{port}/{mode}/v1", "api_key": "x", "model_name": "m"}
if mode.startswith("responses"):
llm["api_type"] = "responses"
cfg = {"functions": {"write_file": {"_type": "repro_write_file"}}, "llms": {"m": llm},
"workflow": {"_type": "react_agent", "llm_name": "m", "tool_names": ["write_file"],
"use_native_tool_calling": True}}
f = Path(tempfile.mkdtemp()) / "c.yml"; f.write_text(json.dumps(cfg))
return await run_workflow(config_file=f, prompt="Write the 40-line report to report.txt.")
if __name__ == "__main__":
srv = ThreadingHTTPServer(("127.0.0.1", 0), H)
threading.Thread(target=srv.serve_forever, daemon=True).start()
print(f"full arguments: {len(json.loads(FULL)['content'])} chars of content; sent: {len(CUT)} chars of JSON, cut inside line 17")
for mode in ("chat-length", "chat-drop", "responses-length"):
RAN.clear()
try:
out = asyncio.run(run(srv.server_address[1], mode))
res = f"returned {out!r}"
except Exception as e: # noqa: BLE001
res = f"raised {type(e).__name__}: {str(e)[:70]}"
ran = f"TOOL RAN with {RAN[0]} of 320 content chars" if RAN else "tool not run"
print(f"{mode:17s} {ran:38s} {res}")Other/Misc.
Related: #2550 / #2551 (tool_calling_agent, same gap), #2518 / #2519.
Found with my cut-off benchmark: https://github.com/simin-yuan/agent-cutoff-bench
Activity
simin-yuan commentedon Nov 3, 2026
I have a fix with tests ready; opening a PR now.
github-actions commentedon Nov 3, 2026
Warning
We recommend completing your migration before the EOL date.
Issue policy: Only bugs should be filed. Feature requests will be closed.
fix(react_agent): do not run native tool calls from a cut-off reply (N…