Skip to content

[BUG]: react_agent with use_native_tool_calling runs a tool call whose arguments were cut off #2552

Closed

Description

@simin-yuan

Version

1.9.0 (PyPI); also reproduced on develop at b63f09b.

Which installation method(s) does this occur on?

Pip (1.9.0) and source (develop).

Describe the bug.

With use_native_tool_calling: true, react_agent turns the first native tool call of a reply into an
AgentAction and runs it. On develop the only check before that is finish_reason == "length" (added in #2519;
1.9.0 has no check). A reply cut off another way still runs the tool with partially parsed arguments:

  1. a stream that ends with no finish_reason and no [DONE] (_type: openai and _type: nim);
  2. the Responses API reporting status: "incomplete" (max_output_tokens).

Expected: the tool is not called; the agent raises "LLM output truncated", as it does for
finish_reason: "length" on develop.

Observed: the tool is called with the partial arguments and the workflow finishes normally (Done.).

Data from a local fake OpenAI-compatible server (not a real provider), default settings apart from
use_native_tool_calling: true, 3 runs per row, run one after another; all 3 agreed. The fake model asks
write_file to write a 40-line file; the arguments stop inside line 17.

LLM type / API how the reply ends 1.9.0 develop b63f09b
openai, Chat Completions finish_reason: "length" tool ran, 16 of 40 lines not run, LLM output truncated
openai, Chat Completions stream ends, no finish_reason tool ran, 16 of 40 lines tool ran, 16 of 40 lines
openai, Responses status: incomplete tool ran, 16 of 40 lines tool ran, 16 of 40 lines
openai, Responses stream ends, no response.completed tool ran, 16 of 40 lines tool ran, 16 of 40 lines
nim (ChatNVIDIA 1.5.0), Chat Completions finish_reason: "length" tool ran, 16 of 40 lines not run
nim, Chat Completions stream ends, no finish_reason tool ran, 16 of 40 lines tool ran, 16 of 40 lines
all three TCP reset not run not run

In the control case (complete reply) the tool runs once and writes all 40 lines. The default text-parsing mode
was not measured meaningfully here (the fake server always answers with a native tool call).

Root cause

packages/nvidia_nat_langchain/src/nat/plugins/langchain/agent/react_agent/agent.py on develop: agent_node
checks only response_metadata.get("finish_reason") == "length" (line 314), then passes
output_message.tool_calls[0] to the tool (lines 322-336). The reply comes from _stream_llm, which accumulates
streamed chunks; LangChain builds the tool call from the cut JSON with a partial-JSON parser, so it looks valid.
Same cause as the tool_calling_agent issue #2550.

Minimum reproducible example

One file, no API key, no network; it starts its own fake server.
pip install "nvidia-nat-langchain[openai]==1.9.0", then python repro_react_cutoff_tool_call.py.

Output on develop b63f09b:

full arguments: 320 chars of content; sent: 183 chars of JSON, cut inside line 17
chat-length       tool not run                           raised RuntimeError: LLM output truncated (finish_reason='length'). Truncated output: ''
chat-drop         TOOL RAN with 132 of 320 content chars returned 'Done.'
responses-length  TOOL RAN with 132 of 320 content chars returned 'Done.'

(On 1.9.0 chat-length also runs the tool.)

repro_react_cutoff_tool_call.py
"""Minimal reproduction: NAT react_agent (use_native_tool_calling: true) runs a streamed tool call whose arguments
were cut off.

No API key, no network. A 60-line fake OpenAI-compatible server streams one write_file call whose JSON
arguments stop mid-string, then ends the stream in one of three ways:
  chat-length       Chat Completions, finish_reason="length"
  chat-drop         Chat Completions, body ends with no finish_reason / [DONE]
  responses-length  Responses API, response.incomplete (reason max_output_tokens)
The tool appends to a list; the script prints whether it ran and with how many characters. react_agent sends the
tool result back as text, so the second turn is recognised by the result text and answered in ReAct format.

usage: python repro_react_cutoff_tool_call.py   (needs: pip install "nvidia-nat-langchain[openai]==1.9.0")
"""
import asyncio, json, os, tempfile, threading
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path

os.environ.setdefault("NO_PROXY", "127.0.0.1")
FULL = json.dumps({"path": "report.txt", "content": "".join(f"line {i:02d}\n" for i in range(1, 41))})
CUT = FULL[: FULL.index("line 17") + 4]          # '{"path": "report.txt", "content": "line 01\nline 02 ... line'
RAN: list[int] = []
FINAL = "Thought: I now know the final answer\nFinal Answer: Done."


class H(BaseHTTPRequestHandler):
    protocol_version = "HTTP/1.1"
    def log_message(self, *a): pass

    def sse(self, obj, ev=None):
        self.wfile.write(((f"event: {ev}\n" if ev else "") + "data: " + json.dumps(obj) + "\n\n").encode())

    def do_POST(self):
        mode = self.path.split("/")[1]
        body = self.rfile.read(int(self.headers["Content-Length"]))
        self.send_response(200); self.send_header("Content-Type", "text/event-stream")
        self.send_header("Connection", "close"); self.end_headers()
        if b"Wrote " in body:                                   # second turn: answer and stop
            if mode.startswith("responses"):
                item = {"id": "m", "type": "message", "role": "assistant", "status": "completed",
                        "content": [{"type": "output_text", "text": FINAL, "annotations": []}]}
                self.sse({"type": "response.output_item.added", "output_index": 0, "item": dict(item, status="in_progress", content=[])}, "response.output_item.added")
                self.sse({"type": "response.output_text.delta", "item_id": "m", "output_index": 0, "content_index": 0, "delta": FINAL}, "response.output_text.delta")
                self.sse({"type": "response.output_item.done", "output_index": 0, "item": item}, "response.output_item.done")
                self.sse({"type": "response.completed", "response": {"id": "r2", "object": "response", "status": "completed",
                          "model": "m", "output": [item]}}, "response.completed")
            else:
                self.sse({"id": "c", "object": "chat.completion.chunk", "created": 0, "model": "m",
                          "choices": [{"index": 0, "delta": {"role": "assistant", "content": FINAL}, "finish_reason": "stop"}]})
                self.wfile.write(b"data: [DONE]\n\n")
            return
        if mode.startswith("responses"):
            fc = {"id": "fc", "type": "function_call", "call_id": "call_1", "name": "write_file", "arguments": "", "status": "in_progress"}
            self.sse({"type": "response.created", "response": {"id": "r1", "object": "response", "status": "in_progress", "model": "m", "output": []}}, "response.created")
            self.sse({"type": "response.output_item.added", "output_index": 0, "item": fc}, "response.output_item.added")
            self.sse({"type": "response.function_call_arguments.delta", "item_id": "fc", "output_index": 0, "delta": CUT}, "response.function_call_arguments.delta")
            self.sse({"type": "response.incomplete", "response": {"id": "r1", "object": "response", "status": "incomplete", "model": "m",
                      "incomplete_details": {"reason": "max_output_tokens"}, "output": [dict(fc, arguments=CUT, status="incomplete")]}}, "response.incomplete")
            return
        def chunk(delta, fin=None):
            return {"id": "c", "object": "chat.completion.chunk", "created": 0, "model": "m", "choices": [{"index": 0, "delta": delta, "finish_reason": fin}]}
        self.sse(chunk({"role": "assistant", "tool_calls": [{"index": 0, "id": "call_1", "type": "function",
                                                              "function": {"name": "write_file", "arguments": CUT}}]}))
        if mode == "chat-length":
            self.sse(chunk({}, "length")); self.wfile.write(b"data: [DONE]\n\n")
        # chat-drop: nothing more; the body simply ends


from pydantic import BaseModel
from nat.builder.builder import Builder
from nat.builder.function_info import FunctionInfo
from nat.cli.register_workflow import register_function
from nat.data_models.function import FunctionBaseConfig


class WriteArgs(BaseModel):
    path: str
    content: str


class WriteFileConfig(FunctionBaseConfig, name="repro_write_file"):
    pass


@register_function(config_type=WriteFileConfig)
async def repro_write_file(_c: WriteFileConfig, _b: Builder):
    async def write_file(args: WriteArgs) -> str:
        """Write content to a file."""
        RAN.append(len(args.content))
        return f"Wrote {len(args.content)} characters to {args.path}"
    yield FunctionInfo.from_fn(write_file, description="Write content to a file.")


async def run(port: int, mode: str) -> str:
    from nat.utils import run_workflow
    llm = {"_type": "openai", "base_url": f"http://127.0.0.1:{port}/{mode}/v1", "api_key": "x", "model_name": "m"}
    if mode.startswith("responses"):
        llm["api_type"] = "responses"
    cfg = {"functions": {"write_file": {"_type": "repro_write_file"}}, "llms": {"m": llm},
           "workflow": {"_type": "react_agent", "llm_name": "m", "tool_names": ["write_file"],
                        "use_native_tool_calling": True}}
    f = Path(tempfile.mkdtemp()) / "c.yml"; f.write_text(json.dumps(cfg))
    return await run_workflow(config_file=f, prompt="Write the 40-line report to report.txt.")


if __name__ == "__main__":
    srv = ThreadingHTTPServer(("127.0.0.1", 0), H)
    threading.Thread(target=srv.serve_forever, daemon=True).start()
    print(f"full arguments: {len(json.loads(FULL)['content'])} chars of content; sent: {len(CUT)} chars of JSON, cut inside line 17")
    for mode in ("chat-length", "chat-drop", "responses-length"):
        RAN.clear()
        try:
            out = asyncio.run(run(srv.server_address[1], mode))
            res = f"returned {out!r}"
        except Exception as e:  # noqa: BLE001
            res = f"raised {type(e).__name__}: {str(e)[:70]}"
        ran = f"TOOL RAN with {RAN[0]} of 320 content chars" if RAN else "tool not run"
        print(f"{mode:17s} {ran:38s} {res}")

Other/Misc.

Related: #2550 / #2551 (tool_calling_agent, same gap), #2518 / #2519.
Found with my cut-off benchmark: https://github.com/simin-yuan/agent-cutoff-bench

Activity

  1. simin-yuan commented on Nov 3, 2026

    @simin-yuan
    ContributorAuthor

    I have a fix with tests ready; opening a PR now.

  2. github-actions commented on Nov 3, 2026

    @github-actions

    Warning

    ⚠️ Deprecation Notice: NVIDIA NeMo Agent Toolkit has been deprecated as of October 2026 and will reach end of life (EOL) by end of year 2026. Depending on your needs, migrate to: NeMo Relay for observability, middleware, and governance. NeMo Helix for a unified CLI with evaluation, optimization, and hardening capabilities. NeMo Fabric for a consistent interface to run agent harnesses and custom agents.
    We recommend completing your migration before the EOL date.

    Issue policy: Only bugs should be filed. Feature requests will be closed.

  3. added theissue type on Nov 3, 2026
  4. added a commit that references this issue on Nov 4, 2026
    16f1116
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions