Skip to content

Generator-backed tool input examples disappear from follow-up requests #1990

Description

@sylvesterkaczmarek

beta_tool and beta_async_tool accept input_examples as Iterable[dict[str, object]], but BaseFunctionTool retains a supplied iterator directly. The first request consumes it during body serialization. Later requests from the same tool runner still carry that exhausted iterator and send input_examples: [] instead of the configured examples.

examples = ({"city": city} for city in ["Kraków", "London"])
tool = beta_tool(lookup, input_examples=examples)
# In a two-turn tool_runner run, the first request includes both examples;
# the follow-up request sends an empty input_examples list.

This silently changes the tool definition during a conversation and loses the examples intended for subsequent model calls. Lists and tuples do not exhibit the problem.

Materialize the iterable once when constructing the function tool, retaining its values for repeated serialization. Empty iterables and omitted examples should keep their respective empty/omitted behavior.

A public-client matrix reproduces the issue across sync/async and JSON/streaming runners, with generator and list-iterator inputs plus list/tuple/empty/omitted controls. Eight cases fail on unchanged main. The change is separate from #1838, which prevents telemetry from consuming the outer messages/tools iterables, and #1953, which concerns ToolError content.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions