Skip to content

feat(telemetry): record gen_ai.tool.definitions on the chat span (typescript) - #6162

Draft
abhiramshaji-cyber wants to merge 2 commits into
strands-agents:mainfrom
abhiramshaji-cyber:feat/ts-chat-span-tool-definitions
Draft

abhiramshaji-cyber wants to merge 2 commits into
strands-agents:mainfrom
abhiramshaji-cyber:feat/ts-chat-span-tool-definitions

Conversation

@abhiramshaji-cyber

Copy link
Copy Markdown

Human Overview

Description

Hey @zhoucy-syn, hope you're doing well! It's me again from #5885. Your replay use case there is what this is built around, so thank you for spelling it out.

With gen_ai_tool_definitions in OTEL_SEMCONV_STABILITY_OPT_IN, each TypeScript chat span now records the tools that exact model call received, so a recorded call can be replayed from the span that owns it.

What this code is for

_invokeModelWithMiddleware in agent.ts runs every model call. It builds an InvokeModelContext (messages, system prompt, toolSpecs), lets InvokeModelStage middleware change it, then opens the chat span with startModelInvokeSpan and streams from the model with ctx.toolSpecs.

Before this PR

A developer runs an agent with two tools and the opt in set, and a middleware narrows the tools to one for this call.

  1. _invokeModelWithMiddleware builds ctx.toolSpecs with both tools; the middleware leaves one.
  2. startModelInvokeSpan is called with messages, modelId and systemPrompt only.
  3. model.stream receives the one tool in streamOptions.toolSpecs.
  4. The chat span has no gen_ai.tool.definitions, so the trace cannot say which tools this call saw.

After this PR

Same developer, same agent, same middleware.

  1. Same as above through step 1.
  2. startModelInvokeSpan also gets toolSpecs: ctx.toolSpecs, the same array model.stream gets.
  3. With the opt in set and at least one tool, the span gets gen_ai.tool.definitions as [{"type":"function","name":"kept","description":"...","parameters":{...}}].
  4. With no tools, or without the opt in, the span is unchanged.
flowchart LR

classDef box fill:#eef1ff,stroke:#4c5bd4,color:#111827
classDef bad fill:#ffd6d6,stroke:#cc0000,color:#111827,stroke-width:3px
classDef good fill:#c8f0c8,stroke:#008800,color:#111827,stroke-width:3px

subgraph BEFORE["BEFORE: 0 tool definitions on chat"]
direction TB
  B1["agent.invoke or agent.stream"] --> B2["InvokeModelStage middleware sets ctx.toolSpecs"]
  B2 --> B3["startModelInvokeSpan(messages, modelId, systemPrompt)"]
  B3 --> B4{"toolSpecs passed?"}
  B4 -->|"never"| B5["chat span: no gen_ai.tool.definitions"]
  B4 -->|"model call itself"| B6["model.stream gets ctx.toolSpecs"]
end

subgraph AFTER["AFTER: the tools this call received"]
direction TB
  A1["agent.invoke or agent.stream"] --> A2["InvokeModelStage middleware sets ctx.toolSpecs"]
  A2 --> A3["startModelInvokeSpan(..., toolSpecs: ctx.toolSpecs)"]
  A3 --> A4{"THE CHANGE: opt in set and toolSpecs.length"}
  A4 -->|"yes"| A5["chat span: gen_ai.tool.definitions, semconv shape"]
  A4 -->|"no tools or no opt in"| A6["chat span unchanged, model.stream as before"]
end

B1 ~~~ A1

class B1,B2,B3,B6 box
class A1,A2,A3,A6 box
class B4,B5 bad
class A4,A5 good

linkStyle 3 stroke:#cc0000,stroke-width:5px
linkStyle 8 stroke:#008800,stroke-width:5px

style BEFORE fill:#fff5f5,stroke:#cc0000,color:#111827
style AFTER fill:#f3fff3,stroke:#008800,color:#111827
Loading

The change

  • telemetry/types.ts: StartModelInvokeSpanOptions gets an optional toolSpecs?: readonly ToolSpec[].
  • agent/agent.ts: the startModelInvokeSpan call passes toolSpecs: ctx.toolSpecs, read after middleware, so it matches what the model gets (including the structured output tool when it is in the registry).
  • telemetry/tracer.ts: behind the existing _includeToolDefinitions check and only when toolSpecs is non empty, maps each spec to { type: 'function', name, description, parameters: inputSchema } and sets gen_ai.tool.definitions. That is the fix.

What changes for a user

I went with the OpenTelemetry GenAI shape (type, name, description, parameters) because @zhoucy-syn proposed it in #5888. That differs from the Strands shape #5439 uses on invoke_agent. @zhoucy-syn and maintainers, would you rather the chat span match invoke_agent for now? Happy to switch.

I'm offering this as a small concrete proposal. Happy to adjust it, or close it if you'd prefer a different direction. Any feedback or tips are very welcome!

Agent._invokeModelWithMiddleware and its caller Agent._invokeModel

Agent._invokeModelWithMiddleware and its caller Agent._invokeModel, Functions view in Architect

Related Issues

Closes #6034

Related: #5888 (Python counterpart), #5439 (invoke_agent), #6033 (runs the model stream in the chat span context, part of #5885). This branch is off main and merges cleanly with both #6033 and #5439.

Documentation PR

None. The Captured Attributes tables in traces.mdx are shared with Python and do not list gen_ai.tool.definitions today; happy to add a row once the shape is settled.

Type of Change

New feature

Testing

tracer.test.node.ts: the chat span records the semconv list for two tools (one without inputSchema, so parameters is left out), and omits the attribute for an empty list, no toolSpecs, and no opt in. agent.tracer.test.node.ts: a middleware drops one of two tools, and the span gets the exact array model.stream received. Both new behavior tests fail with the source change reverted.

npm run lint, format:check, type-check, build, check:browser-bundle and test:coverage (217 files, 6552 passed) all pass.

  • I ran hatch run prepare

Checklist

  • I have read the CONTRIBUTING document
  • I have reviewed and understand every line of code in this PR, including any generated by AI tools, and I can explain why it works
  • My change is focused and reasonably small; I have split unrelated work into separate PRs
  • I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Disclosure: drafted with help from AI tools; I reviewed the change and ran the checks above.

@github-actions github-actions Bot added area-otel Open-telemetry related typescript Pull requests that update typescript code enhancement New feature or request labels Nov 15, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-otel Open-telemetry related enhancement New feature or request typescript Pull requests that update typescript code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] TS: Record gen_ai.tool.definitions on the chat (model invoke) span

1 participant