Repository navigation
feat(telemetry): record gen_ai.tool.definitions on the chat span (typescript) - #6162
Draft
abhiramshaji-cyber wants to merge 2 commits into
Draft
abhiramshaji-cyber wants to merge 2 commits into
abhiramshaji-cyber wants to merge 2 commits into
Conversation
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Human Overview
Description
Hey @zhoucy-syn, hope you're doing well! It's me again from #5885. Your replay use case there is what this is built around, so thank you for spelling it out.
With
gen_ai_tool_definitionsinOTEL_SEMCONV_STABILITY_OPT_IN, each TypeScriptchatspan now records the tools that exact model call received, so a recorded call can be replayed from the span that owns it.What this code is for
_invokeModelWithMiddlewareinagent.tsruns every model call. It builds anInvokeModelContext(messages, system prompt,toolSpecs), letsInvokeModelStagemiddleware change it, then opens thechatspan withstartModelInvokeSpanand streams from the model withctx.toolSpecs.Before this PR
A developer runs an agent with two tools and the opt in set, and a middleware narrows the tools to one for this call.
_invokeModelWithMiddlewarebuildsctx.toolSpecswith both tools; the middleware leaves one.startModelInvokeSpanis called withmessages,modelIdandsystemPromptonly.model.streamreceives the one tool instreamOptions.toolSpecs.chatspan has nogen_ai.tool.definitions, so the trace cannot say which tools this call saw.After this PR
Same developer, same agent, same middleware.
startModelInvokeSpanalso getstoolSpecs: ctx.toolSpecs, the same arraymodel.streamgets.gen_ai.tool.definitionsas[{"type":"function","name":"kept","description":"...","parameters":{...}}].flowchart LR classDef box fill:#eef1ff,stroke:#4c5bd4,color:#111827 classDef bad fill:#ffd6d6,stroke:#cc0000,color:#111827,stroke-width:3px classDef good fill:#c8f0c8,stroke:#008800,color:#111827,stroke-width:3px subgraph BEFORE["BEFORE: 0 tool definitions on chat"] direction TB B1["agent.invoke or agent.stream"] --> B2["InvokeModelStage middleware sets ctx.toolSpecs"] B2 --> B3["startModelInvokeSpan(messages, modelId, systemPrompt)"] B3 --> B4{"toolSpecs passed?"} B4 -->|"never"| B5["chat span: no gen_ai.tool.definitions"] B4 -->|"model call itself"| B6["model.stream gets ctx.toolSpecs"] end subgraph AFTER["AFTER: the tools this call received"] direction TB A1["agent.invoke or agent.stream"] --> A2["InvokeModelStage middleware sets ctx.toolSpecs"] A2 --> A3["startModelInvokeSpan(..., toolSpecs: ctx.toolSpecs)"] A3 --> A4{"THE CHANGE: opt in set and toolSpecs.length"} A4 -->|"yes"| A5["chat span: gen_ai.tool.definitions, semconv shape"] A4 -->|"no tools or no opt in"| A6["chat span unchanged, model.stream as before"] end B1 ~~~ A1 class B1,B2,B3,B6 box class A1,A2,A3,A6 box class B4,B5 bad class A4,A5 good linkStyle 3 stroke:#cc0000,stroke-width:5px linkStyle 8 stroke:#008800,stroke-width:5px style BEFORE fill:#fff5f5,stroke:#cc0000,color:#111827 style AFTER fill:#f3fff3,stroke:#008800,color:#111827The change
telemetry/types.ts:StartModelInvokeSpanOptionsgets an optionaltoolSpecs?: readonly ToolSpec[].agent/agent.ts: thestartModelInvokeSpancall passestoolSpecs: ctx.toolSpecs, read after middleware, so it matches what the model gets (including the structured output tool when it is in the registry).telemetry/tracer.ts: behind the existing_includeToolDefinitionscheck and only whentoolSpecsis non empty, maps each spec to{ type: 'function', name, description, parameters: inputSchema }and setsgen_ai.tool.definitions. That is the fix.What changes for a user
gen_ai_tool_definitions; with it, eachchatspan carries the tools sent on that call.invokeandstreamboth go through_invokeModelWithMiddleware, so both get it.invoke_agentis untouched, so this does not overlap fix(telemetry): record tool definitions on the agent span in TypeScript #5439.I went with the OpenTelemetry GenAI shape (
type,name,description,parameters) because @zhoucy-syn proposed it in #5888. That differs from the Strands shape #5439 uses oninvoke_agent. @zhoucy-syn and maintainers, would you rather thechatspan matchinvoke_agentfor now? Happy to switch.I'm offering this as a small concrete proposal. Happy to adjust it, or close it if you'd prefer a different direction. Any feedback or tips are very welcome!
Agent._invokeModelWithMiddlewareand its callerAgent._invokeModel, Functions view in ArchitectRelated Issues
Closes #6034
Related: #5888 (Python counterpart), #5439 (
invoke_agent), #6033 (runs the model stream in thechatspan context, part of #5885). This branch is offmainand merges cleanly with both #6033 and #5439.Documentation PR
None. The Captured Attributes tables in
traces.mdxare shared with Python and do not listgen_ai.tool.definitionstoday; happy to add a row once the shape is settled.Type of Change
New feature
Testing
tracer.test.node.ts: thechatspan records the semconv list for two tools (one withoutinputSchema, soparametersis left out), and omits the attribute for an empty list, notoolSpecs, and no opt in.agent.tracer.test.node.ts: a middleware drops one of two tools, and the span gets the exact arraymodel.streamreceived. Both new behavior tests fail with the source change reverted.npm run lint,format:check,type-check,build,check:browser-bundleandtest:coverage(217 files, 6552 passed) all pass.hatch run prepareChecklist
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Disclosure: drafted with help from AI tools; I reviewed the change and ran the checks above.