Skip to content

[opentelemetry-util-genai] gen_ai.tool.definitions keeps description and parameters when content capture is off #655

Open

Description

@sfc-gh-zeningchen

Describe your environment

OS: Ubuntu
Python version: Python 3.12.14
Package version: opentelemetry-util-genai 1.2b0.dev (main, 3e6a57d)
GenAI library (e.g. anthropic, openai) and version: n/a, reproduces through the util alone

What happened?

get_content_attributes in util/opentelemetry-util-genai/src/opentelemetry/util/genai/_invocation.py
takes an early return when content capture is disabled. Its comment says the intent is to emit the
attribute while leaving the optional properties out:

# Tool definitions are always captured, the sem conv recommends adding params / description only
# when the content capture mode is set..
if mode not in allowed_modes:
    return (
        {GenAI.GEN_AI_TOOL_DEFINITIONS: serialize(tool_definitions)}
        if tool_definitions
        else {}
    )

serialize() is the same full asdict() serialization the capture-enabled path uses, so
description and parameters go out either way. The code does not do what its comment says.

model/gen-ai/gen-ai-tool-definitions.json in semantic-conventions-genai requires only type and
name. Both the schema and the gen_ai.tool.definitions entry in model/gen-ai/registry.yaml say:

Since this attribute could be large, it's NOT RECOMMENDED to populate non-required properties by
default. Instrumentations MAY provide a way to enable populating optional properties.

parameters holds the tool's full JSON Schema, so it is the large part of the payload, and the
registry entry also carries a "may contain sensitive information" warning. Someone who disables
content capture still receives it.

Steps to Reproduce

import os

os.environ["OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT"] = "NO_CONTENT"

from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
from opentelemetry.sdk.trace.export.in_memory_span_exporter import (
    InMemorySpanExporter,
)
from opentelemetry.util.genai.handler import TelemetryHandler
from opentelemetry.util.genai.types import FunctionToolDefinition

exporter = InMemorySpanExporter()
provider = TracerProvider()
provider.add_span_processor(SimpleSpanProcessor(exporter))

handler = TelemetryHandler(tracer_provider=provider)
invocation = handler.inference(provider="anthropic", request_model="a-model")
invocation.tool_definitions = [
    FunctionToolDefinition(
        name="get_weather",
        description="Get weather by city",
        parameters={
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    )
]
invocation.stop()

span = exporter.get_finished_spans()[0]
print(span.attributes["gen_ai.tool.definitions"])

Expected Result

Only the properties the schema requires:

[{"name":"get_weather","type":"function"}]

Actual Result

[{"name":"get_weather","description":"Get weather by city","parameters":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]},"type":"function"}]

Additional context

There is a second question behind this one, and answering it first may change the fix.

model/gen-ai/spans.yaml marks gen_ai.tool.definitions requirement_level: opt_in and puts it in
the attributes.gen_ai.content group next to gen_ai.input.messages, gen_ai.output.messages and
gen_ai.system_instructions. That placement reads as "the attribute is content and follows the
content-capture setting", which is the opposite of "always captured".

The instrumentations in this repo are split on it. genai-openai gates tool_definitions behind
capture_content in both utils.py (Chat Completions) and response_extractors.py (Responses API).
genai-langchain, genai-agno, genai-portkey and genai-smolagents set it regardless. The same
call therefore yields a different attribute depending on which provider you use.

Suggested order:

  1. Decide whether the attribute follows the content-capture setting, given opt_in and its
    membership in the content attribute group.
  2. Either way, make the capture-disabled path serialize only type and name rather than the whole
    object, so the code matches its comment and the semconv guidance.
  3. Align the instrumentations on the outcome.

References:

Would you like to implement a fix?

No

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions