Enhancement: Add forward compatible instrument creation APIs

As surfaced in #5444, adding advisory attributes to existing Meter.create_* signatures would break third party OTel implementations when callers pass the new arguments. Hence, we must decide on a way to introduce an extendable creation API while preserving existing methods.

Three alternatives to consider are:

Per-instrument methods with options

meter.create_counter_with_options(
    "http.client.requests",
    options=CounterOptions(attributes_advisory=("http.request.method",)),
)

Instrument builders

meter.counter_builder("http.client.requests").with_attributes_advisory(
    ("http.request.method",)
).build()

Generic creation method with typed specifications

meter.create_instrument(
    CounterSpec(
        name="http.client.requests",
        attributes_advisory=("http.request.method",),
    )
)

Add forward compatible **kwargs arguments to existing methods

def create_counter(self, name: str, unit: str = "", description: str = "", **kwargs) -> Counter: ...

The chosen interface should accommodate future creation parameters without repeatedly changing SDK method signatures.

A similar extensibility strategy could also apply to the span creation APIs in #5819, which introduces separate concrete methods to preserve compatibility with existing Tracer implementations.

My personal opinion is that 1 the approach we should take if we want to ensure no potential API breakage. It is nearly identical to the existing API, so users will find it familiar.

While not relevant to instrument creation, option 2 is undesirable because of the overhead introduced by the builder abstraction. My own benchmarking showed that the overhead of builders are ~3x the overhead of a single function call with several arguments. I would personally prefer to align on a design that we can utilize across signals, approach 2 would likely introduce performance issues/bottlenecks for span/log creation.

+1 to Option 1. Implemented in lmolkova@90b6d29 for #5819, keeping span type as a top-level keyword argument.

3. +1 to Option 1. Implemented in lmolkova@90b6d29 for #5819, keeping span type as a top-level keyword argument.

Awesome! I've also updated #5444 to follow Option 1. I will change it if we decide that an alternative approach is superior.

MO none of the options are super "pythonic" but it's just an issue with ABCs in the python. Of these I also prefer option 1.

Hence, we must decide on a way to introduce an extendable creation API while preserving existing methods.

I thought on opentelemetry-sdk: add support for advisory attributes on metric instruments #5444 we had decided to just update the existing API since the SDK is pinned to API. I'm not arguing one way or another but what changed? Is this part of a plan to decouple the API/SDK versions?
Do we want to apply this to Logs API before stability as well vs using **kwargs we discussed before?

What exactly is Option 1 ? A Single InstrumentOptions instead of a Per-Instrument Options (CounterOptions, HistogramOptions) ?

So with this approach, when we add a new option and a custom SDK doesn't support it, it just silently ignores it right ?

Maybe we should add a way for custom SDKs to signal that they support the option, we could add

class Meter(ABC):
    # Subclasses declare which InstrumentOptions field names they handle:
    _SUPPORTED_INSTRUMENT_OPTIONS: ClassVar[frozenset[str]] = frozenset()

and then log something when the user sets an option but the SDK doesn't support it

unsupported = options._non_default_fields() - self._SUPPORTED_INSTRUMENT_OPTIONS
if unsupported:
    _logger.warning(
        "%s does not support InstrumentOptions field(s): %s",
        type(self).__name__,
        ", ".join(sorted(unsupported)),
    )

Honestly I liked the idea of adding the parameter to the existing methods as a 1 time change instead of having to maintain 2 sets of all these methods going forward..

@lmolkova

MO none of the options are super "pythonic" but it's just an issue with ABCs in the python. Of these I also prefer option 1.

Hence, we must decide on a way to introduce an extendable creation API while preserving existing methods.

I thought on opentelemetry-sdk: add support for advisory attributes on metric instruments #5444 we had decided to just update the existing API since the SDK is pinned to API. I'm not arguing one way or another but what changed? Is this part of a plan to decouple the API/SDK versions?

Do we want to apply this to Logs API before stability as well vs using **kwargs we discussed before?

There were concerns in the SIG call last week about potential issues this could maybe created for third-party SDKs. I think @lmolkova or @lzchen might be able to speak more on this. I guess I should add a fourth option of extending existing methods to include **kwargs for forward compatibility.

Once we align on a solution, I think we can apply the same to the Logs API (either the options method or the **kwargs method). We can hold off on stabilization until these issues are resolved.

@DylanRussell

8. What exactly is Option 1 ? A Single InstrumentOptions instead of a Per-Instrument Options (CounterOptions, HistogramOptions) ?
So with this approach, when we add a new option and a custom SDK doesn't support it, it just silently ignores it right ?
Maybe we should add a way for custom SDKs to signal that they support the option, we could add
class Meter(ABC): # Subclasses declare which InstrumentOptions field names they handle: _SUPPORTED_INSTRUMENT_OPTIONS: ClassVar[frozenset[str]] = frozenset()
and then log something when the user sets an option but the SDK doesn't support it
```
unsupported = options._non_default_fields() - self._SUPPORTED_INSTRUMENT_OPTIONS
if unsupported:
    _logger.warning(
        "%s does not support InstrumentOptions field(s): %s",
        type(self).__name__,
        ", ".join(sorted(unsupported)),
    )
```

@DylanRussell Good idea, the only potential concern I would have here is that it would add considerable overhead for "hot" calls (e.g. span creation).

Honestly I liked the idea of adding the parameter to the existing methods as a 1 time change instead of having to maintain 2 sets of all these methods going forward..

I'm adding this back as a fourth option. The reason there is some hesitation here is that under strict SemVer interpretation, adding an argument (even an optional one) to an existing abstract method is a breaking change for concrete implementations. This would never create an issue for the standard OTel SDK, but could (in rare cases) create issues for 3rd party SDK implementations.

I've added a fourth option to the list above. Personally, I would prefer if we went this route, but there is a small risk involved. To demonstrate what issues this could cause, consider the following:

Suppose an existing custom-otel package has the following implementation:

class CustomMeter(APIMeter):
    def create_counter(name: str, unit: str = "", description: str = "") -> CustomCounter: ...
    ...

if we update the APIMeter signature to the following, then existing calls are unaffected.

class APIMeter:
    def create_counter(name: str, unit: str = "", description: str = "", **kwargs) -> APICounter: ...

For example, the following code still works:

custom_meter: CustomMeter = get_custom_meter()
custom_counter = custom_meter.create_counter("hello", unit="ms", description="hello world!") # works

However, suppose we have the following:

def build_counter(meter: APIMeter) -> APICounter:
    return meter.create_counter("hello", unit="ms", description="hello world!", attributes_advisory=("foo", "bar"))

custom_meter: CustomMeter = get_custom_meter()
custom_counter = build_counter(custom_meter) # fails with TypeErorr

The example above is where the potential breakage occurs.

Note that because opentelemetry-sdk pins the version of opentelemetry-api it uses, this cannot happen with opentelemetry-sdk. This issue is only possible with third-party SDKs. Likewise, the issue above is present ONLY when the caller attempts to use new arguments via **kwargs.

One more realistic scenario that could occur in practice is if a user with an existing custom-otel implementation tries to use a newer version of the opentelemetry-instrumentation-requests package which creates internal counters with the new advisory attributes argument. With SemVer semantics, this should always work, but in this case if custom-otel doesn't include the **kwargs param, loading this instrumentation library will result in a crash.

I did AI-assisted research on public code to see whether the third-party risk is real. It is.

If we added kwargs to existing methods, our instrumentations would start to pass new parameters and these custom implementation would start failing:

Implementation	Stars	`opentelemetry-api` constraint	Method without `**kwargs`
Logfire `_ProxyMeter`	4.5k	`>=1.39.0,<1.46.0`	`create_counter`
Logfire `_ProxyTracer`	4.5k	`>=1.39.0,<1.46.0`	`start_span`
Temporal `_ReplaySafeMeter`	1.2k	`>=1.26,<2`	`create_counter`
Polar `PrometheusMeter`	10.5k	`1.45.1` (app lockfile, via Logfire)	`create_up_down_counter`
Datadog `Tracer`	654	`>=1,<2`	`start_span`
Elastic APM `Tracer`	431	unconstrained	`start_span`
Instana `InstanaTracer`	68	`>=1.27.0`	`start_span`
New Relic `Tracer`	222	unconstrained	`start_as_current_span`
Judgeval `ProxyTracer`	1.1k	`>=1.36.0`	`start_span`
Latitude `_LatitudeTracer`	4.7k	`==1.44.0`	`start_span`

[UPDATE]: added otel-api version range, so not all of them would be immediately affected

I did AI-assisted research on public code to see whether the third-party risk is real. It is.

If we added kwargs to existing methods, our instrumentations would start to pass new parameters and these custom implementation would start failing:

Implementation Stars Method without **kwargs

Logfire _ProxyMeter 4.5k create_counter

Temporal _ReplaySafeMeter 1.2k create_counter

Polar PrometheusMeter 10.5k create_up_down_counter

Datadog Tracer 654 start_span

Elastic APM Tracer 431 start_span

Instana InstanaTracer 68 start_span

Logfire _ProxyTracer 4.5k start_span

New Relic Tracer 222 start_as_current_span

Judgeval ProxyTracer 1.1k start_span

Latitude _LatitudeTracer 4.7k start_span

Wow, I did not expect there to be this many 😳

@lmolkova

@lmolkova would your AI mind adding to the table what dependency range each of them uses for opentelemetry-api?

I also wanted to drop this __init_subclass__ hook from SQLAlchemy. Python is flexible if we need it to be. For greenfield code I think the **kwargs approach is best and this feels like a runtime safeguard.

added version range, so only some of them would get affected - nevertheless, it seems quite a few users would be broken.

I'm still in favor of updating the existing method signatures.. Otherwise the existing methods become frozen legacy wrappers, and we basically force everyone to switch methods

16. I'm still in favor of updating the existing method signatures.. Otherwise the existing methods become frozen legacy wrappers, and we basically force everyone to switch methods

we're not forcing anyone to switch, they can stay on old methods forever. The implication is that we break API compatibility which really hurts our reputation and is not allowed by the specification.

if they want the new feature they have to switch right ?

Navigation Menu

Repository navigation

Enhancement: Add forward compatible instrument creation APIs #5877

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Enhancement: Add forward compatible instrument creation APIs #5877

Description

Activity

lmolkova commented on Nov 1, 2026

herin049 commented on Nov 2, 2026

aabmass commented on Nov 2, 2026

DylanRussell commented on Nov 2, 2026

herin049 commented on Nov 2, 2026

herin049 commented on Nov 2, 2026

herin049 commented on Nov 2, 2026

lmolkova commented on Nov 2, 2026

herin049 commented on Nov 2, 2026

aabmass commented on Nov 2, 2026

lmolkova commented on Nov 2, 2026

DylanRussell commented on Nov 3, 2026

lmolkova commented on Nov 3, 2026

DylanRussell commented on Nov 3, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions