diff --git a/openspec/changes/archive/2026-10-15-specify-ai-instrumentation-context-attribution/.openspec.yaml b/openspec/changes/archive/2026-10-15-specify-ai-instrumentation-context-attribution/.openspec.yaml new file mode 100644 index 00000000..a216797d --- /dev/null +++ b/openspec/changes/archive/2026-10-15-specify-ai-instrumentation-context-attribution/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-10-15 diff --git a/openspec/changes/archive/2026-10-15-specify-ai-instrumentation-context-attribution/design.md b/openspec/changes/archive/2026-10-15-specify-ai-instrumentation-context-attribution/design.md new file mode 100644 index 00000000..9726a04d --- /dev/null +++ b/openspec/changes/archive/2026-10-15-specify-ai-instrumentation-context-attribution/design.md @@ -0,0 +1,50 @@ +## Overview + +This change adds no capability. It records the attribution contract that posthog-js, posthog-python +and posthog-go implemented identically for the OpenTelemetry span processor that turns AI SDK spans +into PostHog AI events. + +## Key Decisions + +- **Keep it in `capture-ai` rather than `traces` or a new capability.** `traces` specifies + PostHog's own tracing API, which carries no OpenTelemetry dependency and uses the logs wire keys + (`posthogDistinctId`, `sessionId`); the AI bridge is OTel-native and uses the ingestion keys + (`posthog.distinct_id`, `$session_id`). `capture-ai` is the spec that already governs what + PostHog's AI instrumentation emits, so attribution belongs next to it. A separate + `ai-otel-bridge` capability would be justified if more of the bridge were written down; today + attribution is the only part with three agreeing implementations. +- **Specify the outcome, not the mechanism.** The three SDKs store the snapshot differently (a + `WeakMap`, a lock-guarded map keyed by trace/span id with fork reset, a mutex-guarded map) and + convert values differently, because attribute typing differs per platform. What they agree on is + observable: which keys appear, what wins, and that nothing else on the provider sees the context. + Value conversion is therefore left platform-idiomatic, with only the skip rules pinned, since a + skipped key and a coerced key are both observable at ingestion. +- **Pin the snapshot to span start and the stamp to span end.** All three snapshot in `onStart` and + merge in `onEnd`, which is what makes concurrent requests attribute correctly — a span started + inside one request keeps that request's person even if it ends after the context exits. +- **Require that the live span is not mutated.** Each implementation's first version wrote onto the + live span, and each was changed after review because that leaks the person, session and properties + to every other processor and exporter on the same provider. Writing it down is what stops the next + port from repeating it. +- **Leave the context primitive to the platform.** Node reads an explicit client option because each + client owns its own `AsyncLocalStorage`; Python reads its contextvars-based context; Go reads the + `context.Context` the span was started from. The requirement binds the attribution, not the wiring. + +## Alternatives Considered + +- **Require the AI bridge everywhere.** Rejected. Only three SDKs ship it, and `capture-ai` already + conditions its surface on an SDK supporting AI capture at all. +- **Specify `$ai_session_id` as the session key.** Rejected, and explicitly the other way round: all + three map the context session to `$session_id` so AI events join replays, and deliberately leave + `$ai_session_id` to the instrumentation's conversation id. +- **Wait for a fourth SDK.** Rejected. Three independent implementations agreeing is the strongest + signal this repo gets that a behavior is contractual rather than incidental. + +## Open Questions + +- Whether the value-conversion rules (which types pass through as attributes, which become JSON + strings) should be pinned cross-SDK. They are close but not identical — Go has typed slices, + Python stringifies out-of-range integers — and no user-visible disagreement is known, so the + requirement pins only the skip rules. +- Whether this belongs in `capture-ai` long term, or in a dedicated capability covering the rest of + the OTel AI bridge (span-to-event mapping, redaction, the exporter). Worth a human call. diff --git a/openspec/changes/archive/2026-10-15-specify-ai-instrumentation-context-attribution/proposal.md b/openspec/changes/archive/2026-10-15-specify-ai-instrumentation-context-attribution/proposal.md new file mode 100644 index 00000000..cb7bb68e --- /dev/null +++ b/openspec/changes/archive/2026-10-15-specify-ai-instrumentation-context-attribution/proposal.md @@ -0,0 +1,52 @@ +## Why + +Three SDKs shipped the same behavior in the same week: the OpenTelemetry span processor that +turns AI SDK spans into PostHog AI events now stamps the active PostHog context onto the AI +spans it exports, so a server that traces LLM calls for many users attributes each AI event to +the right person and session instead of to one process-wide resource attribute. + +- [PostHog/posthog-js#5299](https://github.com/PostHog/posthog-js/pull/5299) — `@posthog/ai` + `PostHogSpanProcessor`, new `client` option reading `withContext`/`enterContext` +- [PostHog/posthog-python#1060](https://github.com/PostHog/posthog-python/pull/1060) — + `PostHogSpanProcessor` reading `new_context()` / `identify_context` / `tag` +- [PostHog/posthog-go#371](https://github.com/PostHog/posthog-go/pull/371) — `otel` + `SpanProcessor` reading `RequestContextFromContext` + +`capture-ai` is the only spec that governs how PostHog's AI instrumentation emits AI events, and +it says nothing about attribution: which keys carry the person and session, what wins when the +span already names a user, and whether the instrumentation may mutate the span other processors +and exporters see. All three implementations agreed on every one of those answers, so the +contract exists already — it is just unwritten, and the next port has nothing to copy. + +## What Changes + +- Add one requirement to `capture-ai`, **AI instrumentation attributes events to the active + context**, stating: attribution is read from the SDK's own context primitive; the wire keys are + `posthog.distinct_id` and `$session_id`; `$ai_session_id` stays free for the instrumentation's + conversation ID; precedence runs span attribute → context property → context distinct id / + session id; a span that already names a user keeps it; the live span is never mutated; and + stamping never raises into the instrumented call. +- Note in `Applicability` that this requirement binds any SDK shipping the OTel AI bridge, which + now includes posthog-go, whether or not it exposes `capture_ai`. + +## Capabilities + +### New Capabilities + +_None._ + +### Modified Capabilities + +- `capture-ai`: adds the **AI instrumentation attributes events to the active context** + requirement and one `Applicability` sentence. + +## Impact + +Not a breaking change for the contract: it records what posthog-js, posthog-python and +posthog-go all do as of the PRs above. It is a behavior change for *users* of those SDKs — an app +that already runs PostHog contexts (Django/Flask/ASGI middleware, `NewRequestContextMiddleware`, +posthog-node `withContext`) sees its AI events attributed to the context's person after +upgrading, which each SDK's changeset calls out. + +SDKs with no AI instrumentation, and AI instrumentation on a platform with no ambient or request +context primitive, stay conformant: the requirement is conditioned on both existing. diff --git a/openspec/changes/archive/2026-10-15-specify-ai-instrumentation-context-attribution/specs/capture-ai/spec.md b/openspec/changes/archive/2026-10-15-specify-ai-instrumentation-context-attribution/specs/capture-ai/spec.md new file mode 100644 index 00000000..26ba3402 --- /dev/null +++ b/openspec/changes/archive/2026-10-15-specify-ai-instrumentation-context-attribution/specs/capture-ai/spec.md @@ -0,0 +1,78 @@ +## ADDED Requirements + +### Requirement: AI instrumentation attributes events to the active context + +AI instrumentation that emits AI events from OpenTelemetry spans SHALL attribute the events it +exports to the active context where the SDK has an ambient or request context primitive. This +covers the `PostHogSpanProcessor` / `SpanProcessor` bridge in posthog-js `@posthog/ai`, +posthog-python, and posthog-go `otel`. The requirement is conditioned on both: an SDK with no AI +bridge, or an AI bridge on a platform with no context primitive, is conformant. Where the bridge +takes the SDK client as an option to reach the context (posthog-js `@posthog/ai` reads it only +when `PostHogSpanProcessor` is given `client`), the requirement applies once that option is set. + +Where it applies: + +- The context's distinct id SHALL be exported as the `posthog.distinct_id` attribute, and the + context's session id as `$session_id` — the same session key as `capture`, so AI events join + session replays. `$ai_session_id` SHALL NOT be used for the context session; it stays available + for the instrumentation's own conversation id or an explicit caller-supplied property. +- Each context property SHALL be exported as an attribute under its own key, unchanged. +- Precedence SHALL be: an attribute the span already carries, then a context property, then the + context's distinct id or session id. +- Where the span already names a user, the instrumentation SHALL NOT add `posthog.distinct_id` + from the context or from a context property. The identity attributes recognized for this are + `posthog.distinct_id`, `user.id`, and the instrumentation's own distinct-id metadata keys + (`ai.telemetry.metadata.posthog_distinct_id`, `ai.settings.context.posthog_distinct_id`). +- The active context SHALL be captured when the span starts and applied when it ends, so + concurrent requests each export their own values and a span outliving its context keeps the + context it started in. Whether a context property mutated in place while the span is open is + seen at export is platform-specific. +- The instrumentation SHALL NOT mutate the live span: the attributes SHALL be merged into the + exported copy only, so other processors and exporters on the same tracer provider never observe + the context. Spans the bridge does not export as AI events SHALL be left untouched. +- Attribution SHALL be best effort and SHALL NOT raise into the instrumented call: a failing or + unavailable context source, and a value the platform cannot represent as an attribute, SHALL + skip the affected key rather than propagate an error or drop the span. +- A value that is absent, null, an empty string, or a non-finite number SHALL be skipped rather + than exported. Conversion of the remaining values to attributes is platform-idiomatic. + +#### Scenario: the active context attributes the exported AI event (@server) +- **GIVEN** AI instrumentation wired to the SDK's context primitive +- **AND** an active context with distinct id "user-123", session id "s-1", and property "plan" set to "pro" +- **WHEN** an AI span started inside that context ends +- **THEN** the exported AI event should carry `posthog.distinct_id` "user-123" +- **AND** it should carry `$session_id` "s-1" +- **AND** it should carry "plan" equal to "pro" + +#### Scenario: concurrent contexts do not mix (@server) +- **GIVEN** AI instrumentation wired to the SDK's context primitive +- **AND** two requests running concurrently, one in a context with distinct id "user-a" and one with "user-b" +- **WHEN** an AI span started in each request ends +- **THEN** each exported AI event should carry the distinct id of the context its span started in + +#### Scenario: a span that already names a user keeps it (@server) +- **GIVEN** an active context with distinct id "user-123" +- **WHEN** an AI span carrying its own `user.id` attribute "span-user" ends +- **THEN** the exported AI event should not carry `posthog.distinct_id` "user-123" + +#### Scenario: the span's own attribute wins over a context property (@server) +- **GIVEN** an active context with property "plan" set to "pro" +- **WHEN** an AI span carrying its own "plan" attribute "enterprise" ends +- **THEN** the exported AI event should carry "plan" equal to "enterprise" + +#### Scenario: the live span is not changed (@server) +- **GIVEN** AI instrumentation and a second span processor registered on the same tracer provider +- **AND** an active context with distinct id "user-123" +- **WHEN** an AI span started inside that context ends +- **THEN** the second processor should observe no `posthog.distinct_id` attribute on the span + +#### Scenario: a broken context source does not break the call (@server) +- **GIVEN** AI instrumentation whose context source raises when read +- **WHEN** an AI span ends +- **THEN** the instrumented call should observe no exception +- **AND** the AI event should still be exported, without context attributes + +#### Scenario: unrepresentable context values are skipped (@server) +- **GIVEN** an active context with properties that include a null value, an empty string, and a non-finite number +- **WHEN** an AI span started inside that context ends +- **THEN** the exported AI event should omit those keys rather than export empty or non-finite values diff --git a/openspec/changes/archive/2026-10-15-specify-ai-instrumentation-context-attribution/tasks.md b/openspec/changes/archive/2026-10-15-specify-ai-instrumentation-context-attribution/tasks.md new file mode 100644 index 00000000..674503e0 --- /dev/null +++ b/openspec/changes/archive/2026-10-15-specify-ai-instrumentation-context-attribution/tasks.md @@ -0,0 +1,13 @@ +## 1. Spec delta + +- [x] 1.1 Add exactly one requirement to `capture-ai`, **AI instrumentation attributes events to the active context** +- [x] 1.2 State the wire keys and that `$ai_session_id` is not the session key +- [x] 1.3 State the precedence order and the already-names-a-user carve-out +- [x] 1.4 State that the live span is never mutated and that stamping never raises +- [x] 1.5 Cover each of the above with a scenario +- [x] 1.6 Note in `Applicability` that the requirement binds SDKs shipping the OTel AI bridge + +## 2. Validation + +- [x] 2.1 Check the delta against the shipped posthog-js, posthog-python and posthog-go sources +- [x] 2.2 Sync the delta into `specs/capture-ai/spec.md` and archive the change diff --git a/openspec/specs/capture-ai/spec.md b/openspec/specs/capture-ai/spec.md index 28ecdd7f..581818e5 100644 --- a/openspec/specs/capture-ai/spec.md +++ b/openspec/specs/capture-ai/spec.md @@ -22,7 +22,10 @@ strengthen, never weaken, before GA. `server` — currently implemented by posthog-python and posthog-node. Other server SDKs (Rust, Go, Ruby, ...) adopt this spec if and when they add AI support; until then their AI events ride plain `capture` under its standard -limits. +limits. The **AI instrumentation attributes events to the active context** +requirement binds any SDK shipping the OpenTelemetry AI bridge — today +posthog-js (`@posthog/ai`), posthog-python, and posthog-go (`otel`) — whether or +not that SDK exposes `capture_ai`. ## Public signatures @@ -215,3 +218,79 @@ SDK's AI wrapper libraries capture. It SHALL NOT alter payloads passed to - **AND** its custom properties should equal JSON `{"nested":{},"items":["1",null,2]}` - **AND** the analytics batch endpoint should receive no events +### Requirement: AI instrumentation attributes events to the active context + +AI instrumentation that emits AI events from OpenTelemetry spans SHALL attribute the events it +exports to the active context where the SDK has an ambient or request context primitive. This +covers the `PostHogSpanProcessor` / `SpanProcessor` bridge in posthog-js `@posthog/ai`, +posthog-python, and posthog-go `otel`. The requirement is conditioned on both: an SDK with no AI +bridge, or an AI bridge on a platform with no context primitive, is conformant. Where the bridge +takes the SDK client as an option to reach the context (posthog-js `@posthog/ai` reads it only +when `PostHogSpanProcessor` is given `client`), the requirement applies once that option is set. + +Where it applies: + +- The context's distinct id SHALL be exported as the `posthog.distinct_id` attribute, and the + context's session id as `$session_id` — the same session key as `capture`, so AI events join + session replays. `$ai_session_id` SHALL NOT be used for the context session; it stays available + for the instrumentation's own conversation id or an explicit caller-supplied property. +- Each context property SHALL be exported as an attribute under its own key, unchanged. +- Precedence SHALL be: an attribute the span already carries, then a context property, then the + context's distinct id or session id. +- Where the span already names a user, the instrumentation SHALL NOT add `posthog.distinct_id` + from the context or from a context property. The identity attributes recognized for this are + `posthog.distinct_id`, `user.id`, and the instrumentation's own distinct-id metadata keys + (`ai.telemetry.metadata.posthog_distinct_id`, `ai.settings.context.posthog_distinct_id`). +- The active context SHALL be captured when the span starts and applied when it ends, so + concurrent requests each export their own values and a span outliving its context keeps the + context it started in. Whether a context property mutated in place while the span is open is + seen at export is platform-specific. +- The instrumentation SHALL NOT mutate the live span: the attributes SHALL be merged into the + exported copy only, so other processors and exporters on the same tracer provider never observe + the context. Spans the bridge does not export as AI events SHALL be left untouched. +- Attribution SHALL be best effort and SHALL NOT raise into the instrumented call: a failing or + unavailable context source, and a value the platform cannot represent as an attribute, SHALL + skip the affected key rather than propagate an error or drop the span. +- A value that is absent, null, an empty string, or a non-finite number SHALL be skipped rather + than exported. Conversion of the remaining values to attributes is platform-idiomatic. + +#### Scenario: the active context attributes the exported AI event (@server) +- **GIVEN** AI instrumentation wired to the SDK's context primitive +- **AND** an active context with distinct id "user-123", session id "s-1", and property "plan" set to "pro" +- **WHEN** an AI span started inside that context ends +- **THEN** the exported AI event should carry `posthog.distinct_id` "user-123" +- **AND** it should carry `$session_id` "s-1" +- **AND** it should carry "plan" equal to "pro" + +#### Scenario: concurrent contexts do not mix (@server) +- **GIVEN** AI instrumentation wired to the SDK's context primitive +- **AND** two requests running concurrently, one in a context with distinct id "user-a" and one with "user-b" +- **WHEN** an AI span started in each request ends +- **THEN** each exported AI event should carry the distinct id of the context its span started in + +#### Scenario: a span that already names a user keeps it (@server) +- **GIVEN** an active context with distinct id "user-123" +- **WHEN** an AI span carrying its own `user.id` attribute "span-user" ends +- **THEN** the exported AI event should not carry `posthog.distinct_id` "user-123" + +#### Scenario: the span's own attribute wins over a context property (@server) +- **GIVEN** an active context with property "plan" set to "pro" +- **WHEN** an AI span carrying its own "plan" attribute "enterprise" ends +- **THEN** the exported AI event should carry "plan" equal to "enterprise" + +#### Scenario: the live span is not changed (@server) +- **GIVEN** AI instrumentation and a second span processor registered on the same tracer provider +- **AND** an active context with distinct id "user-123" +- **WHEN** an AI span started inside that context ends +- **THEN** the second processor should observe no `posthog.distinct_id` attribute on the span + +#### Scenario: a broken context source does not break the call (@server) +- **GIVEN** AI instrumentation whose context source raises when read +- **WHEN** an AI span ends +- **THEN** the instrumented call should observe no exception +- **AND** the AI event should still be exported, without context attributes + +#### Scenario: unrepresentable context values are skipped (@server) +- **GIVEN** an active context with properties that include a null value, an empty string, and a non-finite number +- **WHEN** an AI span started inside that context ends +- **THEN** the exported AI event should omit those keys rather than export empty or non-finite values