Skip to content

docs: complete framework coverage matrix #147

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
imran-siddique merged 1 commit into from
Aug 31, 2026
Merged

docs: complete framework coverage matrix #147

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 16 additions & 12 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,18 +71,22 @@ TRACE only works as a standard if it is genuinely neutral. Integrations are list
| [scheduled-agents](scheduled-agents/) | agentrust-io | trace | community |
<!-- integration-index:end -->

### First-party framework coverage

| Framework | Adapter | Released framework exercised in CI | Evidence boundary |
|---|---|---|---|
| Google ADK | [Google ADK](integrations/google-adk/) | Yes - Google ADK 2.7.1 `InMemoryRunner` plugin lifecycle | Callback-visible invocation, model, and available tool identity; no payloads, retries, agent graph, function-body execution, or policy enforcement |
| LangChain | [LangChain](integrations/langchain/) | Yes — LangChain Core 1.6.0 callback contract | Tool identity and outcome plus model identity; no chain topology or runnable state |
| LangGraph | [LangChain](integrations/langchain/) | Yes — LangGraph 1.2.11 `StateGraph` with a nested tool call | Propagated tool callbacks; no nodes, edges, state transitions, checkpoints, or rollback decisions |
| LlamaIndex | [LlamaIndex](integrations/llamaindex/) | No — current tests use representative event objects | Allow-listed tool and model fields; released-framework interoperability remains unverified |

“Adapter exists” and “released framework exercised” are separate claims here.
The adapter README documents the evidence each callback surface can support; a
missing graph or state concept is not inferred into the TRACE record.
### Framework coverage

| Framework | Adapter | Evidence source | Released framework exercised in CI | Evidence boundary |
|---|---|---|---|---|
| Google ADK | [Google ADK](integrations/google-adk/) | First-party `BasePlugin` lifecycle | Yes - Google ADK 2.7.1 `InMemoryRunner` | Callback-visible invocation, model, and available tool identity; no payloads, retries, agent graph, function-body execution, or policy enforcement |
| LangChain | [LangChain](integrations/langchain/) | First-party `BaseCallbackHandler` callbacks | Yes - LangChain Core 1.6.0 callback contract | Tool identity and outcome plus model identity; no chain topology or runnable state |
| LangGraph | [LangChain](integrations/langchain/) | First-party LangChain callbacks propagated by the graph | Yes - LangGraph 1.2.11 `StateGraph` with a nested tool call | Propagated tool callbacks; no nodes, edges, state transitions, checkpoints, or rollback decisions |
| LlamaIndex | [LlamaIndex](integrations/llamaindex/) | First-party `BaseEventHandler` events | No - current tests use representative event objects | Allow-listed tool and model fields; released-framework interoperability remains unverified |
| OpenAI Agents SDK | [OpenAI Agents SDK](integrations/openai-agents/) | First-party `TracingProcessor` spans | Yes - OpenAI Agents SDK 0.22.0 scripted model and tool run | Tool, handoff, agent, and MCP identity and order; no payloads, reasoning traces, guardrail outcomes, session state, or retries |
| Pydantic AI | [OpenTelemetry GenAI](integrations/otel-genai/) | OpenTelemetry GenAI transcription | Yes - Pydantic AI 2.35.1 `TestModel` with a tool call | Telemetry-reported model and tool identity; no payloads; absent `gen_ai.tool.type` is not inferred |

"Adapter exists" and "released framework exercised" are separate claims here.
First-party hooks produce self-origin records with no `origin` block. Telemetry
transcriptions carry their weaker evidence boundary explicitly. Each adapter
README documents what its observation surface can support; a missing concept is
not inferred into the TRACE record.

The [Copilot](copilot/), [Cursor](cursor/), [Windsurf](windsurf/) and
[Gemini CLI](gemini-cli/) drift checks are intentionally outside this manifest
Expand Down
15 changes: 15 additions & 0 deletions integrations/otel-genai/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,21 @@ Three inputs come from the operator, because telemetry does not carry them: the
| `tool_transcript.hash` | canonical digest over each `execute_tool` span's `gen_ai.tool.name`, `gen_ai.tool.call.id`, `gen_ai.tool.type` |
| `tool_transcript.call_count` | number of `execute_tool` spans |

## Pydantic AI interoperability

Pydantic AI needs no dedicated adapter: its instrumentation emits OpenTelemetry
GenAI spans that this adapter already accepts. CI pins Pydantic AI 2.35.1 and
runs a released `Agent` with a `TestModel` and a real tool call, without network
access.

The result remains a telemetry transcription, with `origin.kind: log-import`
and `appraisal.status: none`. The interoperability test also records the current
boundary: Pydantic AI does not emit `gen_ai.tool.type`, so the transcript leaves
that value unset. It emits `gen_ai.agent.call.id` where the conventions define
`gen_ai.agent.id`; neither is used as the TRACE subject identity. Tool arguments,
tool results, and input and output messages are emitted by default and remain
excluded as payloads.

## What is deliberately not mapped

`gen_ai.tool.call.arguments` and `gen_ai.tool.call.result` are **payloads**. Hashing them into the transcript would put request and response content into an artifact whose purpose is being handed to a third party. A test asserts that a span carrying an IBAN in its arguments produces the same transcript hash as one without, so the exclusion cannot regress silently.
Expand Down
Loading