Skip to content

fix(deps): pin Phoenix auto-trace instrumentors for examples group - #26

Merged
Chang Liu (changliu2) merged 2 commits into
mainfrom
fix/phoenix-auto-trace-instrumentor-pins
May 8, 2026
Merged

fix(deps): pin Phoenix auto-trace instrumentors for examples group#26
Chang Liu (changliu2) merged 2 commits into
mainfrom
fix/phoenix-auto-trace-instrumentor-pins

Conversation

@changliu2

Copy link
Copy Markdown
Collaborator

What

Adds the four framework-specific OpenInference instrumentors used by examples/phoenix_auto_trace/eval_*.yaml to the examples optional-dependencies group, with version bounds matched to the framework versions already pinned in the same group.

Why

phoenix.otel.register(auto_instrument=True) auto-loads every installed openinference-instrumentation-* package. A single version-incompatible instrumentor crashes the entire call, breaking unrelated framework demos. Two real failures discovered while smoke-testing main post-#21:

Framework (we ship) Instrumentor that crashes Fix
dspy 2.6.13 openinference-instrumentation-dspy 0.1.34 (expects LM.acall not present) pin <0.1.20
crewai 1.6.1 openinference-instrumentation-crewai 1.1.4 (requires crewai>=1.10.1) pin <1.0.0

Plus the openai and litellm instrumentors were not declared at all — customers had to discover them.

Pins

toml "openinference-instrumentation-openai>=0.1.45" "openinference-instrumentation-litellm>=0.1.30" "openinference-instrumentation-dspy>=0.1.19,<0.1.20" # dspy 2.6.x "openinference-instrumentation-crewai>=0.1.22,<1.0.0" # crewai 1.6.x

A comment in pyproject.toml reminds future maintainers to bump these together when bumping crewai >= 1.10 or dspy >= 2.7.

Verification

Tier 1 unit tests: uv run pytest -q -> 528 passed, 14 skipped, 0 failed.

Azure smoke test on main @ 9fa3b30 -> 8/8 configs pass, 90 transcripts scored end-to-end through the SvelteKit viewer:

# Config Suite / Run Seeds Time Scores
1 travel_planner_langgraph travel-planner-langgraph-v1 / demo-1 5 2m 09s 5/5
2 travel_planner_neurosan travel-planner-neurosan-v1 / custom-otel 5 3m 15s 5/5
3 pipes/health_assistant health-assistant-v1 / gpt54-eval 15 5m 15s 15/15
4 phoenix_auto_trace/eval_openai framework-eval-mini / openai 13 4m 05s 13/13
5 phoenix_auto_trace/eval_langchain framework-eval-mini / langchain 13 4m 20s 13/13
6 phoenix_auto_trace/eval_litellm framework-eval-mini / litellm 13 192m ⚠️ 13/13
7 phoenix_auto_trace/eval_dspy framework-eval-mini / dspy 13 5m 22s 13/13
8 phoenix_auto_trace/eval_crewai framework-eval-mini / crewai 13 65m ⚠️ 13/13

Viewer (npm run dev in viewer/) renders all 4 suites; framework-eval-mini correctly groups all 5 framework runs; every run-detail page returns HTTP 200 with ~1.4MB of hydrated transcripts and scores.

Notes (not addressed in this PR)

  • litellm and crewai outliers above were Azure throttling + RemoteDisconnected retries, not code bugs (the rate-limit retry code from feat: coordinated rate-limit retry with per-model adaptive backoff #17 worked, just slowly).
  • openinference-instrumentation-crewai 0.1.22 still emits a non-fatal DependencyConflict: requested 'crewai >= 1.9.0' but found 'crewai 1.6.1' warning at register time. Harmless but worth bumping crewai in a follow-up.

Customers running `examples/phoenix_auto_trace/eval_*.yaml` previously
hit a hard crash inside `phoenix.otel.register(auto_instrument=True)`
because Phoenix auto-loads every installed
`openinference-instrumentation-*` package and a single mismatched
version (e.g. instrumentor expects an attribute the framework version
we ship does not have) brings down the whole call.

Add the four framework-specific instrumentors needed by the
phoenix_auto_trace demos to the `[project.optional-dependencies].examples`
group with version bounds matched to the framework versions already
pinned in the same group:

- openinference-instrumentation-openai>=0.1.45
- openinference-instrumentation-litellm>=0.1.30
- openinference-instrumentation-dspy>=0.1.19,<0.1.20  (dspy 2.6.x)
- openinference-instrumentation-crewai>=0.1.22,<1.0.0  (crewai 1.6.x)

Verified by an 8/8 smoke pass on main (90 transcripts scored across
LangGraph, NeurOSan, hosted Azure, and 5 phoenix_auto_trace frameworks)
plus Tier 1 unit tests (528 passed).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pre-merge audit caught a silent skip: the previous pin
`openinference-instrumentation-crewai>=0.1.22,<1.0.0` resolved to
0.1.22, which requires `crewai>=1.9.0`. Combined with our
`crewai[azure-ai-inference]>=1.6.1` framework pin, this surfaced as

    opentelemetry.instrumentation.instrumentor ERROR:
      DependencyConflict: requested: `crewai >= 1.9.0` but found: `crewai 1.6.1`

at `register(auto_instrument=True)` time, with the crewai instrumentor
silently disabled. The hard crash this PR fixes was real, but 1 of the 4
pinned instrumentors never actually attached.

PyPI inspection of `openinference-instrumentation-crewai` shows that
0.1.17 is the last release compatible with crewai < 1.9.0 (it requires
`crewai>=0.119.0`, which 1.6.1 satisfies). Tighten the pin to
`>=0.1.17,<0.1.18` so the instrumentor and the framework agree.

Verified locally:
- `register(auto_instrument=True)` -> no DependencyConflict, no crash
- `uv sync --extra examples --extra otel` resolves cleanly
- Unit tests: 528 passed, 14 skipped

Bumping crewai to 1.9.0+ (and the instrumentor to 1.x) is the proper
long-term move; deferred to its own PR so this one stays narrowly
scoped to the auto-instrument crash fix.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@changliu2
Chang Liu (changliu2) merged commit 6294124 into main May 8, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants