fix: customer-readiness audit on main

[Chang Liu (changliu2)]

Summary

Customer-readiness audit on main before customer distribution. Three parallel audits (sensitive-content, docs UX, CLI/YAML functional) surfaced ~19 issues across P0–P2. This PR addresses all of them, with pip install chosen as the canonical setup path and "33 auto-instrumented frameworks" as the headline reach number.

Net change: 21 files, +211/-385 (4 internal-only docs deleted, 1 file renamed for safety).

What changed

Sensitive content removed (P0)

• Deleted an internal-only access-and-repo-workflow guide from the docs tree.
• Stripped internal commentary from examples/travel_planner_langgraph/eval_config.yaml.
• Renamed examples/phoenix_auto_trace/eval_framework.yaml → eval_framework_template.yaml and added a TEMPLATE-ONLY header banner. The original had a REPLACE_WITH_FRAMEWORK placeholder that crashed with ModuleNotFoundError if a customer ran it as-is.
• Dropped internal-only phrasing from README.md and docs/status-and-roadmap.md.
• Rewrote AGENTS.md prohibitions generically (removed enumeration of internal-only categories).

Setup path normalized (P1)

• Switched docs/quickstart.md, docs/reference/cli.md, examples/README.md, examples/pipes/README.md from uv venv / uv sync / uv run to pip install -e ".[otel,langgraph]" with venv activation, matching the README's canonical flow.
• docs/quickstart.md: clearly mark Phoenix as optional, in a second terminal so customers don't think phoenix serve hangs the pipeline.
• README.md: collapsed the verbose macOS litellm explanation into a Troubleshooting section at the bottom, plus added Windows PYTHONUTF8=1 and Docker-preflight notes.

Doc consolidation (responding to "too many .md docs")

• Deleted docs/glossary.md — terminology bridge folded into docs/concepts.md.
• Deleted docs/ai-agent-onboarding.md — paste-in prompt folded into AGENTS.md as a final "Paste-in prompt for end users" section.
• Deleted docs/targets/otel-agent.md — trace-capture content (Phoenix register snippet, target.trace YAML, "why trace capture matters" rationale) inlined into docs/targets/callable.md.
• Updated all references across README.md, AGENTS.md, docs/targets/model-and-tools.md.

Stale YAML examples (P1)

Aligned 4 docs to current schema documented in CONFIG_REFERENCE.md:

• docs/writing-eval-specs.md: nest judge: under pipeline:.
• examples/concepts/README.md: concept: foo → concept.name: foo.
• examples/phoenix_auto_trace/README.md: nest rollout: under pipeline:.
• CONFIG_REFERENCE.md: minimal example was missing target.model (violated the "exactly one of model/callable/endpoint" rule documented in the same file).

Customer-facing terminology (P2)

• Reframed "OTel-traced agents" → "any agent or multi-agent system" across README.md, docs/targets/callable.md, AGENTS.md. OTel trace capture is positioned as the optional upgrade, not a prerequisite.
• examples/agents/README.md: replaced "p2m" with "Adaptive Eval"; demoted external connector to "advanced — prefer target.callable".
• examples/phoenix_auto_trace/README.md: framework count 28 → 33 (matches main README), with honest note that 28 are demoed and the remaining handful follow the same pip install ... + register(auto_instrument=True) pattern. Softened "proves the spec's core claim §4.4.3" / "zero per-framework maintenance" wording.

Windows reliability (P1)

• examples/phoenix_auto_trace/_tools.py: replaced → with -> in MOCK_FLIGHTS route data and simulate_tool() (these strings are printed to console — the unicode arrow caused UnicodeEncodeError on default Windows codepages).
• examples/phoenix_auto_trace/README.md: replaced → arrows with -> and added $env:PYTHONUTF8 = "1" escape hatch.

Verification

• ✅ 505 unit tests pass (uv run pytest -q, 33s).
• ✅ All 7 example pipeline configs validate via p2m.config.load_config.
• ✅ examples.phoenix_auto_trace._tools imports and simulate_tool() returns expected output.
• ✅ rg "glossary\.md|ai-agent-onboarding\.md|otel-agent\.md|access-and-repo-workflow\.md" returns zero matches across the repo.
• ✅ Phase 1 of the audit (CLI smoke tests) had already verified 11/14 configs pass at 1+1 seeds; this PR doesn't touch any code paths exercised there.

What this PR does NOT include

• Two other open branches (yemingtang/rate-limit-retry, yemingtang/graceful-seed-failure) — should land before final customer cut, tracked separately.
• The original audit also flagged the framework-count claim itself ("33") for honesty review; per direction here, "33 auto-instrumented frameworks" stays as the headline with explanatory note.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

[Jake Present (jakepresent)]

Customer-readiness substance LGTM - sensitive scrubs, pip path, and doc consolidation are all clean.

Body undersells scope: actual diff is 50 files, +2375/-809. A few unmentioned things worth surfacing in the body so reviewers (and security/privacy) see them:

• p2m/core/otel.py + otel_session.py: threading.Lock -> per-loop asyncio.Lock concurrency fix. Looks right - good defensive comment on why threading.Lock would deadlock the event loop with concurrency > 1.
• examples/travel_planner_neurosan/travel_planner_eval.md and _langgraph/travel_planner_eval.md: real semantic edits to the eval specs, not just terminology - quality bullets dropped, safety bullets added.
• New tests/regression/t5_lg_safety.yaml + t5_ns_safety.yaml.
• pyproject.toml: adds arize-phoenix>=15.0.0 as a direct optional dep, lowers OTel api/sdk floor 1.41 -> 1.39. Worth a one-line rationale.

Heads up that the eval-spec edits will reshape my Verification matrix TP-* rows in #15 - I'll re-anchor on whatever this lands as. No blocker.

[Chang Liu (changliu2)]

Thanks for the LGTM on substance Jake Present (@jakepresent). Quick clarifier on scope — the actual PR #21 diff is what the body describes (21 files, +211/-385); the items you flagged as unmentioned all landed on main earlier via PR #13:

File / change	Where it actually landed
p2m/core/otel.py + otel_session.py threading.Lock -> per-loop asyncio.Lock	PR #13 commit dfadadd
examples/travel_planner_{langgraph,neurosan}/travel_planner_eval.md semantic edits	PR #13 commit bd125d4
tests/regression/t5_lg_safety.yaml + t5_ns_safety.yaml	PR #13 commit f63f27c
pyproject.toml: arize-phoenix>=15.0.0 in [otel], OTel api/sdk floor 1.41 -> 1.39	PR #13 commit 5c7a991

You can verify against the GitHub Files-Changed tab on this PR — additions: 211, deletions: 385, changed_files: 21 (no p2m/, tests/regression/, or pyproject.toml entries). My guess is your local working tree had unmerged work or a stale main when you computed the 50-file/+2375 delta.

If that re-anchors things for the Verification matrix in #15, helpful pointers:

• Eval-spec edits: anchor TP-* rows on the bd125d4 versions of those files (already on main).
• OTel concurrency fix rationale lives in the commit message of dfadadd — happy to break it out into docs/ in a follow-up if it'd help your matrix.

No changes to this PR; the delta is intentionally doc/content-only as scoped.

[Jake Present (jakepresent)]

You're right - apologies for the noise. Re-pulled and confirmed: PR #21 is what the body describes (21 files, +211/-385) and the items I flagged all landed via PR #13's merge ~4 hours before my review. My local origin/main was stale when I diffed, so I was looking at PR #13's content layered onto PR #21. Withdrawing the body-vs-scope critique. Thanks for the bd125d4 pointer for the TP-* matrix rows - I'll re-anchor on that. Moving PR #21 to approve.

[Jake Present (jakepresent)]

Approving per the comment above. Customer-readiness work LGTM.