Skip to content

perf(openai): optimized turn session — background prewarm and prefix telemetry (PR8)#723

Merged
gnanam1990 merged 9 commits into
mainfrom
perf/openai-optimized-session
Jul 18, 2026
Merged

perf(openai): optimized turn session — background prewarm and prefix telemetry (PR8)#723
gnanam1990 merged 9 commits into
mainfrom
perf/openai-optimized-session

Conversation

@Vasanthdev2004

@Vasanthdev2004 Vasanthdev2004 commented Jul 18, 2026

Copy link
Copy Markdown
Collaborator

Summary

Implements PR8 of the Zero Agent Performance Program: one optimized provider turn-session, for the official OpenAI path only, behind the TurnSession seam from #720. Feature-gated and disabled by default (ZERO_OPENAI_TURN_SESSION, off unless set truthy; 0/false never enable).

What the optimized session does

  • Background connection prewarm at run start: one bounded, unauthenticated HEAD probe to the provider base URL, launched in a goroutine so the TCP+TLS handshake lands in the shared connection pool while the loop assembles the first prompt. One attempt, 3s cap, no retries, no bearer token on the probe, status irrelevant (a 401/404/405 still completes the handshake). The probe is issued directly through the client — not via the retry helper — so it stamps only its own provider_prewarm span and can never contaminate the provider_connect A/B metric (regression-tested). When the transport cannot retain idle connections (the macOS shared transport disables keep-alives), the probe is skipped entirely rather than spent — no request, no trace stamps (tested). Advisory by contract — it can never fail or delay a run.
  • Request-parameter stability telemetry: a per-session fingerprint over the wire-affecting request parameters — the session's model and max-tokens, the reasoning effort as it would appear on the wire, the prompt-cache key, and an order-preserving digest of the advertised tools (request serialization preserves tool order, so a reorder is real drift; descriptions are hashed too) — emitting prefix_stable / prefix_drift counters. Messages are excluded (they grow every turn; this fingerprints request parameters, not conversation content). Scope, stated plainly: this is per-provider telemetry, not a complete compatibility fingerprint — it deliberately does not incorporate the prompt-prefix hashes (complete_prefix) the agent loop already emits, and any future stateful response reuse must combine both signals plus the remaining wire-affecting fields before reusing provider-side state. On chat completions there is no server-side response state to invalidate, so drift is telemetry-only today.
  • Stream is Provider.StreamCompletion verbatim. The session never changes a request.

Gate and allowlist

providers.OptimizedTurnSessions returns the optimized session only when: the env gate is on, the resolved provider kind is official OpenAI (explicitly not openai-compatible gateways — the constructor merges the two kinds, so the gate branches on the resolved kind), the profile is not the ChatGPT catalog, and the provider value is the concrete OpenAI provider (a fake, chatgpt-flavored, or nil provider falls back safely). Everything else — and every run with the gate off — leaves Options.TurnSessionProvider nil, so the loop executes literally today's code path, not an equivalent one.

Wiring is headless-exec only (the benchmarkable surface): exec sets TurnSessionProvider and, when escalation is enabled and the run start is optimized, a ModelSessionSwitcher that re-evaluates eligibility per switched model and falls back to the default adapter otherwise. The TUI is deliberately untouched — a mechanical follow-up once measured.

Performance expectations (no unmeasured claims)

Expected effect is confined to the first request of a run on Linux/Windows: the handshake overlaps prompt assembly, so the first provider_connect span should shrink by roughly one TCP+TLS round-trip set. Two honest caveats, documented in code:

  • macOS skips the probe entirely: the shared transport deliberately disables keep-alives there (degraded pooled connections are indistinguishable from backend slowness), so a probed connection could never be reused — the session detects the non-reusable transport and spends nothing.
  • The pool's 30s idle timeout bounds the warm window; a slower start degrades to a cold dial plus one cheap background HEAD — never worse than today.

Of the plan's four configs: "reused connection" is already the baseline (the shared transport pools connections); "previous-response reuse" is inapplicable on chat completions (previous_response_id is a Responses-API field, which the allowlist excludes) — both documented rather than built, so no retention-semantics change exists in this PR and the privacy opt-in rule is satisfied by omission.

A/B on identical tasks: run the same zero-perf-bench turn suite with ZERO_OPENAI_TURN_SESSION unset vs =1 (the child zero exec --trace inherits the environment) and compare the first-request provider_connect span; optimized traces additionally carry provider_prewarm + prewarm_attempts=1.

Safety properties

Preserves permissions, sandboxing, secret scrubbing, output ceilings, transcript ordering, one result per tool call, reconnect/stall-retry behavior, and cancellation. Gate off ⇒ nil session field ⇒ the loop's default-adapter path, byte-identical. The prewarm probe carries no credentials. No new dependencies.

Verification

  • go build ./..., go vet ./..., gofmt clean on all touched files
  • Full suites green: internal/trace, internal/zeroruntime, internal/providers (incl. the 180s openai suite), internal/agent, targeted internal/cli
  • New tests: prewarm sends exactly one HEAD (even on 405; a second Prewarm never re-probes), connect-failure is non-fatal, trace stamps land under provider_prewarm and a regression asserts the probe never stamps provider_connect; the probe is skipped on keep-alive-disabled transports (zero requests, zero counters — the macOS case), and probe-expecting tests pin reusable transports so all three platforms take the same test path; Stream delegates verbatim (SSE-verified against the direct provider); fingerprint ignores message growth but drifts on cache-key change, tool-set change, reasoning-effort change, description-only change, and tool reorder (wire order is preserved, not sorted), and an effort the wire omits fingerprints like no effort; stable/drift counters; gate default-off, falsey/truthy parsing, official-OpenAI-only, rejects compatible/chatgpt-catalog/foreign-provider values, refuses on capability-resolution failure; the switched-model fallback preserves the switched profile's resolved capabilities; end-to-end exec test proving the server sees the prewarm HEAD plus the turn's POST under the gate, and fallback-to-default for non-OpenAI providers with the gate on
  • Existing escalation and switcher wiring tests pass unchanged

Dependencies

Summary by CodeRabbit

  • New Features
    • Added an experimental optimized OpenAI turn-session mode for headless zero exec, enabled with ZERO_OPENAI_TURN_SESSION=1.
    • Prewarms connections to improve startup performance and records request-prefix stability telemetry.
    • Supports optimized session handling when switching models during escalation.
  • Bug Fixes
    • Automatically falls back to the standard execution path for unsupported providers or profiles.
  • Documentation
    • Documented configuration, supported values, default behavior, and benchmarking guidance.

@github-actions

github-actions Bot commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

Zero automated PR review

Verdict: No blockers found

Blockers

  • None found.

Validation

  • [pass] Diff hygiene: git diff --check
  • [pass] Tests: go test ./...
  • [pass] Build: go run ./cmd/zero-release build
  • [pass] Smoke build: go run ./cmd/zero-release smoke

Scope

Head: 37ea2a7be5d7
Changed files (8): README.md, internal/cli/exec.go, internal/cli/exec_turn_session_test.go, internal/providers/openai/session.go, internal/providers/openai/session_test.go, internal/providers/turn_session.go, internal/providers/turn_session_gate_test.go, internal/trace/trace.go

This deterministic review checks validation status and basic diff hygiene. A human reviewer still owns product judgment and design quality.

@coderabbitai

coderabbitai Bot commented Jul 18, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: f930f5ab-6d6a-471a-bb84-a09f9348ee87

📥 Commits

Reviewing files that changed from the base of the PR and between 95fad4f and 37ea2a7.

📒 Files selected for processing (3)
  • internal/cli/exec.go
  • internal/providers/turn_session.go
  • internal/providers/turn_session_gate_test.go
🚧 Files skipped from review as they are similar to previous changes (2)
  • internal/cli/exec.go
  • internal/providers/turn_session_gate_test.go

Walkthrough

Adds an opt-in OpenAI turn-session provider for headless zero exec, including connection prewarming, prefix telemetry, model escalation switching, eligibility checks, trace events, documentation, and end-to-end coverage.

Changes

OpenAI turn-session optimization

Layer / File(s) Summary
Session gate and trace contracts
internal/providers/turn_session.go, internal/trace/trace.go, internal/providers/turn_session_gate_test.go, README.md
Adds the ZERO_OPENAI_TURN_SESSION gate, official OpenAI eligibility checks, default-session fallback, trace keys, selector tests, and configuration documentation.
OpenAI session behavior
internal/providers/openai/session.go, internal/providers/openai/session_test.go
Adds one-time asynchronous HEAD prewarming, streamed request delegation, deterministic prefix fingerprinting, stability/drift counters, unsupported compaction, and idempotent close behavior with focused tests.
Exec wiring and validation
internal/cli/exec.go, internal/cli/exec_turn_session_test.go
Passes optimized sessions and escalation switching into agent.Run, and tests optimized and fallback provider paths.

Estimated code review effort: 4 (Complex) | ~45 minutes

Possibly related PRs

  • Gitlawb/zero#515: Adds the PromptCacheKey request plumbing used by prefix fingerprinting.
  • Gitlawb/zero#720: Provides the turn-session and model-switching seams used by this implementation.
  • Gitlawb/zero#700: Adds related provider prewarm and prefix tracing infrastructure.

Suggested reviewers: gnanam1990, anandh8x, kevincodex1

Sequence Diagram(s)

sequenceDiagram
  participant CLI
  participant runExec
  participant OptimizedTurnSessions
  participant Agent
  participant OpenAIProvider

  CLI->>runExec: execute headless command
  runExec->>OptimizedTurnSessions: resolve eligible session
  OptimizedTurnSessions->>OpenAIProvider: validate provider and capabilities
  runExec->>Agent: pass session provider and model switcher
  Agent->>OpenAIProvider: prewarm and stream completion
Loading
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 18.18% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: optimized OpenAI turn sessions with background prewarm and prefix telemetry.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch perf/openai-optimized-session

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@internal/cli/exec_turn_session_test.go`:
- Around line 72-79: Update the polling deadline in the prewarm probe test to
exceed the production prewarmTimeout bound of 3 seconds, while preserving the
existing polling loop and exact-one-probe assertion.

In `@internal/providers/openai/session.go`:
- Around line 154-163: Update canonicalToolsDigest to include each tool’s
description alongside its name and parameters when constructing the fingerprint,
so description-only changes affect prefix_stable. Add a regression test covering
two otherwise identical tools with different descriptions and verify their
digests differ.

In `@internal/providers/turn_session.go`:
- Around line 48-52: The optimized-session fallback in
internal/providers/turn_session.go around resolveCapabilities must not construct
a provider with empty capabilities; return (nil, false) when capability
resolution fails so callers retain the default path. In internal/cli/exec.go
around the switched-model fallback session provider, resolve the switched
model’s capabilities before constructing the provider and preserve that
capability projection.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: dc1817cb-d875-483c-9188-d7299148c037

📥 Commits

Reviewing files that changed from the base of the PR and between 30e2c3f and 08904df.

📒 Files selected for processing (8)
  • README.md
  • internal/cli/exec.go
  • internal/cli/exec_turn_session_test.go
  • internal/providers/openai/session.go
  • internal/providers/openai/session_test.go
  • internal/providers/turn_session.go
  • internal/providers/turn_session_gate_test.go
  • internal/trace/trace.go

Comment thread internal/cli/exec_turn_session_test.go
Comment thread internal/providers/openai/session.go Outdated
Comment thread internal/providers/turn_session.go
@Vasanthdev2004

Copy link
Copy Markdown
Collaborator Author

Addressed in 35dfe9fd — two fixes as suggested, one half deliberately kept as-is:

  • Tool descriptions now feed the prefix digest. Right call — descriptions are model-visible request bytes, so a description-only change is genuine prefix drift. canonicalToolsDigest renders name + description + parameters, and TestTurnSessionFingerprintDriftOnToolDescription covers exactly the two-identical-tools-different-descriptions case.
  • Gate now refuses on capability-resolution failure. OptimizedTurnSessions returns (nil, false) instead of shipping an optimized session with empty capabilities. It's effectively unreachable (resolveProfile has already succeeded two lines up), but the default path is the right answer for an unknown-caps state.
  • Test poll deadline raised to 5s, above the 3s prewarmTimeout, so a slow probe can't flake the exactly-one-HEAD assertion.

The one part I'm keeping: the exec fallback wrap for a switched model stays zero-caps. That fallback fires only when the switched model is not eligible for the optimized session, and it mirrors the agent loop's own legacy-switcher wrap, which also carries zero capabilities — the two fallback paths should stay consistent, and nothing consumes capabilities on that path today. Resolving caps there would also mean a second registry resolve per swap for parity with a path that deliberately reports "unknown". Happy to revisit when something actually reads capabilities on the fallback path.

CI running on the new head.

@gnanam1990 gnanam1990 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed at 08904df. Requesting changes for the following verified issues:

  1. internal/providers/openai/session.go: the prewarm uses providerio.SendWithRetry, which also records provider_connect. Optimized traces therefore combine the HEAD prewarm with the completion POST, so the documented A/B provider_connect comparison cannot isolate the claimed improvement. Keep the probe under its dedicated provider_prewarm span without contaminating provider_connect.

  2. internal/providers/turn_session.go and internal/cli/exec.go: capability-resolution failures and switched-model fallback sessions currently use empty capabilities. Preserve the resolved projection, or safely retain the default path when resolution fails.

  3. internal/providers/openai/session.go: canonicalToolsDigest omits tool descriptions even though descriptions are serialized on the OpenAI request. Include descriptions and add a description-only drift regression test.

  4. internal/providers/openai/session.go: macOS still sends the extra HEAD probe although the shared transport disables keep-alive reuse there. That creates overhead with no reusable connection and conflicts with the stated no-regression expectation. Skip the ineffective probe on that transport/platform.

  5. internal/cli/exec_turn_session_test.go: the two-second polling deadline is shorter than the production three-second prewarm bound. Use a deadline longer than the bound; the package-private prewarmTimeout constant cannot be referenced directly from the CLI package.

Focused race tests, vet, build, formatting, and diff checks passed. All GitHub checks are green. The reported lint findings are pre-existing and outside this PRs changed files.

coderabbitai[bot]
coderabbitai Bot previously approved these changes Jul 18, 2026

@anandh8x anandh8x left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed through head 35dfe9f. The focused session/gate race tests, vet, and diff hygiene pass, and the description/capability/test-bound fixes are sound. Requesting changes for four remaining PR8 blockers:\n\n1. calls using the traced run context. That helper records , so an optimized run emits one connect span for the HEAD probe plus another for the completion POST. samples every span by name, which contaminates the exact A/B metric this PR says to compare. Keep the probe solely under (for example, issue the single HEAD without the provider-connect instrumentation) and add a trace regression asserting prewarm does not increment/add .\n\n2. The compatibility fingerprint excludes and sorts the tool slice. Both can change the actual OpenAI wire request; tool order is preserved by request serialization, and the split explicitly lists prompt-cache key in . Because is per-session, the cross-session rationale for excluding the cache key does not apply. Include the cache key, preserve wire tool order, and replace the current ignore/order-insensitive tests with drift regressions.\n\n3. On macOS the default transport disables keep-alives, yet the optimized path still launches the extra HEAD that cannot be reused. Skip the prewarm when the effective transport cannot retain idle connections (and test it), rather than adding known overhead under a performance flag.\n\n4. The program release gates require benchmark results for performance-changing PRs: baseline vs optimized P50/P95 plus success/verification rate on identical tasks. The PR currently gives only instructions and an expected effect. Please attach the actual A/B results after fixing the trace contamination; the performance-smoke job does not exercise/prove this gated path.\n\nAlso, PR8's planned includes the instruction/tools hashes established by PR2. If this partial per-provider fingerprint intentionally remains telemetry-only, narrow its naming/claims accordingly; it must not be presented as sufficient compatibility protection for future response-state reuse without incorporating the existing complete-prefix signal and remaining wire-affecting fields.

@anandh8x

Copy link
Copy Markdown
Collaborator

Clarification for my requested-changes review: the identifiers stripped by formatting are Prewarm, providerio.SendWithRetry, provider_connect, provider_prewarm, PromptCacheKey, RequestPrefix, and lastFingerprint. Specifically, Prewarm currently calls providerio.SendWithRetry with the traced context, adding an unwanted provider_connect sample; and RequestPrefix compatibility must treat PromptCacheKey and wire tool order as drift. The four requested changes and validation summary otherwise rendered correctly.

@Vasanthdev2004

Copy link
Copy Markdown
Collaborator Author

Thanks both — the reviews converged on real problems and b0905a62 addresses them. Point by point:

Trace contamination (both of you, #1): fixed. The probe no longer goes through the retry helper — it issues one client.Do directly, so the only stamp is provider_prewarm. TestTurnSessionPrewarmDoesNotStampProviderConnect is the regression you asked for: it fails if the probe ever adds a provider_connect span.

macOS overhead (both): fixed by skipping, not documenting. transportRetainsIdleConns inspects the client's transport; when keep-alives are disabled (the macOS shared transport) the probe is skipped entirely — no request, no counter, no span. TestTurnSessionPrewarmSkippedWhenKeepAlivesDisabled proves a keep-alive-disabled transport receives zero probe requests.

Fingerprint semantics (anand #2): you're right on both counts, fixed. The cross-session rationale for excluding prompt_cache_key doesn't hold for a per-session comparator, and it is a serialized request field — it's in the digest now, per the split's RequestPrefix list. Tool order is preserved (the wire serializes tools in order, so a reorder is real drift) and descriptions were already added last round. The ignore/order-insensitive tests are replaced with drift regressions: cache-key drift, reorder drift, description drift.

Naming/claims (anand, last point): narrowed. The fingerprint doc and the PR body now state explicitly that this is per-provider request-parameter telemetry, NOT a complete compatibility fingerprint — it deliberately does not incorporate the complete_prefix prompt-hash signal, and any future stateful reuse must combine both plus the remaining wire-affecting fields before touching provider-side state.

Capabilities (gnanam #2): the gate half landed in the previous commit — capability-resolution failure now returns the default path instead of an empty-caps optimized session (your "or safely retain the default path" branch). The exec fallback wrap keeps zero caps deliberately: it only fires for switch targets ineligible for optimization and mirrors the agent loop's own legacy wrap; happy to revisit when something consumes capabilities on that path.

Benchmark results (anand #4): the one open item. Agreed the gate requires numbers, and with the trace contamination fixed the A/B is now clean to run. A real measurement needs a live official-OpenAI account run of the bench suite (the performance-smoke job doesn't exercise this gated path, as you note) — I'll run baseline vs ZERO_OPENAI_TURN_SESSION=1 on identical tasks and attach P50/P95 for the first-request provider_connect plus success rate, and the PR stays gated-off regardless until those numbers justify a default. Keeping the changes-requested state until that lands is fair.

coderabbitai[bot]
coderabbitai Bot previously approved these changes Jul 18, 2026
@Vasanthdev2004

Copy link
Copy Markdown
Collaborator Author

@anandh8x — with the trace contamination fixed in b0905a62, the A/B is clean to measure. I don't have official-OpenAI credits on my side; since the numbers are the remaining gate, could you (or whoever holds the OpenAI account) run the A/B and attach the results?

Exact procedure, both runs on identical tasks:

# baseline
go run ./cmd/zero-perf-bench turn --suite <tasks.json> --model <official-openai-model> --binary ./zero

# optimized
ZERO_OPENAI_TURN_SESSION=1 go run ./cmd/zero-perf-bench turn --suite <tasks.json> --model <official-openai-model> --binary ./zero

Compare: first-request provider_connect P50/P95, plus success rate. The optimized traces should additionally carry provider_prewarm + prewarm_attempts=1 and — with the fix — provider_connect counts identical between configs except for duration. The nav suite is enough to bound cost; happy to rerun a bigger suite if the first numbers look interesting.

Everything else from both reviews is resolved on the current head; once the numbers are attached this should be ready for a re-pass.

coderabbitai[bot]
coderabbitai Bot previously approved these changes Jul 18, 2026

@gnanam1990 gnanam1990 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed at 95fad4f. The macOS probe-test regression is fixed correctly: probe-expecting tests now pin reusable transports and the focused race suite passes locally.

Three blockers remain:

  1. internal/cli/exec.go:439 still constructs the switched-model fallback session with an empty ProviderCapabilities projection. The earlier review thread covered both the initial and switched paths, but only the initial OptimizedTurnSessions failure path was corrected. Preserve the switched models resolved capabilities or safely retain the appropriate default path.

  2. The performance-program release evidence is still missing. Please attach baseline-versus-optimized P50/P95 and success/verification-rate results for identical tasks after the trace fix; instructions and the performance-smoke check do not demonstrate the gated paths effect.

  3. The PR descriptions Verification section is stale: it still says the fingerprint ignores cache keys and is tool-order-insensitive, contradicting the current implementation and tests. Update it to describe cache-key drift, wire-order preservation, description hashing, and the macOS skip behavior.

Focused race tests, vet, build, formatting, diff hygiene, and changed-package lint pass locally. The newest platform CI run was still in progress at review time.

@Vasanthdev2004

Copy link
Copy Markdown
Collaborator Author

All three addressed:

  1. Switched-model fallback capabilities: done (internal/providers/turn_session.go + internal/cli/exec.go). You and the bot both asked and you're right that the thread covered both paths — I've stopped defending the zero-caps wrap. New providers.DefaultTurnSessions resolves the switched profile's capability projection and wraps in the default adapter; the exec fallback now uses it, so an ineligible switch target keeps its resolved capabilities. Resolution failure degrades to an unknown projection rather than blocking the swap (the default session has no capability-dependent behavior, and a swap must never fail on a projection error). TestDefaultTurnSessionsPreservesResolvedCapabilities covers it.

  2. Benchmark evidence: with anand per maintainer decision. The A/B run needs official-OpenAI credits which I don't hold; the exact reproducible procedure (both configs, identical tasks, what to compare) is in my earlier comment addressed to him. The PR stays gated-off regardless until the numbers justify anything more — happy for the changes-requested state to stand until they're attached.

  3. PR description refreshed. The Verification section now describes the current behavior: cache-key/reorder/description drift with wire-order preservation, the provider_connect isolation regression, the macOS skip (zero requests, zero counters), platform-pinned probe tests, and the fallback capabilities test.

CI running on the new head.

@anandh8x anandh8x left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed current head 95fad4f. The implementation fixes from my prior review are sound: prewarm no longer contaminates provider_connect, non-reusable transports skip the probe, cache-key and wire tool-order changes now drift, and the focused OpenAI TurnSession race suite passes. I am not approving yet because the PR8 release evidence is still missing: attach baseline-versus-optimized first-request provider_connect P50/P95 plus success and verification rates from identical official-OpenAI tasks. The PR description Verification section must also be updated because it still says cache keys are ignored and tools are order-insensitive, contradicting the current code and tests. Once those two items land, this is ready for the final approval pass.

@Vasanthdev2004

Copy link
Copy Markdown
Collaborator Author

@anandh8x — crossing wires slightly: your re-review snapshot was 95fad4f, and two things landed right after it:

  • The PR description is already refreshed — the Verification section now describes cache-key/reorder/description drift with wire-order preservation, the provider_connect isolation regression, and the macOS skip. That item is done.
  • 37ea2a7b also resolves gnanam's switched-model capabilities point (providers.DefaultTurnSessions), so every code finding from all review rounds is now closed on the current head.

That leaves exactly one open item: the benchmark run, and it needs an official-OpenAI account, which I don't have — see my earlier comment with the exact A/B procedure addressed to you. Two ways to close it:

  1. If you (or whoever holds the OpenAI account) can run the two commands there and paste the first-request provider_connect P50/P95 + success rate, I'll fold the table into the PR body.
  2. If you can't run it either, say so and I'll take it back to the maintainer to decide how we get credits — no point leaving the PR wedged on an account question.

Either way the PR ships gated-off, so the numbers gate the claim, not user-facing behavior.

@gnanam1990 gnanam1990 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed at 37ea2a7. No remaining code findings.

The switched-model fallback now preserves the resolved capability projection through providers.DefaultTurnSessions, with focused regression coverage. The PR description has also been refreshed to match the current cache-key, tool-order, description-drift, provider-connect isolation, and macOS skip behavior.

Focused race tests, vet, build, formatting, diff hygiene, and changed-package lint all pass locally.

One release blocker remains: attach baseline-versus-optimized official-OpenAI results for identical tasks, including first-request provider_connect P50/P95 plus success and verification rates. The documented procedure and performance-smoke check do not substitute for measured evidence of the gated path.

Once those results are attached and the final CI run is green, the code is ready for approval.

@gnanam1990 gnanam1990 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed at 37ea2a7. Approved: no remaining code findings. The prewarm isolation, transport gating, fingerprint semantics, capability propagation, cross-platform tests, documentation, and focused validation are sound. Live official-OpenAI A/B benchmark evidence is intentionally deferred to the planned OAuth OpenAI work and is not a merge blocker for this feature-gated, default-off change.

@Vasanthdev2004

Copy link
Copy Markdown
Collaborator Author

@anandh8x — following up on the numbers: please run the A/B yourself on your GPT account. You're the reviewer gating on the evidence and you hold the access; everything else on this PR is done and both of you have confirmed no remaining code findings.

One technical note so the run measures the right path: a ChatGPT-subscription OAuth login resolves to the chatgpt catalog, which this PR's allowlist deliberately excludes — with that profile the gate silently falls back to the default path and the A/B shows nothing. The run needs a plain api.openai.com profile with an API key. The nav suite is small, so the cost is trivial on any pay-as-you-go key.

Procedure is in my earlier comment (baseline unset vs ZERO_OPENAI_TURN_SESSION=1, identical tasks). Paste the two trace outputs — or just the first-request provider_connect P50/P95 and success rate — and I'll fold the table into the PR body for the final pass.

@anandh8x anandh8x left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved at 37ea2a7. All code findings are resolved: prewarm tracing is isolated, non-reusable transports skip the probe, fingerprint telemetry matches cache-key and wire tool-order behavior, switched-model capabilities are preserved, cross-platform coverage is in place, and CI is green. Following the maintainer decision, live official-OpenAI A/B evidence is deferred to the planned OAuth/OpenAI work and is not a merge blocker for this default-off feature gate.

@gnanam1990
gnanam1990 merged commit 60dc84e into main Jul 18, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants