diff --git a/docs/decisions/0011-internal-llm-abstraction.md b/docs/decisions/0011-internal-llm-abstraction.md index b1c4b9e2..b80b6679 100644 --- a/docs/decisions/0011-internal-llm-abstraction.md +++ b/docs/decisions/0011-internal-llm-abstraction.md @@ -2,7 +2,7 @@ - **Status**: Accepted - **Date**: 2026-06-03 -- **Related**: [0004-vercel-ai-sdk-multi-llm.md](0004-vercel-ai-sdk-multi-llm.md) (supersedes), [0003-pure-ts-engine-not-langgraph-python.md](0003-pure-ts-engine-not-langgraph-python.md), [0006-os-keychain-for-api-keys.md](0006-os-keychain-for-api-keys.md), [0018-desktop-execution-and-rust-egress.md](0018-desktop-execution-and-rust-egress.md) (per-host egress + key handling), [0024-agent-first-entry-point-agentsession.md](0024-agent-first-entry-point-agentsession.md) (seam reused by chat-mode agents), [tech-stack.md](../tech-stack.md) +- **Related**: [0004-vercel-ai-sdk-multi-llm.md](0004-vercel-ai-sdk-multi-llm.md) (supersedes), [0003-pure-ts-engine-not-langgraph-python.md](0003-pure-ts-engine-not-langgraph-python.md), [0006-os-keychain-for-api-keys.md](0006-os-keychain-for-api-keys.md), [0018-desktop-execution-and-rust-egress.md](0018-desktop-execution-and-rust-egress.md) (per-host egress + key handling), [0024-agent-first-entry-point-agentsession.md](0024-agent-first-entry-point-agentsession.md) (seam reused by chat-mode agents), [0030-llm-seam-shape-amendment-reasoning-response-format-provider-executed.md](0030-llm-seam-shape-amendment-reasoning-response-format-provider-executed.md) (amends the seam shape), [tech-stack.md](../tech-stack.md) ## Context @@ -33,6 +33,13 @@ The seam is the immovable contract; the adapter implementation behind it is deli > agents call providers through the identical contract, so no vendor SDK type crosses the seam for > sessions either. The seam's types and contract are unchanged. +> Amended 2026-06-07: the seam *shape* grows by three additive features — a reasoning channel, +> `LlmRequest.responseFormat`, and `providerExecuted` — per +> [ADR-0030](0030-llm-seam-shape-amendment-reasoning-response-format-provider-executed.md). The +> ADR-0011 decision is unchanged (an internal, provider-agnostic seam in Relavium/Zod types, no +> vendor SDK type crossing it); only the seam's shape is extended, at the M1 freeze boundary before +> any consumer narrows on it. + Considered options: 1. **Internal abstraction over official provider SDKs (`@relavium/llm`)** — owned seam, thin per-provider adapters, no framework. *Chosen.* diff --git a/docs/decisions/0030-llm-seam-shape-amendment-reasoning-response-format-provider-executed.md b/docs/decisions/0030-llm-seam-shape-amendment-reasoning-response-format-provider-executed.md new file mode 100644 index 00000000..e4850475 --- /dev/null +++ b/docs/decisions/0030-llm-seam-shape-amendment-reasoning-response-format-provider-executed.md @@ -0,0 +1,143 @@ +# ADR-0030: `@relavium/llm` seam-shape amendment — reasoning channel, responseFormat, providerExecuted + +- **Status**: Accepted +- **Date**: 2026-06-07 +- **Related**: [0011-internal-llm-abstraction.md](0011-internal-llm-abstraction.md) (the seam ADR this amends), [0029-tool-policy-hardening.md](0029-tool-policy-hardening.md) (the same "tighten the contract before it has consumers" move; the tool allowlist that makes `providerExecuted` matter), [0006-os-keychain-for-api-keys.md](0006-os-keychain-for-api-keys.md), [../reference/shared-core/llm-provider-seam.md](../reference/shared-core/llm-provider-seam.md) (the seam's one canonical home), [../standards/error-handling.md](../standards/error-handling.md) + +## Context + +The `@relavium/llm` seam — the request/result/stream/usage/content shapes in +[`packages/llm/src/types.ts`](../../packages/llm/src/types.ts) and +[`packages/shared/src/content.ts`](../../packages/shared/src/content.ts) — is the immovable contract +of [ADR-0011](0011-internal-llm-abstraction.md). The three Phase-1 adapters (Anthropic, the shared +OpenAI/DeepSeek adapter, Gemini) now pass the shared conformance suite, so the seam is at the **M1 +freeze boundary**. Crucially, **no consumer beyond the adapters exists yet** — the `FallbackChain` +(1.K), the engine (`AgentRunner`/`WorkflowEngine`, 1.O/1.N), the session layer (1.V–1.Z) and the +surfaces are all unbuilt. This is the same situation [ADR-0029](0029-tool-policy-hardening.md) acted +on: a contract change is nearly free before it has consumers and a breaking change after. + +The ADR-0011 seam rule (recorded in [llm-provider-seam.md](../reference/shared-core/llm-provider-seam.md)) +distinguishes two kinds of change: **adding a provider id is additive** (no ADR — it alters no +existing type); **changing the seam *shape*** — the request/result/stream/usage/content types — is a +real amendment that requires an ADR. Three shape gaps were assessed against the actual pinned SDKs +and the already-merged contracts. Each is a genuine **cross-provider** concern the current seam +cannot express, not a single-provider quirk (those go through `providerOptions` + a capability flag): + +1. **Reasoning is advertised but undeliverable.** `CapabilityFlags.reasoning` exists and is set + `true` for Anthropic/Gemini/DeepSeek, yet the seam has **no channel** to carry the reasoning it + promises, so all three adapters silently drop it today. The three providers expose reasoning in + three incompatible native shapes (Anthropic first-class `thinking`/`signature` block-deltas; + Gemini `thought`-flagged parts with a base64 `thoughtSignature`; DeepSeek/Kimi an untyped + `reasoning_content` field over the OpenAI-compatible wire). Reasoning text can only reach a + consumer today as a vendor-shaped blob on `LlmResult.raw` — re-introducing exactly the + vendor-coupling ADR-0011 exists to prevent, and making the ephemeral-signature guarantee + unenforceable at the seam. + +2. **`responseFormat` is the missing mechanism for an already-merged contract.** `output_schema` is + already shipped on `agent`/`transform` nodes ([`packages/shared/src/node.ts`](../../packages/shared/src/node.ts) + `OutputSchemaSchema`), but the LLM seam has no way to ask a model for structured output, so the + node feature is unimplementable until the seam can carry it. + +3. **`providerExecuted` distinguishes server-run tools from engine-run tools.** Providers increasingly + run tools on their own side (Anthropic `web_search`/`code_execution`, Gemini `googleSearch`, + OpenAI Responses built-ins). Without a discriminator, the engine `ToolDispatcher` (1.T, allowlist + [ADR-0029](0029-tool-policy-hardening.md)) cannot tell "I must run this" from "the provider already + ran this" — risking double-execution and mis-applying the engine's tool-permission model to a call + the engine never makes. + +An adversarial assessment refined the urgency: only a change that adds a **member to a discriminated +union** (`StreamChunk` / `ContentPart`) is genuinely breaking-to-add-later, because every consumer's +exhaustive `switch` + `never`-exhaustiveness check breaks at compile time. Adding an **optional +field** (to `LlmRequest`, or to an existing union arm) is backwards-compatible and could in principle +be deferred to the consumer that needs it. We nonetheless settle all three **now**, in one amendment, +because (a) the reasoning channel and the provider-executed stream chunk *are* union-member additions +that must land before consumers narrow on the frozen shape, and (b) doing the one ADR + one seam edit +once — while the only consumers are the three adapters we are already editing — is cheaper and less +error-prone than three separate future amendments, each re-touching every adapter. + +## Decision + +**We will extend the `@relavium/llm` seam shape with three additive features, recorded as an +amendment to (not a supersession of) [ADR-0011](0011-internal-llm-abstraction.md).** ADR-0011's +decision — an internal, provider-agnostic seam in Relavium/Zod types with no vendor SDK type crossing +it — is unchanged; this only grows the seam's shape. The canonical types live in +[llm-provider-seam.md](../reference/shared-core/llm-provider-seam.md); the additions are: + +**1. Reasoning channel (additive).** +- `ContentPart` gains a `reasoning` arm: `{ type: 'reasoning', text, signature?, redacted? }`. +- `StreamChunk` gains `reasoning_start` / `reasoning_delta` / `reasoning_end` (mirroring the + `tool_call_*` triad; `id` correlates deltas to the terminating `reasoning_end`, which carries the + optional `signature`/`redacted`). +- `Usage` gains an optional `reasoningTokens` — **observability only**; the cost math is unchanged + (every provider counts reasoning inside `outputTokens` for billing, so `CostTracker` keeps billing + `outputTokens` whole — `reasoningTokens` is never an additional cost line). + +**2. `LlmRequest.responseFormat` (additive, optional).** A discriminated union +`{ type: 'text' } | { type: 'json', schema, name?, strict? }` (`schema` is the one canonical +`JSONSchema7`). Each adapter lowers `json` to the provider's **native** structured-output mode where +one exists (OpenAI `response_format: json_schema`; Gemini `responseJsonSchema` + JSON mime type; +Anthropic `output_config`/forced tool) — native-vs-forced-tool is a per-adapter implementation detail, +not a seam concern. We deliberately **drop** the opencode `{ type: 'tool' }` variant: "force a +specific tool" is already expressed by `toolChoice: { name }`, so a third variant would be a +redundant second way to force a tool. + +**3. `providerExecuted` (additive).** +- `ContentPart` `tool_call` and `tool_result` gain an optional `providerExecuted?: boolean`. +- `StreamChunk` gains a `tool_result` arm carrying a provider-executed result + (`{ type: 'tool_result', id, name, result, isError?, providerExecuted: true }`) — distinct from the + engine-executed `tool_call_start/delta/end` triad. A `providerExecuted === true` call is **skipped** + by the engine `ToolDispatcher` (1.T): the engine neither runs it nor applies its allowlist to it; + it only records/forwards it. + +**Alternatives weighed.** *(i)* `providerOptions` + the capability flag + `raw` (rejected: +`providerOptions` is request-inbound only and cannot carry reasoning/results back; `raw` is a +vendor-shaped `unknown`, so consumers would pattern-match vendor shapes — the exact coupling ADR-0011 +forbids). *(ii)* Defer all three until their consumers exist (rejected for the union-member additions — +breaking-to-add-later; and bundling the optional fields into the same one-time amendment is cheaper +than three future re-touches of every adapter). *(iii)* A full ~16-member opencode-style event union +with `step-*`, media, audio, citations (rejected: speculative; those are deferrable optional/feature +additions to add with their capability when demanded — this amendment stays minimal). + +**Guardrails (binding).** +- **Reasoning is ephemeral.** A provider-signed reasoning block (`signature`) is a same-provider, + same-turn continuity token. It is **never persisted** to a session, **never replayed across a + provider boundary** (the `FallbackChain`, 1.K, strips reasoning parts when failing over to another + provider), and **never written to a run event or log**. The engine does not interpret it; only the + originating adapter feeds it back. `signature` is an opaque `string` (no `Buffer`/Node type — the + seam stays platform-free, `tsconfig.seam.json` `types: []`). +- **No vendor type crosses the seam.** Each provider's native reasoning/structured-output/server-tool + shape is normalized to these canonical types inside the adapter; `responseFormat` carries one + canonical `JSONSchema7`. +- **`providerExecuted` and the engine tool-security model stay disjoint** — the dispatcher applies its + allowlist only to engine-executed calls; a provider-executed call is never run by the engine. +- **Usage stays NET** ([cost-tracker](../../packages/llm/src/cost-tracker.ts)); `reasoningTokens` is + an extra disjoint observability count, not a new billable class. + +Per-workstream, this lands the **shape** plus the **reasoning + structured-output behavior** wired in +every adapter that supports it (Anthropic/Gemini/DeepSeek reasoning; all three structured output; +OpenAI chat emits no reasoning) with conformance scenarios. `providerExecuted` lands as **shape only** +(no Phase-1 server-tool support is common-path), reserved so 1.T/1.O are born handling it. + +## Consequences + +### Positive + +- The seam is extended at its cheapest possible moment — three adapters, zero downstream consumers — + avoiding a future breaking discriminated-union change + superseding ADR + consumer rework. +- `CapabilityFlags.reasoning` stops being a dangling promise; reasoning reaches the UI/session as a + canonical, vendor-neutral channel with an enforceable ephemerality guarantee. +- `output_schema` becomes implementable; the engine can request structured output through one + canonical field, each adapter using the best native mechanism. +- The engine tool loop is born knowing the difference between a call it must run and one the provider + already ran — no double-execution, no mis-scoped permission. + +### Negative + +- A larger seam surface: more `StreamChunk`/`ContentPart` arms for every future consumer to handle + (mitigated — the additions are minimal and each carries an exhaustiveness obligation that catches + omissions at compile time). +- `providerExecuted` ships as reserved shape with no Phase-1 emitter, i.e. shape ahead of behavior + (accepted deliberately: the union-member reservation is the breaking-to-add-later part). +- The reasoning ephemerality guarantee is a standing correctness/data-handling obligation every later + consumer (fallback, session persistence, run-event logging) must uphold — called out as design + notes on 1.K and 1.Z. diff --git a/docs/decisions/README.md b/docs/decisions/README.md index 7f8fdf21..6a1688f3 100644 --- a/docs/decisions/README.md +++ b/docs/decisions/README.md @@ -73,6 +73,7 @@ flowchart TD | 0027 | [Expression sandbox for `condition` / `transform` / `merge_fn`](0027-expression-sandbox.md) | Accepted | 2026-06-05 | | 0028 | [Workflow resource governance — pre-egress budget, run timeout, concurrency cap](0028-workflow-resource-governance.md) | Accepted | 2026-06-05 | | 0029 | [Tool-policy hardening — command match, tool narrowing, secret interpolation, SSRF](0029-tool-policy-hardening.md) | Accepted | 2026-06-05 | +| 0030 | [`@relavium/llm` seam-shape amendment — reasoning channel, responseFormat, providerExecuted](0030-llm-seam-shape-amendment-reasoning-response-format-provider-executed.md) | Accepted | 2026-06-07 | ## Creating a new ADR diff --git a/docs/reference/shared-core/llm-provider-seam.md b/docs/reference/shared-core/llm-provider-seam.md index 642d6306..12a539da 100644 --- a/docs/reference/shared-core/llm-provider-seam.md +++ b/docs/reference/shared-core/llm-provider-seam.md @@ -39,10 +39,18 @@ interface LlmRequest { temperature?: number; maxTokens?: number; // REQUIRED downstream for Anthropic; we default it stopSequences?: string[]; + responseFormat?: ResponseFormat; // structured-output request (ADR-0030) signal?: AbortSignal; // cancellation; host-injected transport (desktop aborts the Rust llm_stream egress, ADR-0018) providerOptions?: Record; // typed escape hatch (caching, reasoning, etc.) } +// Structured-output contract (ADR-0030). Each adapter lowers `json` to the provider's native mode +// (OpenAI json_schema; Gemini responseJsonSchema; Anthropic output_config; DeepSeek json_object — no +// schema enforcement, so its fidelity is "parseable JSON", not schema-validated). +type ResponseFormat = + | { type: 'text' } + | { type: 'json'; schema: JSONSchema7; name?: string; strict?: boolean }; + interface LlmMessage { role: 'user' | 'assistant' | 'tool'; content: ContentPart[]; // normalized parts, not raw strings @@ -50,8 +58,9 @@ interface LlmMessage { type ContentPart = | { type: 'text'; text: string } - | { type: 'tool_call'; id: string; name: string; args: unknown } // assistant -> wants tool - | { type: 'tool_result'; toolCallId: string; result: unknown; isError?: boolean }; + | { type: 'reasoning'; text: string; signature?: string; redacted?: boolean } // ADR-0030; signature is ephemeral + | { type: 'tool_call'; id: string; name: string; args: unknown; providerExecuted?: boolean } // assistant -> wants tool + | { type: 'tool_result'; toolCallId: string; result: unknown; isError?: boolean; providerExecuted?: boolean }; interface ToolDef { name: string; @@ -74,15 +83,20 @@ interface Usage { outputTokens: number; cacheReadTokens?: number; // Anthropic/DeepSeek expose; others undefined cacheWriteTokens?: number; + reasoningTokens?: number; // ADR-0030 — OBSERVABILITY only; a subset of outputTokens (≤), never billed separately costMicrocents?: number; // integer micro-cents (canonical unit defined below); computed by a pricing table keyed on canonical model id } // Normalized streaming — one discriminated union for ALL providers type StreamChunk = | { type: 'text_delta'; text: string } + | { type: 'reasoning_start'; id: string } // ADR-0030 — reasoning channel + | { type: 'reasoning_delta'; id: string; text: string } + | { type: 'reasoning_end'; id: string; signature?: string; redacted?: boolean } // signature/redacted both surfaced on the stream | { type: 'tool_call_start'; id: string; name: string } - | { type: 'tool_call_delta'; id: string; argsJsonDelta: string } // partial JSON + | { type: 'tool_call_delta'; id: string; argsJsonDelta: string } // partial JSON; count/timing is provider-dependent — accumulate, parse at tool_call_end | { type: 'tool_call_end'; id: string } + | { type: 'tool_result'; id: string; name: string; result: unknown; isError?: boolean; providerExecuted: true } // ADR-0030 — provider-run tool; engine records, never runs | { type: 'stop'; stopReason: StopReason; usage: Usage } | { type: 'error'; error: LlmError }; @@ -173,6 +187,40 @@ would require a real (superseding) ADR is changing the seam shape itself: the request/result/stream types, the normalization rules, or the `LlmError` contract above. +### Seam-shape amendments ([ADR-0030](../../decisions/0030-llm-seam-shape-amendment-reasoning-response-format-provider-executed.md)) + +Three cross-provider shape additions were made under ADR-0030 (a real amendment to +ADR-0011, decided before the seam froze at M1, while the only consumers were the +adapters): + +- **Reasoning channel.** `ContentPart` gains a `reasoning` arm + (`{ type: 'reasoning', text, signature?, redacted? }`); `StreamChunk` gains + `reasoning_start` / `reasoning_delta` / `reasoning_end` (mirroring the + `tool_call_*` triad; `reasoning_end` carries the optional `signature` and + `redacted` flag — both surfaced on the streaming path, symmetric with the + non-streaming `reasoning` content part); `Usage` gains an optional + `reasoningTokens` (**observability only** — already inside `outputTokens` for + billing on Anthropic/OpenAI; on Gemini, thinking tokens are billed *separately* + from candidates, so the adapter sums both into `outputTokens` and surfaces the + thinking subset as `reasoningTokens`). **Reasoning is ephemeral:** a provider-signed + `signature` is never persisted to a session, never replayed across a provider + boundary on fallback, and never written to a run event or log — the engine does + not interpret it; only the originating adapter feeds it back (a same-provider, + same-turn obligation owned by the 1.K `FallbackChain` strip-on-failover, not yet + exercised — no consumer beyond the adapters exists). +- **`responseFormat`** on `LlmRequest` — `{ type: 'text' } | { type: 'json', schema, name?, strict? }`, + one canonical JSON-Schema each adapter lowers to the provider's native + structured-output mode (OpenAI `response_format`, Gemini `responseJsonSchema`, + Anthropic `output_config`). This is the seam mechanism that realizes a node's + `output_schema`. (The opencode `{ type: 'tool' }` variant is deliberately not + adopted — `toolChoice: { name }` already forces a specific tool.) +- **`providerExecuted`** — an optional flag on `ContentPart` `tool_call`/`tool_result` + plus a provider-executed `tool_result` `StreamChunk` arm, distinguishing a tool + the **provider** ran on its own side (server-side/built-in) from one the engine + runs. The engine `ToolDispatcher` skips `providerExecuted` calls (no + double-execution, and the allowlist applies only to engine-run calls). Phase-1 + adapters reserve the shape but emit no server-tool calls (off the common path). + ## What must be normalized The seam's value is entirely in the normalization the adapters perform. Each of diff --git a/docs/roadmap/current.md b/docs/roadmap/current.md index a09cc15b..ee2de125 100644 --- a/docs/roadmap/current.md +++ b/docs/roadmap/current.md @@ -2,7 +2,7 @@ > Status: Living -> Last updated: 2026-06-05 +> Last updated: 2026-06-06 - **Related**: [README.md](README.md), [phases/phase-0-foundations.md](phases/phase-0-foundations.md), [phases/phase-1-engine-and-llm.md](phases/phase-1-engine-and-llm.md), [../project-structure.md](../project-structure.md), [../tech-stack.md](../tech-stack.md) @@ -93,19 +93,22 @@ The next checkpoint is global milestone **M1 — LLM seam proven** (see the ## Immediate next steps -Phase 1 is underway. **Wave 0 — 1.L.0** (`@relavium/shared` reconciliation) merged in **PR #6**, -and the **Wave-1 seam trio — 1.A** (seam types), **1.B** (CostTracker + pricing), **1.E** -(ToolNormalizer) — merged in **PR #7** (2026-06-06): the frozen `LLMProvider` contract, the -integer-micro-cent cost table, and the canonical tool↔wire normalizer — all pure-TS behind the seam -(no provider SDK yet). Per the +Phase 1 is underway. **Wave 0 — 1.L.0** (`@relavium/shared` reconciliation) merged in **PR #6**; +the **Wave-1 seam trio — 1.A** (seam types), **1.B** (CostTracker + pricing), **1.E** (ToolNormalizer) +— merged in **PR #7**; and the **adapter lane — 1.C** (`AnthropicAdapter`), **1.I** (`LlmError` +classification), **1.F** (conformance harness), **1.D** (capabilities + `providerOptions`) — merged in +**PR #8** (2026-06-06): the seam fence now has its **first real consumer** (`@anthropic-ai/sdk` under +`packages/llm/src/adapters/`), proven end-to-end against recorded fixtures by the shared conformance +spec, with classified errors and capability gating — all behind the frozen `LLMProvider` seam. Per the [sequencing plan](phases/phase-1-engine-and-llm.md#sequencing--parallelization), the next work runs two parallel lanes: -1. **Adapter lane (seam)** — **1.C — `AnthropicAdapter`** *(critical path)*, the first adapter and - the seam fence's **first real consumer** (`@anthropic-ai/sdk` lands in - `packages/llm/src/adapters/`), with **1.I — `LlmError` classification** (the fallback contract), - then **1.F — conformance harness** ‖ **1.D — capabilities + `providerOptions`**. This proves the - seam end-to-end and opens 1.G/1.H → **M1** +1. **Adapter lane (seam)** — the remaining two adapters, now that the harness exists: **1.G — + OpenAI-compatible adapter** (OpenAI + DeepSeek via a custom `baseURL`) ‖ **1.H — `GeminiAdapter`** + (`@google/genai`, leaning hardest on the 1.E OpenAPI-subset reshape + id-synthesis). Both bind to + the **1.F** conformance spec; when all three pass it that is **1.J — conformance green = M1**. In + parallel, **1.K — `FallbackChain` runner** (retryable/fatal routing on `LlmError`, per-attempt + usage → `CostTracker`) — the seam's first policy layer ([ADR-0011](../decisions/0011-internal-llm-abstraction.md), [llm-provider-seam.md](../reference/shared-core/llm-provider-seam.md)). 2. **Engine lane** — **1.L — `WorkflowYAMLParser`** *(critical path)* — scaffold `packages/core` and diff --git a/docs/roadmap/phases/phase-1-engine-and-llm.md b/docs/roadmap/phases/phase-1-engine-and-llm.md index a184cece..f0607a27 100644 --- a/docs/roadmap/phases/phase-1-engine-and-llm.md +++ b/docs/roadmap/phases/phase-1-engine-and-llm.md @@ -2,8 +2,17 @@ > Status: In progress — the critical path (Product Phase 1). Wave 0 (**1.L.0**) landed in > **PR #6**; the Wave-1 seam trio — **1.A** (types), **1.B** (CostTracker), **1.E** (ToolNormalizer) -> — landed in **PR #7** (2026-06-06). Next: the adapter lane (**1.C** → {**1.D** ‖ **1.F**}, with -> **1.I**) ‖ the **1.L** parser. +> — landed in **PR #7**; the **adapter lane — 1.C** (`AnthropicAdapter`), **1.I** (`LlmError`), +> **1.F** (conformance harness), **1.D** (capabilities + `providerOptions`) — landed in **PR #8** +> (2026-06-06): the seam is proven end-to-end against a real provider. **In flight (PR after #8):** the +> remaining adapters **1.G** (OpenAI/DeepSeek) ‖ **1.H** (Gemini), bundled with the **seam-shape +> amendment [ADR-0030](../../decisions/0030-llm-seam-shape-amendment-reasoning-response-format-provider-executed.md)** +> (reasoning channel + `responseFormat` + `providerExecuted`) — decided **before the M1 freeze** while +> the only consumers were the adapters; reasoning + structured output are wired in every adapter that +> supports them, with conformance scenarios. Then **1.J** (conformance green = **M1**) and **1.K** +> (FallbackChain — born with the ADR-0030 obligation to **strip the ephemeral reasoning signature when +> failing over** to another provider); ‖ the **1.L** engine parser. *(Session persistence, 1.X/1.Z, +> must exclude the reasoning signature — non-persisting.)* - **Related**: [../README.md](../README.md), [phase-0-foundations.md](phase-0-foundations.md), [phase-2-cli.md](phase-2-cli.md), [../../architecture/shared-core-engine.md](../../architecture/shared-core-engine.md), [../../architecture/execution-model.md](../../architecture/execution-model.md), [../../architecture/multi-llm-providers.md](../../architecture/multi-llm-providers.md), [../../reference/shared-core/llm-provider-seam.md](../../reference/shared-core/llm-provider-seam.md), [../../reference/shared-core/node-types.md](../../reference/shared-core/node-types.md), [../../reference/shared-core/built-in-tools.md](../../reference/shared-core/built-in-tools.md), [../../reference/contracts/sse-event-schema.md](../../reference/contracts/sse-event-schema.md), [../../standards/testing.md](../../standards/testing.md), [../../standards/error-handling.md](../../standards/error-handling.md), [../../decisions/0011-internal-llm-abstraction.md](../../decisions/0011-internal-llm-abstraction.md) @@ -208,7 +217,7 @@ from a provider field. expected micro-cents; an unknown model id raises a typed, user-facing error rather than silently pricing at zero. -### 1.C — `AnthropicAdapter` (the first adapter, proves the seam) — *critical path* +### 1.C — `AnthropicAdapter` (the first adapter, proves the seam) — *critical path* · ✅ **Done (PR #8)** The reference adapter over `@anthropic-ai/sdk`. It establishes the normalization patterns the conformance harness then enforces across all adapters. @@ -234,7 +243,7 @@ patterns the conformance harness then enforces across all adapters. recorded Anthropic fixtures: streams text, calls a tool and returns a normalized `tool_call`, returns usage, maps stop reasons, and surfaces a classified `LlmError`. -### 1.D — Capabilities + the typed `providerOptions` escape hatch +### 1.D — Capabilities + the typed `providerOptions` escape hatch — ✅ **Done (PR #8)** Keep the common path narrow and stable; push provider-specific features off it. @@ -272,7 +281,7 @@ on it. shapes and back; the Gemini reshape rejects an unsupported schema with a typed error and the id-synthesis test proves a stable id across a multi-tool streamed turn. -### 1.F — Conformance harness (shared spec + fixture recorder) — *critical path* +### 1.F — Conformance harness (shared spec + fixture recorder) — *critical path* · ✅ **Done (PR #8)** The single spec every adapter must pass, plus the fixture-recording mechanism. This is the biggest leverage point for the in-house abstraction. @@ -332,7 +341,7 @@ hardest on 1.E. including a tool call whose id is synthesized and a `SAFETY` stop mapped to `content_filter`. -### 1.I — `LlmError` classification (the fallback contract) +### 1.I — `LlmError` classification (the fallback contract) — ✅ **Done (PR #8)** The classification the `FallbackChain` depends on, normalized inside each adapter. @@ -380,10 +389,17 @@ budgets. Adapters stay dumb; this owns the policy. accurate across failover. - Surface the final outcome plus the attempt trace (which providers were tried, why each failed) for the run event/log. +- **Strip the ephemeral reasoning signature on failover** ([ADR-0030](../../decisions/0030-llm-seam-shape-amendment-reasoning-response-format-provider-executed.md) + guardrail): a provider-signed `reasoning` part is a same-provider, same-turn + continuity token. When advancing to a *different* provider, drop every `reasoning` + part (and any carried `signature`) from the request before re-issuing — a signature + is never replayed across a provider boundary. (Within the *same* provider, only the + originating adapter feeds it back.) **Acceptance:** unit tests prove: a primary failing with a retryable error fails over -to the next provider and the run succeeds; a fatal error stops the chain; and -per-attempt cost is summed across a failover. +to the next provider and the run succeeds; a fatal error stops the chain; per-attempt +cost is summed across a failover; and **a cross-provider failover carries no `reasoning` +part or `signature` into the next provider's request**. ### 1.L.0 — Reconcile `@relavium/shared` to the 2026-06-05 contract — ✅ **Done (PR #6)** · *critical path, do first* @@ -517,7 +533,14 @@ Executes a single agent node end-to-end against `@relavium/llm`. `CostTracker`, emitting `cost:updated` (`{ nodeId, model, inputTokens, outputTokens, costMicrocents, cumulativeCostMicrocents }`). - Handle the tool-call loop: dispatch tool calls through the `ToolRegistry` (1.T), - feed results back, and continue until a non-`tool_use` stop. + feed results back, and continue until a non-`tool_use` stop. **Same-provider signed + reasoning** ([ADR-0030](../../decisions/0030-llm-seam-shape-amendment-reasoning-response-format-provider-executed.md)): + within one tool-loop continuation that stays on the *originating* provider, the + signed `reasoning` block must be **preserved and re-fed** to that adapter — Anthropic's + interleaved-thinking continuation rejects a tool-use turn whose prior signed thinking + block was dropped. (Cross-provider failover instead **strips** it — see the 1.K note.) + The adapters currently drop reasoning on lowering; 1.O/1.K owns the same-provider + feedback path. - Thread `AbortSignal` for cancellation; map a final node failure to `node:failed` with a user-safe message + internal correlation id. @@ -653,9 +676,9 @@ These build the `AgentSession` entry point ([ADR-0024](../../decisions/0024-agen - **1.V — `AgentSession` entry point.** Wrap `AgentRunner` in a multi-turn session (session context, one bound agent + its fallback chain). *Acceptance:* a session runs a multi-turn conversation with a tool round-trip through the same `AgentRunner` path a workflow agent node uses. - **1.W — `session:*` event namespace.** Emit session lifecycle events on the shared `RunEventBus` with the same `sequenceNumber` gap/resync logic ([sse-event-schema.md](../../reference/contracts/sse-event-schema.md)). *Acceptance:* session events are disjoint from `run:*` and gap-detected identically. -- **1.X — Session persistence.** `agent_sessions` + `session_messages` via `@relavium/db` into `history.db` ([database-schema.md](../../reference/desktop/database-schema.md)). *Acceptance:* a session round-trips to the DB and resumes. **Note:** adding these two tables requires a regenerated Drizzle migration snapshot (the schema-migration drift CI gate). +- **1.X — Session persistence.** `agent_sessions` + `session_messages` via `@relavium/db` into `history.db` ([database-schema.md](../../reference/desktop/database-schema.md)). *Acceptance:* a session round-trips to the DB and resumes. **Note:** adding these two tables requires a regenerated Drizzle migration snapshot (the schema-migration drift CI gate). **ADR-0030 ephemerality:** a `reasoning` part's `signature`/`redacted` continuity token must **not** be persisted to `session_messages` — strip it (keep reasoning *text* if a transcript needs it, drop the opaque signature). *Acceptance also asserts:* a round-tripped session row carries no reasoning `signature`. - **1.Y — Session checkpoint/resume.** Reuse the idempotency-key logic so a session resumes after a restart. -- **1.Z — Export-to-workflow serializer.** Session → `.relavium.yaml` **linear-chain scaffold + transcript** ([ADR-0026](../../decisions/0026-session-export-to-workflow.md)). Includes a **`WorkflowDefinition` → YAML emitter** (deterministic key ordering, the `metadata` transcript block, secret exclusion) — 1.L is parse-only, so this workstream owns serialization. *Acceptance:* an exported session parses as a valid workflow whose agent nodes mirror the turns; **parse → serialize round-trips** (including `metadata`); no `secret` value is serialized. +- **1.Z — Export-to-workflow serializer.** Session → `.relavium.yaml` **linear-chain scaffold + transcript** ([ADR-0026](../../decisions/0026-session-export-to-workflow.md)). Includes a **`WorkflowDefinition` → YAML emitter** (deterministic key ordering, the `metadata` transcript block, secret exclusion) — 1.L is parse-only, so this workstream owns serialization. *Acceptance:* an exported session parses as a valid workflow whose agent nodes mirror the turns; **parse → serialize round-trips** (including `metadata`); no `secret` value is serialized; and **no reasoning `signature` is serialized** (ADR-0030 ephemerality — the signature is a transient same-provider token, never written to a committable artifact, same exclusion as `secret`). - **1.AA — Node-harness chat regression.** The session counterpart of 1.U: a multi-turn chat with a tool call and an export, run green in CI. ### 1.AB — Expression sandbox (QuickJS-wasm) — *critical path*, folds into 1.P diff --git a/packages/llm/package.json b/packages/llm/package.json index e7a802b2..744c9d11 100644 --- a/packages/llm/package.json +++ b/packages/llm/package.json @@ -29,7 +29,9 @@ }, "dependencies": { "@anthropic-ai/sdk": "catalog:", + "@google/genai": "catalog:", "@relavium/shared": "workspace:*", + "openai": "catalog:", "zod": "catalog:" }, "devDependencies": { diff --git a/packages/llm/src/adapters/anthropic.test.ts b/packages/llm/src/adapters/anthropic.test.ts index 3fe0a0bf..28aaf08b 100644 --- a/packages/llm/src/adapters/anthropic.test.ts +++ b/packages/llm/src/adapters/anthropic.test.ts @@ -413,12 +413,14 @@ describe('AnthropicAdapter — stream edge cases', () => { ev('message_delta', { type: 'message_delta', delta: { stop_reason: 'end_turn', stop_sequence: null }, - // cumulative usage the SDK delivers on the delta — must reach the stop chunk + // cumulative usage the SDK delivers on the delta — must reach the stop chunk, including the + // authoritative thinking count carried in output_tokens_details (ADR-0030). usage: { input_tokens: 10, output_tokens: 5, cache_read_input_tokens: 8, cache_creation_input_tokens: 3, + output_tokens_details: { thinking_tokens: 4 }, }, }) + ev('message_stop', { type: 'message_stop' }) + @@ -436,21 +438,104 @@ describe('AnthropicAdapter — stream edge cases', () => { outputTokens: 5, cacheReadTokens: 8, cacheWriteTokens: 3, + reasoningTokens: 4, // read from the message_delta's output_tokens_details, not dropped }); } }); + + it('emits a transport error (not a clean stop) when the stream ends before message_delta', async () => { + // A stream cut after some content but before the terminal message_delta — must surface as an + // error, never a successful stop that hides the truncation. + const body = + ev('message_start', { + type: 'message_start', + message: { + id: 'm', + type: 'message', + role: 'assistant', + model: 'm', + content: [], + stop_reason: null, + stop_sequence: null, + usage: { input_tokens: 10, output_tokens: 1 }, + }, + }) + + ev('content_block_start', { + type: 'content_block_start', + index: 0, + content_block: { type: 'text', text: '' }, + }) + + ev('content_block_delta', { + type: 'content_block_delta', + index: 0, + delta: { type: 'text_delta', text: 'partial' }, + }) + + '\n'; + const adapter = createAnthropicAdapter({ + fetch: () => Promise.resolve(sse(body)), + maxRetries: 0, + }); + const chunks = await collect(adapter.stream(REQ, 'k')); + expect(chunks.some((c) => c.type === 'text_delta')).toBe(true); + const last = chunks.at(-1); + expect(last?.type).toBe('error'); + if (last?.type === 'error') { + expect(last.error.kind).toBe('transport'); + expect(last.error.retryable).toBe(true); + } + }); + + it('carries the redacted flag onto a streamed reasoning_end (asymmetry fix)', async () => { + const body = + ev('message_start', { + type: 'message_start', + message: { + id: 'm', + type: 'message', + role: 'assistant', + model: 'm', + content: [], + stop_reason: null, + stop_sequence: null, + usage: { input_tokens: 5, output_tokens: 1 }, + }, + }) + + ev('content_block_start', { + type: 'content_block_start', + index: 0, + content_block: { type: 'redacted_thinking', data: 'opaque' }, + }) + + ev('content_block_stop', { type: 'content_block_stop', index: 0 }) + + ev('message_delta', { + type: 'message_delta', + delta: { stop_reason: 'end_turn', stop_sequence: null }, + usage: { output_tokens: 3 }, + }) + + ev('message_stop', { type: 'message_stop' }) + + '\n'; + const adapter = createAnthropicAdapter({ + fetch: () => Promise.resolve(sse(body)), + maxRetries: 0, + }); + const chunks = await collect(adapter.stream(REQ, 'k')); + const end = chunks.find((c) => c.type === 'reasoning_end'); + expect(end).toMatchObject({ type: 'reasoning_end', redacted: true }); + }); }); describe('AnthropicAdapter — content mapping + cancellation', () => { - it('mapContent keeps text + tool_use and skips off-common-path blocks (thinking)', () => { + it('mapContent maps thinking → reasoning (with signature) + text + tool_use', () => { // A fixture of the vendor content-block union (ToolUseBlock has extra fields) — cast at the - // test boundary; mapContent only reads type/text/id/name/input. + // test boundary; mapContent reads type/text/id/name/input + thinking/signature. const parts = mapContent([ { type: 'thinking', thinking: 'hmm', signature: 'sig' }, + { type: 'redacted_thinking', data: 'opaque' }, { type: 'text', text: 'hi', citations: null }, { type: 'tool_use', id: 't1', name: 'f', input: { a: 1 } }, ] as Anthropic.ContentBlock[]); expect(parts).toEqual([ + { type: 'reasoning', text: 'hmm', signature: 'sig' }, // ADR-0030 + { type: 'reasoning', text: '', redacted: true }, { type: 'text', text: 'hi' }, { type: 'tool_call', id: 't1', name: 'f', args: { a: 1 } }, ]); @@ -514,3 +599,121 @@ describe('anthropicErrorToLlmError — error-type table (status-less)', () => { expect(anthropicErrorToLlmError(err).kind).toBe('unknown'); }); }); + +describe('AnthropicAdapter — reasoning + structured output (ADR-0030)', () => { + const REQ2 = { + model: 'm', + maxTokens: 8, + messages: [{ role: 'user' as const, content: [{ type: 'text' as const, text: 'hi' }] }], + }; + const ev = (type: string, data: unknown): string => + `event: ${type}\ndata: ${JSON.stringify(data)}\n\n`; + const sse = (body: string): Response => + new Response(body, { status: 200, headers: { 'content-type': 'text/event-stream' } }); + + it('folds thinking blocks into reasoning_start/delta/end carrying the signature', async () => { + const body = + ev('message_start', { + type: 'message_start', + message: { + id: 'm', + type: 'message', + role: 'assistant', + model: 'm', + content: [], + stop_reason: null, + stop_sequence: null, + usage: { input_tokens: 5, output_tokens: 1 }, + }, + }) + + ev('content_block_start', { + type: 'content_block_start', + index: 0, + content_block: { type: 'thinking', thinking: '', signature: '' }, + }) + + ev('content_block_delta', { + type: 'content_block_delta', + index: 0, + delta: { type: 'thinking_delta', thinking: 'let me think' }, + }) + + ev('content_block_delta', { + type: 'content_block_delta', + index: 0, + delta: { type: 'signature_delta', signature: 'sig-abc' }, + }) + + ev('content_block_stop', { type: 'content_block_stop', index: 0 }) + + ev('content_block_start', { + type: 'content_block_start', + index: 1, + content_block: { type: 'text', text: '' }, + }) + + ev('content_block_delta', { + type: 'content_block_delta', + index: 1, + delta: { type: 'text_delta', text: 'answer' }, + }) + + ev('content_block_stop', { type: 'content_block_stop', index: 1 }) + + ev('message_delta', { + type: 'message_delta', + delta: { stop_reason: 'end_turn', stop_sequence: null }, + usage: { output_tokens: 9 }, + }) + + ev('message_stop', { type: 'message_stop' }) + + '\n'; + const adapter = createAnthropicAdapter({ + fetch: () => Promise.resolve(sse(body)), + maxRetries: 0, + }); + const chunks = await collect(adapter.stream(REQ2, 'k')); + expect(chunks.find((c) => c.type === 'reasoning_start')).toMatchObject({ id: 'reasoning-0' }); + expect(chunks.find((c) => c.type === 'reasoning_delta')).toMatchObject({ + id: 'reasoning-0', + text: 'let me think', + }); + const end = chunks.find((c) => c.type === 'reasoning_end'); + expect(end).toMatchObject({ id: 'reasoning-0', signature: 'sig-abc' }); + expect(chunks.some((c) => c.type === 'text_delta')).toBe(true); + }); + + it('mapUsage surfaces thinking tokens as reasoningTokens (billing unchanged)', () => { + expect( + mapUsage({ + input_tokens: 10, + output_tokens: 20, + output_tokens_details: { thinking_tokens: 8 }, + }), + ).toEqual({ inputTokens: 10, outputTokens: 20, reasoningTokens: 8 }); + }); + + it('lowers responseFormat json to output_config', async () => { + let sent: Record = {}; + const adapter = createAnthropicAdapter({ + fetch: (_i, init) => { + sent = parseJsonBody(init); + return Promise.resolve( + new Response( + JSON.stringify({ + id: 'm', + type: 'message', + role: 'assistant', + model: 'm', + content: [{ type: 'text', text: '{}' }], + stop_reason: 'end_turn', + stop_sequence: null, + usage: { input_tokens: 1, output_tokens: 1 }, + }), + { status: 200, headers: { 'content-type': 'application/json' } }, + ), + ); + }, + maxRetries: 0, + }); + await adapter.generate( + { ...REQ2, responseFormat: { type: 'json', schema: { type: 'object' } } }, + 'k', + ); + expect(sent['output_config']).toEqual({ + format: { type: 'json_schema', schema: { type: 'object' } }, + }); + }); +}); diff --git a/packages/llm/src/adapters/anthropic.ts b/packages/llm/src/adapters/anthropic.ts index 3618da59..76d87d04 100644 --- a/packages/llm/src/adapters/anthropic.ts +++ b/packages/llm/src/adapters/anthropic.ts @@ -19,6 +19,8 @@ import type { Usage, } from '../types.js'; +import { isAbortSignal } from './shared.js'; + /** * The reference adapter over `@anthropic-ai/sdk` (1.C) — the seam fence's first real consumer and * the first place a vendor SDK is imported (allowed only under `src/adapters/*`). It establishes the @@ -75,6 +77,7 @@ export function mapUsage(usage: { output_tokens: number; cache_read_input_tokens?: number | null; cache_creation_input_tokens?: number | null; + output_tokens_details?: { thinking_tokens?: number | null } | null; }): Usage { const out: Usage = { inputTokens: usage.input_tokens, outputTokens: usage.output_tokens }; if (usage.cache_read_input_tokens != null) { @@ -83,10 +86,15 @@ export function mapUsage(usage: { if (usage.cache_creation_input_tokens != null) { out.cacheWriteTokens = usage.cache_creation_input_tokens; } + // Thinking tokens are already inside output_tokens (billing unchanged); surface for visibility (ADR-0030). + const thinking = usage.output_tokens_details?.thinking_tokens ?? 0; + if (thinking > 0) { + out.reasoningTokens = thinking; + } return out; } -/** Fold an Anthropic message's content blocks into canonical content parts (text + tool_call). */ +/** Fold an Anthropic message's content blocks into canonical content parts (text + tool_call + reasoning). */ export function mapContent(blocks: readonly Anthropic.ContentBlock[]): ContentPart[] { const parts: ContentPart[] = []; for (const block of blocks) { @@ -96,8 +104,17 @@ export function mapContent(blocks: readonly Anthropic.ContentBlock[]): ContentPa parts.push( normalizeToolCall(PROVIDER, { id: block.id, name: block.name, args: block.input }), ); + } else if (block.type === 'thinking') { + // Reasoning (ADR-0030); the signature is the ephemeral same-provider continuity token. + parts.push( + block.signature.length > 0 + ? { type: 'reasoning', text: block.thinking, signature: block.signature } + : { type: 'reasoning', text: block.thinking }, + ); + } else if (block.type === 'redacted_thinking') { + parts.push({ type: 'reasoning', text: '', redacted: true }); } - // thinking / server-tool blocks are off the common path — reachable via LlmResult.raw. + // other server-tool blocks remain off the common path — reachable via LlmResult.raw. } return parts; } @@ -177,7 +194,11 @@ export function anthropicErrorToLlmError(err: unknown): LlmError { // --- Request building: canonical → Anthropic wire -------------------------------------------- -function toAnthropicBlock(part: ContentPart): Anthropic.ContentBlockParam { +// Reasoning parts are filtered out before this point (ephemeral, not replayed to the wire — ADR-0030), +// so the wire-able content is the closed text / tool_call / tool_result set. +function toAnthropicBlock( + part: Exclude, +): Anthropic.ContentBlockParam { switch (part.type) { case 'text': return { type: 'text', text: part.text }; @@ -195,7 +216,7 @@ function toAnthropicBlock(part: ContentPart): Anthropic.ContentBlockParam { } return block; } - /* v8 ignore next 4 -- defensive: ContentPart is a closed 3-variant union */ + /* v8 ignore next 4 -- defensive: the wire-able content is a closed 3-variant union */ default: { const unreachable: never = part; throw new Error(`unhandled content part: ${String(unreachable)}`); @@ -204,10 +225,15 @@ function toAnthropicBlock(part: ContentPart): Anthropic.ContentBlockParam { } function toAnthropicMessage(message: LlmMessage): Anthropic.MessageParam { - // Anthropic has only user/assistant roles; tool results ride in a user-role message. + // Anthropic has only user/assistant roles; tool results ride in a user-role message. Reasoning + // parts are ephemeral (ADR-0030) and dropped here — they are never replayed to the provider. return { role: message.role === 'assistant' ? 'assistant' : 'user', - content: message.content.map(toAnthropicBlock), + content: message.content + .filter( + (part): part is Exclude => part.type !== 'reasoning', + ) + .map(toAnthropicBlock), }; } @@ -259,6 +285,12 @@ function buildCommonBody( if (req.toolChoice !== undefined) { body.tool_choice = toAnthropicToolChoice(req.toolChoice); } + if (req.responseFormat?.type === 'json') { + // Native structured output via output_config (ADR-0030); the canonical JSON-Schema bridges here. + body.output_config = { + format: { type: 'json_schema', schema: req.responseFormat.schema as Record }, + }; + } if (req.temperature !== undefined) { body.temperature = req.temperature; } @@ -275,11 +307,6 @@ function buildCommonBody( return { ...req.providerOptions, ...body }; } -/** True for a real `AbortSignal` (the host passes one; it structurally satisfies AbortSignalLike). */ -function isAbortSignal(value: unknown): value is AbortSignal { - return typeof AbortSignal !== 'undefined' && value instanceof AbortSignal; -} - /** Bridge the host's `AbortSignalLike` (a real `AbortSignal` at runtime) to the SDK's signal option. */ function buildRequestOptions(req: LlmRequest): { signal?: AbortSignal } { return isAbortSignal(req.signal) ? { signal: req.signal } : {}; @@ -307,6 +334,7 @@ function mergeDeltaUsage( output_tokens: number; cache_read_input_tokens?: number | null; cache_creation_input_tokens?: number | null; + output_tokens_details?: { thinking_tokens?: number | null } | null; }, ): Usage { const merged: Usage = { @@ -321,12 +349,109 @@ function mergeDeltaUsage( if (cacheWrite != null) { merged.cacheWriteTokens = cacheWrite; } + // message_delta carries the authoritative cumulative thinking count (same semantics as output_tokens). + // Fall back to the message_start value only if the delta omits the details field entirely. + const thinking = delta.output_tokens_details?.thinking_tokens ?? prev.reasoningTokens ?? 0; + if (thinking > 0) { + merged.reasoningTokens = thinking; + } return merged; } +/** Per-index reasoning-block state: the synthesized chunk id, accumulating signature, redacted flag. */ +interface ReasoningBlock { + readonly id: string; + signature?: string; + readonly redacted: boolean; +} + /** * Fold one content-block stream event into the `StreamChunk` to emit (or `undefined`), tracking the - * Anthropic tool-call id by content-block index so delta/stop chunks carry the matching id. + * tool-call id and the reasoning block by content-block index so delta/stop chunks carry the matching + * id (and the reasoning signature accumulates onto the terminating `reasoning_end`). ADR-0030. + */ +function handleContentBlockStart( + event: Anthropic.RawContentBlockStartEvent, + toolIdByIndex: Map, + reasoningByIndex: Map, +): StreamChunk | undefined { + const block = event.content_block; + if (block.type === 'tool_use') { + toolIdByIndex.set(event.index, block.id); + return { type: 'tool_call_start', id: block.id, name: block.name }; + } + if (block.type === 'thinking' || block.type === 'redacted_thinking') { + const id = `reasoning-${String(event.index)}`; + reasoningByIndex.set(event.index, { id, redacted: block.type === 'redacted_thinking' }); + return { type: 'reasoning_start', id }; + } + return undefined; +} + +function handleContentBlockDelta( + event: Anthropic.RawContentBlockDeltaEvent, + toolIdByIndex: Map, + reasoningByIndex: Map, +): StreamChunk | undefined { + const delta = event.delta; + if (delta.type === 'text_delta') { + return { type: 'text_delta', text: delta.text }; + } + if (delta.type === 'input_json_delta') { + const id = toolIdByIndex.get(event.index); + return id === undefined + ? undefined + : { type: 'tool_call_delta', id, argsJsonDelta: delta.partial_json }; + } + if (delta.type === 'thinking_delta') { + const reasoning = reasoningByIndex.get(event.index); + return reasoning === undefined + ? undefined + : { type: 'reasoning_delta', id: reasoning.id, text: delta.thinking }; + } + if (delta.type === 'signature_delta') { + const reasoning = reasoningByIndex.get(event.index); + if (reasoning !== undefined) { + // The signature streams incrementally like thinking text — append, don't overwrite. + reasoning.signature = (reasoning.signature ?? '') + delta.signature; + } + return undefined; + } + return undefined; +} + +function handleContentBlockStop( + event: Anthropic.RawContentBlockStopEvent, + toolIdByIndex: Map, + reasoningByIndex: Map, +): StreamChunk | undefined { + const toolId = toolIdByIndex.get(event.index); + if (toolId !== undefined) { + return { type: 'tool_call_end', id: toolId }; + } + const reasoning = reasoningByIndex.get(event.index); + if (reasoning === undefined) { + return undefined; + } + // Carry both the accumulated signature and the redacted flag (asymmetry fix: non-streaming + // mapContent already sets redacted; the stream must too — ADR-0030). + const end: Extract = { + type: 'reasoning_end', + id: reasoning.id, + }; + if (reasoning.signature !== undefined) { + end.signature = reasoning.signature; + } + if (reasoning.redacted) { + end.redacted = true; + } + return end; +} + +/** + * Fold one content-block stream event into the `StreamChunk` to emit (or `undefined`) by delegating + * to the per-phase handlers, which track the tool-call id and reasoning block by content-block index + * so delta/stop chunks carry the matching id (and the reasoning signature accumulates). ADR-0030. */ function contentBlockToChunk( event: @@ -334,40 +459,26 @@ function contentBlockToChunk( | Anthropic.RawContentBlockDeltaEvent | Anthropic.RawContentBlockStopEvent, toolIdByIndex: Map, + reasoningByIndex: Map, ): StreamChunk | undefined { if (event.type === 'content_block_start') { - if (event.content_block.type === 'tool_use') { - toolIdByIndex.set(event.index, event.content_block.id); - return { - type: 'tool_call_start', - id: event.content_block.id, - name: event.content_block.name, - }; - } - return undefined; + return handleContentBlockStart(event, toolIdByIndex, reasoningByIndex); } if (event.type === 'content_block_delta') { - if (event.delta.type === 'text_delta') { - return { type: 'text_delta', text: event.delta.text }; - } - if (event.delta.type === 'input_json_delta') { - const id = toolIdByIndex.get(event.index); - return id === undefined - ? undefined - : { type: 'tool_call_delta', id, argsJsonDelta: event.delta.partial_json }; - } - return undefined; + return handleContentBlockDelta(event, toolIdByIndex, reasoningByIndex); } - // content_block_stop - const id = toolIdByIndex.get(event.index); - return id === undefined ? undefined : { type: 'tool_call_end', id }; + return handleContentBlockStop(event, toolIdByIndex, reasoningByIndex); } /** Fold the Anthropic SSE event stream into the canonical `StreamChunk` sequence. */ async function* streamChunks(client: Anthropic, req: LlmRequest): AsyncIterable { const toolIdByIndex = new Map(); + const reasoningByIndex = new Map(); let usage: Usage = { inputTokens: 0, outputTokens: 0 }; let stopReason: StopReason = 'stop'; + // The message_delta event carries the authoritative stop_reason + final usage; a stream that ends + // without it was truncated and must not be reported as a successful stop. + let sawStop = false; let sdkStream: AsyncIterable; try { sdkStream = await client.messages.create( @@ -385,12 +496,13 @@ async function* streamChunks(client: Anthropic, req: LlmRequest): AsyncIterable< } else if (event.type === 'message_delta') { stopReason = mapStopReason(event.delta.stop_reason); usage = mergeDeltaUsage(usage, event.usage); + sawStop = true; } else if ( event.type === 'content_block_start' || event.type === 'content_block_delta' || event.type === 'content_block_stop' ) { - const chunk = contentBlockToChunk(event, toolIdByIndex); + const chunk = contentBlockToChunk(event, toolIdByIndex, reasoningByIndex); if (chunk !== undefined) { yield chunk; } @@ -401,6 +513,19 @@ async function* streamChunks(client: Anthropic, req: LlmRequest): AsyncIterable< yield { type: 'error', error: anthropicErrorToLlmError(err) }; return; } + // No message_delta arrived → the SSE stream was cut before completion. Surface a retryable + // transport error rather than a clean stop that hides the lost tail. + if (!sawStop) { + yield { + type: 'error', + error: makeLlmError({ + provider: PROVIDER, + kind: 'transport', + message: 'stream ended before message_delta (truncated response)', + }), + }; + return; + } yield { type: 'stop', stopReason, usage }; } diff --git a/packages/llm/src/adapters/gemini.test.ts b/packages/llm/src/adapters/gemini.test.ts new file mode 100644 index 00000000..b5bc9f23 --- /dev/null +++ b/packages/llm/src/adapters/gemini.test.ts @@ -0,0 +1,503 @@ +import { describe, expect, it } from 'vitest'; + +import { GeminiToolCallIds } from '../tool-normalizer.js'; +import type { LlmRequest, StreamChunk } from '../types.js'; +import { + buildGeminiRequest, + createGeminiAdapter, + geminiAdapter, + geminiErrorToLlmError, + mapContent, + mapStopReason, + mapUsage, + type GeminiRequest, + type GeminiResponse, + type GeminiTransport, +} from './gemini.js'; + +async function collect(stream: AsyncIterable): Promise { + const chunks: StreamChunk[] = []; + for await (const chunk of stream) { + chunks.push(chunk); + } + return chunks; +} + +/** A transport that returns a fixed response and captures the request it was handed. */ +function fakeTransport( + response: GeminiResponse, + stream: readonly GeminiResponse[] = [response], +): GeminiTransport & { lastRequest?: GeminiRequest } { + const holder: GeminiTransport & { lastRequest?: GeminiRequest } = { + generate: (request) => { + holder.lastRequest = request; + return Promise.resolve(response); + }, + stream: (request) => { + holder.lastRequest = request; + return Promise.resolve( + (async function* () { + await Promise.resolve(); + for (const item of stream) { + yield item; + } + })(), + ); + }, + }; + return holder; +} + +const REQ: LlmRequest = { + model: 'gemini-2.0-flash', + messages: [{ role: 'user', content: [{ type: 'text', text: 'hi' }] }], +}; + +describe('Gemini adapter', () => { + it('exposes the gemini id and capability surface', () => { + expect(geminiAdapter.id).toBe('gemini'); + expect(geminiAdapter.supports.tools).toBe(true); + expect(geminiAdapter.supports.streaming).toBe(true); + }); + + it('maps finish reasons (STOP+tools → tool_use; SAFETY → content_filter; MALFORMED → error)', () => { + expect(mapStopReason('STOP', false)).toBe('stop'); + expect(mapStopReason('STOP', true)).toBe('tool_use'); + expect(mapStopReason(undefined, true)).toBe('tool_use'); + expect(mapStopReason('MAX_TOKENS', false)).toBe('length'); + expect(mapStopReason('SAFETY', false)).toBe('content_filter'); + expect(mapStopReason('RECITATION', false)).toBe('content_filter'); + expect(mapStopReason('MALFORMED_FUNCTION_CALL', false)).toBe('error'); + expect(mapStopReason('UNEXPECTED_TOOL_CALL', false)).toBe('error'); + expect(mapStopReason('SOMETHING_NEW', false)).toBe('stop'); + }); + + it('maps usage to NET, subtracting cached content from the prompt count', () => { + expect( + mapUsage({ promptTokenCount: 100, candidatesTokenCount: 20, cachedContentTokenCount: 30 }), + ).toEqual({ inputTokens: 70, outputTokens: 20, cacheReadTokens: 30 }); + expect(mapUsage({ promptTokenCount: 12, candidatesTokenCount: 7 })).toEqual({ + inputTokens: 12, + outputTokens: 7, + }); + expect(mapUsage({})).toEqual({ inputTokens: 0, outputTokens: 0 }); + }); + + it('mapContent maps thought parts → reasoning, text, and a synthesized tool call', () => { + const response: GeminiResponse = { + candidates: [ + { + content: { + parts: [ + { text: 'thinking...', thought: true, thoughtSignature: 'sig' }, + { text: 'here' }, + { functionCall: { name: 'get_weather', args: { city: 'Paris' } } }, + ], + }, + }, + ], + }; + const parts = mapContent(response, new GeminiToolCallIds()); + expect(parts[0]).toEqual({ type: 'reasoning', text: 'thinking...', signature: 'sig' }); // ADR-0030 + expect(parts[1]).toEqual({ type: 'text', text: 'here' }); + expect(parts[2]).toMatchObject({ + type: 'tool_call', + name: 'get_weather', + args: { city: 'Paris' }, + }); + if (parts[2]?.type === 'tool_call') { + expect(parts[2].id.length).toBeGreaterThan(0); // synthesized + } + }); +}); + +describe('geminiErrorToLlmError — classification', () => { + it('classifies abort, status-bearing, and unknown throwables', () => { + const abort = Object.assign(new Error('aborted'), { name: 'AbortError' }); + expect(geminiErrorToLlmError(abort)).toMatchObject({ kind: 'cancelled', retryable: false }); + expect(geminiErrorToLlmError({ status: 429, message: 'rate' })).toMatchObject({ + kind: 'rate_limit', + retryable: true, + status: 429, + }); + expect(geminiErrorToLlmError({ status: 401, message: 'auth' })).toMatchObject({ + kind: 'auth', + retryable: false, + }); + expect(geminiErrorToLlmError('boom')).toMatchObject({ kind: 'unknown', retryable: false }); + }); +}); + +describe('Gemini adapter — request building (buildGeminiRequest)', () => { + it('routes system → systemInstruction, tools → functionDeclarations, and tool choice modes', () => { + const request = buildGeminiRequest({ + model: 'gemini-2.0-flash', + system: 'be terse', + temperature: 0.4, + maxTokens: 64, + stopSequences: ['END'], + toolChoice: 'required', + tools: [ + { + name: 'get_weather', + parameters: { type: 'object', properties: { city: { type: 'string' } } }, + }, + ], + messages: [{ role: 'user', content: [{ type: 'text', text: 'hi' }] }], + }); + expect(request.config['systemInstruction']).toBe('be terse'); + expect(request.config['temperature']).toBe(0.4); + expect(request.config['maxOutputTokens']).toBe(64); + expect(request.config['stopSequences']).toEqual(['END']); + expect(request.config['toolConfig']).toEqual({ functionCallingConfig: { mode: 'ANY' } }); + expect(request.config['tools']).toMatchObject([ + { functionDeclarations: [{ name: 'get_weather' }] }, + ]); + expect(request.contents).toEqual([{ role: 'user', parts: [{ text: 'hi' }] }]); + }); + + it('maps a named tool choice to ANY + allowedFunctionNames', () => { + const request = buildGeminiRequest({ ...REQ, toolChoice: { name: 'get_weather' } }); + expect(request.config['toolConfig']).toEqual({ + functionCallingConfig: { mode: 'ANY', allowedFunctionNames: ['get_weather'] }, + }); + }); + + it('threads a real AbortSignal into config.abortSignal', () => { + const controller = new AbortController(); + const request = buildGeminiRequest({ ...REQ, signal: controller.signal }); + expect(request.config['abortSignal']).toBe(controller.signal); + }); + + it('maps none/auto tool choice modes', () => { + expect(buildGeminiRequest({ ...REQ, toolChoice: 'none' }).config['toolConfig']).toEqual({ + functionCallingConfig: { mode: 'NONE' }, + }); + expect(buildGeminiRequest({ ...REQ, toolChoice: 'auto' }).config['toolConfig']).toEqual({ + functionCallingConfig: { mode: 'AUTO' }, + }); + }); + + it('round-trips tool_call → functionCall and tool_result → functionResponse by name', () => { + const request = buildGeminiRequest({ + model: 'gemini-2.0-flash', + messages: [ + { + role: 'assistant', + content: [ + { + type: 'tool_call', + id: 'gemini-tool-0-get_weather', + name: 'get_weather', + args: { city: 'Paris' }, + }, + ], + }, + { + role: 'tool', + content: [ + { type: 'tool_result', toolCallId: 'gemini-tool-0-get_weather', result: { tempC: 18 } }, + ], + }, + ], + }); + expect(request.contents[0]).toEqual({ + role: 'model', + parts: [{ functionCall: { name: 'get_weather', args: { city: 'Paris' } } }], + }); + // the tool result resolves the function name from the matching tool_call id (Gemini has no id) + expect(request.contents[1]).toEqual({ + role: 'user', + parts: [{ functionResponse: { name: 'get_weather', response: { tempC: 18 } } }], + }); + }); + + it('wraps a non-object tool result and lets providerOptions only ADD (mapped fields win)', () => { + const request = buildGeminiRequest({ + model: 'gemini-2.0-flash', + providerOptions: { cachedContent: 'abc', temperature: 99 }, + temperature: 0.1, + messages: [ + { role: 'tool', content: [{ type: 'tool_result', toolCallId: 'x', result: 'plain text' }] }, + ], + }); + expect(request.contents[0]?.parts[0]).toEqual({ + functionResponse: { name: 'x', response: { result: 'plain text' } }, + }); + expect(request.config['cachedContent']).toBe('abc'); // escape-hatch field + expect(request.config['temperature']).toBe(0.1); // mapped field wins over providerOptions + }); + + it('strips httpOptions from providerOptions to prevent SSRF via baseUrl redirect', () => { + const request = buildGeminiRequest({ + ...REQ, + providerOptions: { + httpOptions: { baseUrl: 'https://attacker.example' }, + cachedContent: 'kept', + }, + }); + expect(request.config['httpOptions']).toBeUndefined(); + expect(request.config['cachedContent']).toBe('kept'); // only transport keys are stripped + }); +}); + +describe('Gemini adapter — generate / stream via injected transport', () => { + const textResponse: GeminiResponse = { + candidates: [{ content: { parts: [{ text: 'Hello' }] }, finishReason: 'STOP' }], + usageMetadata: { promptTokenCount: 5, candidatesTokenCount: 2 }, + }; + + it('generate folds a response into an LlmResult', async () => { + const adapter = createGeminiAdapter({ transport: fakeTransport(textResponse) }); + const result = await adapter.generate(REQ, 'k'); + expect(result.content).toEqual([{ type: 'text', text: 'Hello' }]); + expect(result.stopReason).toBe('stop'); + expect(result.usage).toEqual({ inputTokens: 5, outputTokens: 2 }); + }); + + it('generate surfaces a transport rejection as a classified LlmProviderError', async () => { + const transport: GeminiTransport = { + generate: () => Promise.reject(Object.assign(new Error('rl'), { status: 429 })), + stream: () => Promise.reject(new Error('unused')), + }; + const adapter = createGeminiAdapter({ transport }); + let caught: unknown; + try { + await adapter.generate(REQ, 'sk-SECRET-123'); + } catch (err) { + caught = err; + } + expect(caught).toBeInstanceOf(Error); + expect(JSON.stringify(caught)).not.toContain('SECRET'); + }); + + it('stream folds text + a tool call (start/delta/end) then a terminal stop', async () => { + const toolChunk: GeminiResponse = { + candidates: [ + { + content: { parts: [{ functionCall: { name: 'get_weather', args: { city: 'Paris' } } }] }, + finishReason: 'STOP', + }, + ], + usageMetadata: { promptTokenCount: 20, candidatesTokenCount: 15 }, + }; + const adapter = createGeminiAdapter({ transport: fakeTransport(toolChunk, [toolChunk]) }); + const chunks = await collect(adapter.stream(REQ, 'k')); + expect(chunks.find((c) => c.type === 'tool_call_start')).toMatchObject({ name: 'get_weather' }); + expect(chunks.some((c) => c.type === 'tool_call_delta')).toBe(true); + expect(chunks.some((c) => c.type === 'tool_call_end')).toBe(true); + const stop = chunks.at(-1); + expect(stop?.type).toBe('stop'); + if (stop?.type === 'stop') { + expect(stop.stopReason).toBe('tool_use'); + } + }); + + it('stream yields a single error chunk when the transport fails to start', async () => { + const transport: GeminiTransport = { + generate: () => Promise.reject(new Error('unused')), + stream: () => Promise.reject(Object.assign(new Error('overloaded'), { status: 503 })), + }; + const adapter = createGeminiAdapter({ transport }); + const chunks = await collect(adapter.stream(REQ, 'k')); + expect(chunks).toHaveLength(1); + expect(chunks[0]?.type).toBe('error'); + if (chunks[0]?.type === 'error') { + expect(chunks[0].error.kind).toBe('overloaded'); + } + }); +}); + +describe('Gemini adapter — remaining branches', () => { + it('maps all content-filter finish reasons', () => { + for (const reason of ['BLOCKLIST', 'PROHIBITED_CONTENT', 'SPII', 'IMAGE_SAFETY'] as const) { + expect(mapStopReason(reason, false)).toBe('content_filter'); + } + }); + + it('generate tolerates a response with no usage metadata', async () => { + const adapter = createGeminiAdapter({ + transport: { + generate: () => + Promise.resolve({ + candidates: [{ content: { parts: [{ text: 'hi' }] }, finishReason: 'STOP' }], + }), + stream: () => Promise.reject(new Error('unused')), + }, + }); + const result = await adapter.generate(REQ, 'k'); + expect(result.usage).toEqual({ inputTokens: 0, outputTokens: 0 }); + }); + + it('folds a mid-stream iteration error into an error chunk', async () => { + const adapter = createGeminiAdapter({ + transport: { + generate: () => Promise.reject(new Error('unused')), + stream: () => + Promise.resolve( + (async function* () { + await Promise.resolve(); + yield { candidates: [{ content: { parts: [{ text: 'partial' }] } }] }; + throw Object.assign(new Error('mid-stream'), { status: 500 }); + })(), + ), + }, + }); + const chunks = await collect(adapter.stream(REQ, 'k')); + expect(chunks.some((c) => c.type === 'text_delta')).toBe(true); + expect(chunks.at(-1)?.type).toBe('error'); + }); + + it('drops a message that lowers to zero parts', () => { + const request = buildGeminiRequest({ + model: 'gemini-2.0-flash', + messages: [ + { role: 'user', content: [] }, + { role: 'user', content: [{ type: 'text', text: 'hi' }] }, + ], + }); + expect(request.contents).toEqual([{ role: 'user', parts: [{ text: 'hi' }] }]); + }); +}); + +describe('Gemini adapter — reasoning + structured output (ADR-0030)', () => { + it('lowers responseFormat json to responseMimeType + responseJsonSchema', () => { + const request = buildGeminiRequest({ + ...REQ, + responseFormat: { type: 'json', schema: { type: 'object' } }, + }); + expect(request.config['responseMimeType']).toBe('application/json'); + expect(request.config['responseJsonSchema']).toEqual({ type: 'object' }); + }); + + it('mapUsage adds thoughtsTokenCount into outputTokens (Gemini bills them separately)', () => { + expect( + mapUsage({ promptTokenCount: 10, candidatesTokenCount: 20, thoughtsTokenCount: 6 }), + ).toEqual({ inputTokens: 10, outputTokens: 26, reasoningTokens: 6 }); + }); + + it('stream folds thought parts into reasoning_start/delta/end (signature) then text', async () => { + const response: GeminiResponse = { + candidates: [ + { + content: { + parts: [ + { text: 'pondering', thought: true, thoughtSignature: 'sig' }, + { text: 'final answer' }, + ], + }, + finishReason: 'STOP', + }, + ], + usageMetadata: { promptTokenCount: 5, candidatesTokenCount: 4, thoughtsTokenCount: 2 }, + }; + const adapter = createGeminiAdapter({ transport: fakeTransport(response, [response]) }); + const chunks = await collect(adapter.stream(REQ, 'k')); + expect(chunks.find((c) => c.type === 'reasoning_start')).toMatchObject({ id: 'reasoning-0' }); + expect(chunks.find((c) => c.type === 'reasoning_delta')).toMatchObject({ text: 'pondering' }); + expect(chunks.find((c) => c.type === 'reasoning_end')).toMatchObject({ signature: 'sig' }); + const types = chunks.map((c) => c.type); + expect(types.indexOf('reasoning_end')).toBeLessThan(types.indexOf('text_delta')); + const stop = chunks.at(-1); + expect(stop?.type).toBe('stop'); + if (stop?.type === 'stop') { + expect(stop.usage.reasoningTokens).toBe(2); + } + }); +}); + +describe('Gemini adapter — reasoning close edges', () => { + it('closes reasoning before a tool call', async () => { + const r: GeminiResponse = { + candidates: [ + { + content: { + parts: [{ text: 'think', thought: true }, { functionCall: { name: 'f', args: {} } }], + }, + finishReason: 'STOP', + }, + ], + }; + const types = ( + await collect(createGeminiAdapter({ transport: fakeTransport(r, [r]) }).stream(REQ, 'k')) + ).map((c) => c.type); + expect(types.indexOf('reasoning_end')).toBeLessThan(types.indexOf('tool_call_start')); + }); + + it('closes reasoning after a thought-only stream (before stop)', async () => { + const r: GeminiResponse = { + candidates: [ + { content: { parts: [{ text: 'just thinking', thought: true }] }, finishReason: 'STOP' }, + ], + }; + const chunks = await collect( + createGeminiAdapter({ transport: fakeTransport(r, [r]) }).stream(REQ, 'k'), + ); + expect(chunks.some((c) => c.type === 'reasoning_end')).toBe(true); + expect(chunks.at(-1)?.type).toBe('stop'); + }); +}); + +describe('Gemini adapter — usage, truncation, refusal, malformed tool (review fixes)', () => { + it('mapUsage adds toolUsePromptTokenCount to input (disjoint from prompt tokens)', () => { + expect( + mapUsage({ promptTokenCount: 10, candidatesTokenCount: 4, toolUsePromptTokenCount: 6 }), + ).toEqual({ inputTokens: 16, outputTokens: 4 }); + }); + + it('emits a transport error when a stream ends without a finishReason (truncated)', async () => { + const r: GeminiResponse = { candidates: [{ content: { parts: [{ text: 'partial' }] } }] }; + const chunks = await collect( + createGeminiAdapter({ transport: fakeTransport(r, [r]) }).stream(REQ, 'k'), + ); + expect(chunks.some((c) => c.type === 'text_delta')).toBe(true); + const last = chunks.at(-1); + expect(last?.type).toBe('error'); + if (last?.type === 'error') { + expect(last.error.kind).toBe('transport'); + expect(last.error.retryable).toBe(true); + } + }); + + it('maps a blocked-prompt generate (no candidate + promptFeedback) to content_filter', async () => { + const r: GeminiResponse = { promptFeedback: { blockReason: 'SAFETY' } }; + const result = await createGeminiAdapter({ transport: fakeTransport(r) }).generate(REQ, 'k'); + expect(result.content).toEqual([]); + expect(result.stopReason).toBe('content_filter'); + }); + + it('maps a blocked-prompt stream to a content_filter stop (terminal, not truncation)', async () => { + const r: GeminiResponse = { promptFeedback: { blockReason: 'SAFETY' } }; + const chunks = await collect( + createGeminiAdapter({ transport: fakeTransport(r, [r]) }).stream(REQ, 'k'), + ); + const stop = chunks.at(-1); + expect(stop?.type).toBe('stop'); + if (stop?.type === 'stop') { + expect(stop.stopReason).toBe('content_filter'); + } + }); + + it('treats BLOCKED_REASON_UNSPECIFIED as NOT blocked (the sentinel is not a real block)', async () => { + // A normal response that happens to carry the unspecified sentinel must not be mis-mapped to + // content_filter. Here it rides alongside a real candidate. + const r: GeminiResponse = { + candidates: [{ content: { parts: [{ text: 'ok' }] }, finishReason: 'STOP' }], + promptFeedback: { blockReason: 'BLOCKED_REASON_UNSPECIFIED' }, + }; + const result = await createGeminiAdapter({ transport: fakeTransport(r) }).generate(REQ, 'k'); + expect(result.stopReason).toBe('stop'); + expect(result.content).toEqual([{ type: 'text', text: 'ok' }]); + }); + + it('skips a nameless functionCall in a stream (no invalid name:"" tool_call_start)', async () => { + const r: GeminiResponse = { + candidates: [{ content: { parts: [{ functionCall: { args: {} } }] }, finishReason: 'STOP' }], + }; + const chunks = await collect( + createGeminiAdapter({ transport: fakeTransport(r, [r]) }).stream(REQ, 'k'), + ); + expect(chunks.some((c) => c.type === 'tool_call_start')).toBe(false); + expect(chunks.at(-1)?.type).toBe('stop'); + }); +}); diff --git a/packages/llm/src/adapters/gemini.ts b/packages/llm/src/adapters/gemini.ts new file mode 100644 index 00000000..8648202c --- /dev/null +++ b/packages/llm/src/adapters/gemini.ts @@ -0,0 +1,551 @@ +import { GoogleGenAI } from '@google/genai'; + +import type { ContentPart, StopReason } from '@relavium/shared'; + +import { assertStreamable, assertSupported } from '../capabilities.js'; +import { LlmProviderError, kindFromHttpStatus, makeLlmError } from '../llm-error.js'; +import { GeminiToolCallIds, normalizeToolCall, toWire } from '../tool-normalizer.js'; +import type { + CapabilityFlags, + LlmError, + LlmMessage, + LlmProvider, + LlmRequest, + LlmResult, + StreamChunk, + ToolChoice, + ToolDef, + Usage, +} from '../types.js'; + +import { REASONING_ID, isAbortSignal } from './shared.js'; + +/** + * The Gemini adapter (1.H) over `@google/genai` — the riskiest adapter: a restricted tool schema and + * **no native tool-call ids** (synthesized via the 1.E `ToolNormalizer`). Like the others it lives + * behind the seam (SDK imported only here, nothing vendor-shaped escapes). Because `GoogleGenAI` has + * no `fetch` hook (unlike the Anthropic/OpenAI SDKs), the network call is isolated behind an injected + * **`GeminiTransport`**: the default wraps the real SDK, while the conformance harness injects a fake + * that replays recorded SDK-shaped responses — keeping the fold/normalization (the part conformance + * proves) identical and the conformance module free of any vendor import. See + * [llm-provider-seam.md](../../../../docs/reference/shared-core/llm-provider-seam.md). + */ + +const PROVIDER = 'gemini'; + +/** Gemini's common-path capability surface (restricted tool schema; ids synthesized). */ +const GEMINI_SUPPORTS: CapabilityFlags = { + tools: true, + streaming: true, + parallelToolCalls: true, + vision: true, + promptCache: true, + reasoning: true, +}; + +const ZERO_USAGE: Usage = { inputTokens: 0, outputTokens: 0 }; + +const isRecord = (value: unknown): value is Record => + typeof value === 'object' && value !== null && !Array.isArray(value); + +/** + * True when the prompt was actually blocked. Gemini's `blockReason` enum includes the + * `BLOCKED_REASON_UNSPECIFIED` sentinel that does **not** mean "blocked" — treat only a real, + * specified reason as a content-filter block. + */ +function isPromptBlocked(promptFeedback: { blockReason?: string } | undefined): boolean { + const reason = promptFeedback?.blockReason; + return reason !== undefined && reason !== 'BLOCKED_REASON_UNSPECIFIED'; +} + +/** Remove transport-level keys that the SDK exposes for URL/header override — SSRF guard. */ +function stripTransportKeys(opts: Record): Record { + const rest: Record = {}; + for (const [k, v] of Object.entries(opts)) { + // `httpOptions.baseUrl`/`headers` would redirect egress (and the API key) to an arbitrary host. + if (k !== 'httpOptions') { + rest[k] = v; + } + } + return rest; +} + +// --- Structural views of the SDK shapes (so the fold + conformance stay vendor-type-free) ------- + +/** The subset of a Gemini `functionCall` part the fold reads. */ +interface GeminiFunctionCall { + name?: string; + args?: Record; +} + +/** The subset of a Gemini content part the fold reads. */ +interface GeminiPart { + text?: string; + thought?: boolean; + thoughtSignature?: string; + functionCall?: GeminiFunctionCall; +} + +/** The subset of a `GenerateContentResponse` the fold reads (the real SDK type satisfies this). */ +export interface GeminiResponse { + candidates?: Array<{ content?: { parts?: GeminiPart[] }; finishReason?: string }>; + // Present (with a blockReason) when the prompt itself is blocked and no candidate is produced. + promptFeedback?: { blockReason?: string }; + usageMetadata?: { + promptTokenCount?: number; + candidatesTokenCount?: number; + cachedContentTokenCount?: number; + thoughtsTokenCount?: number; + toolUsePromptTokenCount?: number; + }; +} + +/** The lowered request the transport sends (a plain object the SDK accepts via a boundary cast). */ +export interface GeminiRequest { + model: string; + contents: Array<{ role: 'user' | 'model'; parts: Array> }>; + config: Record; +} + +/** + * The injected network seam. The default wraps `@google/genai`; the conformance harness injects a + * replay implementation. Keeping it here lets the one adapter run on every host (ADR-0018) and lets + * tests drive the fold without a vendor import. + */ +export interface GeminiTransport { + generate(request: GeminiRequest, key: string): Promise; + stream(request: GeminiRequest, key: string): Promise>; +} + +// --- Normalization: Gemini wire → canonical -------------------------------------------------- + +/** Map a Gemini finish reason to the canonical enum; a `STOP` with tool calls is `tool_use`. */ +export function mapStopReason(reason: string | undefined, hasToolCalls: boolean): StopReason { + switch (reason) { + case 'MAX_TOKENS': + return 'length'; + case 'SAFETY': + case 'RECITATION': + case 'BLOCKLIST': + case 'PROHIBITED_CONTENT': + case 'SPII': + case 'IMAGE_SAFETY': + return 'content_filter'; + case 'MALFORMED_FUNCTION_CALL': + case 'UNEXPECTED_TOOL_CALL': + // A broken/invalid tool call is a terminal failure — surface it as 'error' rather than masking + // it as a clean 'stop' (both are tool-call faults in the pinned SDK enum; the only adapter + // that emits 'error'). + return 'error'; + case 'STOP': + case undefined: + return hasToolCalls ? 'tool_use' : 'stop'; + default: + return 'stop'; // an unknown/future reason degrades, consistent with the other adapters + } +} + +/** + * Map Gemini usage to the canonical **NET** `Usage`. Per the `GenerateContentResponseUsageMetadata` + * contract (the `generateContent`/`generateContentStream` shape this adapter consumes), + * `totalTokenCount = promptTokenCount + candidatesTokenCount + toolUsePromptTokenCount + thoughtsTokenCount` + * — the four are **disjoint additive** terms (unlike Anthropic/OpenAI, where thinking is already inside + * output). `promptTokenCount` includes cached content (subtracted out for NET input). + */ +export function mapUsage(usage: { + promptTokenCount?: number; + candidatesTokenCount?: number; + cachedContentTokenCount?: number; + thoughtsTokenCount?: number; + toolUsePromptTokenCount?: number; +}): Usage { + const cached = usage.cachedContentTokenCount ?? 0; + const thinking = usage.thoughtsTokenCount ?? 0; + const out: Usage = { + // Tool-use prompt tokens are input-priced and disjoint from promptTokenCount — include them so + // input is not undercounted on grounded/tool-use calls. + inputTokens: + Math.max(0, (usage.promptTokenCount ?? 0) - cached) + (usage.toolUsePromptTokenCount ?? 0), + // Thinking tokens are billed separately from candidates — sum both to match totalTokenCount (ADR-0030). + outputTokens: (usage.candidatesTokenCount ?? 0) + thinking, + }; + if (cached > 0) { + out.cacheReadTokens = cached; + } + // Surface the thinking subset for observability (ADR-0030); already included in outputTokens above. + if (thinking > 0) { + out.reasoningTokens = thinking; + } + return out; +} + +/** Fold a non-streaming Gemini response into canonical content parts (text + synthesized tool_call). */ +export function mapContent(response: GeminiResponse, ids: GeminiToolCallIds): ContentPart[] { + const parts: ContentPart[] = []; + for (const part of response.candidates?.[0]?.content?.parts ?? []) { + if (part.functionCall !== undefined) { + const name = part.functionCall.name ?? ''; + parts.push( + normalizeToolCall(PROVIDER, { + id: ids.synthesize(name), // Gemini has no native id — mint a stable one (1.E) + name, + args: part.functionCall.args ?? {}, + }), + ); + } else if (part.text !== undefined && part.text.length > 0) { + if (part.thought === true) { + // Reasoning (ADR-0030); thoughtSignature is the ephemeral same-provider continuity token. + parts.push( + part.thoughtSignature !== undefined && part.thoughtSignature.length > 0 + ? { type: 'reasoning', text: part.text, signature: part.thoughtSignature } + : { type: 'reasoning', text: part.text }, + ); + } else { + parts.push({ type: 'text', text: part.text }); + } + } + } + return parts; +} + +/** Classify any transport/SDK throwable into a normalized `LlmError` — no vendor shape escapes. */ +export function geminiErrorToLlmError(err: unknown): LlmError { + if (err instanceof Error && err.name === 'AbortError') { + return makeLlmError({ provider: PROVIDER, kind: 'cancelled', message: 'request aborted' }); + } + // The SDK's `ApiError` (and the conformance replay) carry a numeric `status`; classify by it. + if (isRecord(err) && typeof err['status'] === 'number') { + const status = err['status']; + const message = typeof err['message'] === 'string' ? err['message'] : 'gemini API error'; + return makeLlmError({ provider: PROVIDER, kind: kindFromHttpStatus(status), message, status }); + } + return makeLlmError({ + provider: PROVIDER, + kind: 'unknown', + message: err instanceof Error ? err.message : 'unknown provider error', + }); +} + +// --- Request building: canonical → Gemini wire ----------------------------------------------- + +function toGeminiToolChoice(choice: ToolChoice): Record { + // Gemini's function-calling mode: AUTO (default), NONE, or ANY (force a call; a named call uses + // ANY + allowedFunctionNames). + if (choice === 'none') { + return { functionCallingConfig: { mode: 'NONE' } }; + } + if (choice === 'required') { + return { functionCallingConfig: { mode: 'ANY' } }; + } + if (choice === 'auto') { + return { functionCallingConfig: { mode: 'AUTO' } }; + } + return { functionCallingConfig: { mode: 'ANY', allowedFunctionNames: [choice.name] } }; +} + +function toGeminiTool(toolDef: ToolDef): Record { + const wire = toWire(toolDef, PROVIDER); + /* v8 ignore next 3 -- defensive: toWire('gemini') always returns the functionDeclarations shape */ + if (!('functionDeclarations' in wire)) { + throw new Error('unreachable: the Gemini wire shape always carries functionDeclarations'); + } + return { functionDeclarations: wire.functionDeclarations }; +} + +/** Build the Gemini message contents, mapping tool results back to function responses by name. */ +function toGeminiContents( + messages: readonly LlmMessage[], +): Array<{ role: 'user' | 'model'; parts: Array> }> { + // Gemini references a tool result by function name, not id — recover the name from the matching + // tool_call in this same request (its synthesized id ↔ name pairing is carried in the messages). + const nameById = new Map(); + for (const message of messages) { + for (const part of message.content) { + if (part.type === 'tool_call') { + nameById.set(part.id, part.name); + } + } + } + const contents: Array<{ role: 'user' | 'model'; parts: Array> }> = []; + for (const message of messages) { + const parts = toGeminiParts(message, nameById); + if (parts.length > 0) { + contents.push({ role: message.role === 'assistant' ? 'model' : 'user', parts }); + } + } + return contents; +} + +function toGeminiParts( + message: LlmMessage, + nameById: ReadonlyMap, +): Array> { + const parts: Array> = []; + for (const part of message.content) { + if (part.type === 'text') { + parts.push({ text: part.text }); + } else if (part.type === 'tool_call') { + parts.push({ functionCall: { name: part.name, args: part.args } }); + } else if (part.type === 'tool_result') { + const name = nameById.get(part.toolCallId) ?? part.toolCallId; + parts.push({ functionResponse: { name, response: toResponseObject(part.result) } }); + } + // reasoning parts are ephemeral (ADR-0030) — dropped here, never replayed to the provider. + } + return parts; +} + +/** Gemini's `functionResponse.response` must be an object; wrap a non-object result. */ +function toResponseObject(result: unknown): Record { + return isRecord(result) ? result : { result }; +} + +/** Lower a canonical request into the Gemini request shape (system → `systemInstruction`, etc.). */ +export function buildGeminiRequest(req: LlmRequest): GeminiRequest { + const config: Record = {}; + if (req.system !== undefined) { + config['systemInstruction'] = req.system; + } + if (req.tools !== undefined) { + config['tools'] = req.tools.map(toGeminiTool); + } + if (req.toolChoice !== undefined) { + config['toolConfig'] = toGeminiToolChoice(req.toolChoice); + } + if (req.responseFormat?.type === 'json') { + // Native structured output (ADR-0030): JSON mime type + the canonical schema as responseJsonSchema. + config['responseMimeType'] = 'application/json'; + config['responseJsonSchema'] = req.responseFormat.schema; + } + if (req.temperature !== undefined) { + config['temperature'] = req.temperature; + } + if (req.maxTokens !== undefined) { + config['maxOutputTokens'] = req.maxTokens; + } + if (req.stopSequences !== undefined) { + config['stopSequences'] = req.stopSequences; + } + if (isAbortSignal(req.signal)) { + config['abortSignal'] = req.signal; + } + // The typed escape hatch (1.D): caller-supplied Gemini config the common path doesn't model. + // Strip httpOptions before the merge: a caller-supplied httpOptions.baseUrl is forwarded verbatim + // to the SDK's patchHttpOptions path, which would redirect the request — and the real API key — + // to an attacker-controlled URL (SSRF). Transport-level config is never safe to forward from an + // untrusted providerOptions payload. + const merged = + req.providerOptions === undefined + ? config + : { + ...stripTransportKeys(req.providerOptions), + ...config, // mapped fields win + }; + return { model: req.model, contents: toGeminiContents(req.messages), config: merged }; +} + +// --- The default transport (the only place the SDK is used) ----------------------------------- + +/* v8 ignore start -- the live-only real-SDK transport; the fold it feeds is fully covered offline via an injected GeminiTransport */ +const sdkTransport: GeminiTransport = { + async generate(request: GeminiRequest, key: string): Promise { + const client = new GoogleGenAI({ apiKey: key }); + return client.models.generateContent(request); + }, + async stream(request: GeminiRequest, key: string): Promise> { + const client = new GoogleGenAI({ apiKey: key }); + return client.models.generateContentStream(request); + }, +}; +/* v8 ignore stop */ + +// --- Streaming fold -------------------------------------------------------------------------- + +/** Mutable fold state threaded across the streamed Gemini responses. */ +interface GeminiStreamState { + reasoningOpen: boolean; + reasoningSignature: string | undefined; + hasToolCalls: boolean; + /** A terminal signal (candidate finishReason or prompt block) was seen — else the stream truncated. */ + sawTerminal: boolean; + /** The prompt was blocked (content_filter), not a normal completion. */ + blocked: boolean; + finishReason: string | undefined; + usage: Usage; + readonly ids: GeminiToolCallIds; +} + +/** Emit `reasoning_end` (with the accumulated signature) if reasoning is open, then reset the track. */ +function closeReasoning(state: GeminiStreamState, out: StreamChunk[]): void { + if (!state.reasoningOpen) { + return; + } + out.push( + state.reasoningSignature === undefined + ? { type: 'reasoning_end', id: REASONING_ID } + : { type: 'reasoning_end', id: REASONING_ID, signature: state.reasoningSignature }, + ); + state.reasoningOpen = false; + state.reasoningSignature = undefined; // reset so a later reasoning segment starts with a fresh signature +} + +/** Fold one Gemini content part into the chunks to emit, mutating the streamed fold state. */ +function foldGeminiPart(part: GeminiPart, state: GeminiStreamState): StreamChunk[] { + const out: StreamChunk[] = []; + if (part.functionCall !== undefined) { + const name = part.functionCall.name ?? ''; + if (name.length === 0) { + // A functionCall with no name can't form a valid (nonEmptyString) tool_call chunk — skip it, + // matching the OpenAI stream guard. (The non-streaming path is stricter: normalizeToolCall + // throws on an empty name; in a stream we drop the malformed part and continue.) + return out; + } + closeReasoning(state, out); + const id = state.ids.synthesize(name); + // Gemini delivers the whole args object in one event — emit start/delta/end together. + out.push( + { type: 'tool_call_start', id, name }, + { type: 'tool_call_delta', id, argsJsonDelta: JSON.stringify(part.functionCall.args ?? {}) }, + { type: 'tool_call_end', id }, + ); + state.hasToolCalls = true; + return out; + } + if (part.text === undefined || part.text.length === 0) { + return out; + } + if (part.thought === true) { + if (!state.reasoningOpen) { + out.push({ type: 'reasoning_start', id: REASONING_ID }); + state.reasoningOpen = true; + } + if (part.thoughtSignature !== undefined && part.thoughtSignature.length > 0) { + state.reasoningSignature = part.thoughtSignature; + } + out.push({ type: 'reasoning_delta', id: REASONING_ID, text: part.text }); + return out; + } + closeReasoning(state, out); + out.push({ type: 'text_delta', text: part.text }); + return out; +} + +/** Fold one streamed Gemini response into chunks, updating the terminal/usage tracking on `state`. */ +function foldGeminiResponse(response: GeminiResponse, state: GeminiStreamState): StreamChunk[] { + const out: StreamChunk[] = []; + if (response.usageMetadata) { + state.usage = mapUsage(response.usageMetadata); + } + if (isPromptBlocked(response.promptFeedback)) { + state.blocked = true; + state.sawTerminal = true; + } + const candidate = response.candidates?.[0]; + if (candidate?.finishReason !== undefined) { + state.finishReason = candidate.finishReason; + state.sawTerminal = true; + } + for (const part of candidate?.content?.parts ?? []) { + out.push(...foldGeminiPart(part, state)); + } + return out; +} + +async function* streamChunks( + transport: GeminiTransport, + request: GeminiRequest, + key: string, +): AsyncIterable { + const state: GeminiStreamState = { + reasoningOpen: false, + reasoningSignature: undefined, + hasToolCalls: false, + sawTerminal: false, + blocked: false, + finishReason: undefined, + usage: ZERO_USAGE, + ids: new GeminiToolCallIds(), + }; + let sdkStream: AsyncIterable; + try { + sdkStream = await transport.stream(request, key); + } catch (err) { + yield { type: 'error', error: geminiErrorToLlmError(err) }; + return; + } + try { + for await (const response of sdkStream) { + yield* foldGeminiResponse(response, state); + } + } catch (err) { + yield { type: 'error', error: geminiErrorToLlmError(err) }; + return; + } + const tail: StreamChunk[] = []; + closeReasoning(state, tail); + yield* tail; + // No finishReason and no block → truncated stream; surface a retryable transport error. + if (!state.sawTerminal) { + yield { + type: 'error', + error: makeLlmError({ + provider: PROVIDER, + kind: 'transport', + message: 'stream ended before a finishReason (truncated response)', + }), + }; + return; + } + const stopReason: StopReason = state.blocked + ? 'content_filter' + : mapStopReason(state.finishReason, state.hasToolCalls); + yield { type: 'stop', stopReason, usage: state.usage }; +} + +// --- The adapter ----------------------------------------------------------------------------- + +/** Dependencies the conformance replayer / tests inject (the network transport). */ +export interface GeminiAdapterDeps { + /** Override the network transport (the replayer injects a recorded-response transport). */ + readonly transport?: GeminiTransport; +} + +/** Build a Gemini `LlmProvider`. Exposed as `geminiAdapter`; the factory enables DI for 1.F. */ +export function createGeminiAdapter(deps: GeminiAdapterDeps = {}): LlmProvider { + const transport = deps.transport ?? sdkTransport; + return { + id: PROVIDER, + supports: GEMINI_SUPPORTS, + async generate(req: LlmRequest, key: string): Promise { + assertSupported(PROVIDER, GEMINI_SUPPORTS, req); // fail fast on an unsupported feature + try { + const response = await transport.generate(buildGeminiRequest(req), key); + const ids = new GeminiToolCallIds(); + const content = mapContent(response, ids); + const hasToolCalls = content.some((part) => part.type === 'tool_call'); + const candidate = response.candidates?.[0]; + // A blocked prompt yields no candidate but a promptFeedback.blockReason — normalize it to + // content_filter, not a clean stop that masks the block as an empty success. + const blocked = candidate === undefined && isPromptBlocked(response.promptFeedback); + return { + content, + stopReason: blocked + ? 'content_filter' + : mapStopReason(candidate?.finishReason, hasToolCalls), + usage: response.usageMetadata ? mapUsage(response.usageMetadata) : ZERO_USAGE, + raw: response, + }; + } catch (err) { + throw new LlmProviderError(geminiErrorToLlmError(err)); + } + }, + stream(req: LlmRequest, key: string): AsyncIterable { + assertSupported(PROVIDER, GEMINI_SUPPORTS, req); // fail fast on an unsupported feature + assertStreamable(PROVIDER, GEMINI_SUPPORTS); + return streamChunks(transport, buildGeminiRequest(req), key); + }, + }; +} + +/** The production Gemini adapter. */ +export const geminiAdapter: LlmProvider = createGeminiAdapter(); diff --git a/packages/llm/src/adapters/index.ts b/packages/llm/src/adapters/index.ts index 43199dfc..789c0c5b 100644 --- a/packages/llm/src/adapters/index.ts +++ b/packages/llm/src/adapters/index.ts @@ -2,8 +2,14 @@ * The `@relavium/llm/adapters` entry point — the **platform-coupled** zone where provider SDKs are * imported (the seam fence's one legal area). Kept separate from the seam barrel (`@relavium/llm`) * so the engine, which consumes only the `LlmProvider` seam and gets concrete adapters injected at - * the surface, stays platform-free. The OpenAI/DeepSeek (1.G) and Gemini (1.H) adapters land here. + * the surface, stays platform-free. */ export { anthropicAdapter, createAnthropicAdapter } from './anthropic.js'; export type { AnthropicAdapterDeps } from './anthropic.js'; + +export { openaiAdapter, deepseekAdapter, createOpenAiAdapter } from './openai.js'; +export type { OpenAiAdapterDeps } from './openai.js'; + +export { geminiAdapter, createGeminiAdapter } from './gemini.js'; +export type { GeminiAdapterDeps } from './gemini.js'; diff --git a/packages/llm/src/adapters/openai.test.ts b/packages/llm/src/adapters/openai.test.ts new file mode 100644 index 00000000..026e88d7 --- /dev/null +++ b/packages/llm/src/adapters/openai.test.ts @@ -0,0 +1,790 @@ +import { APIConnectionError, APIConnectionTimeoutError, APIError, APIUserAbortError } from 'openai'; +import { describe, expect, it } from 'vitest'; + +import { InvalidBaseUrlError } from '../errors.js'; +import type { StreamChunk } from '../types.js'; +import { + createOpenAiAdapter, + deepseekAdapter, + mapContent, + mapStopReason, + mapUsage, + openaiAdapter, + openaiErrorToLlmError, +} from './openai.js'; + +const isRecord = (value: unknown): value is Record => + typeof value === 'object' && value !== null && !Array.isArray(value); + +function parseJsonBody(init: RequestInit | undefined): Record { + const raw = typeof init?.body === 'string' ? init.body : '{}'; + const parsed: unknown = JSON.parse(raw); + if (!isRecord(parsed)) { + throw new Error('expected a JSON object request body'); + } + return parsed; +} + +async function collect(stream: AsyncIterable): Promise { + const chunks: StreamChunk[] = []; + for await (const chunk of stream) { + chunks.push(chunk); + } + return chunks; +} + +const completion = (message: unknown, finishReason = 'stop'): string => + JSON.stringify({ + id: 'c', + object: 'chat.completion', + created: 0, + model: 'gpt-4o', + choices: [{ index: 0, message, finish_reason: finishReason, logprobs: null }], + usage: { prompt_tokens: 1, completion_tokens: 1, total_tokens: 2 }, + }); + +const okResponse = (): Response => + new Response(completion({ role: 'assistant', content: 'ok', refusal: null }), { + status: 200, + headers: { 'content-type': 'application/json' }, + }); + +/** Build an SSE Response from a list of chunk objects (shared across the streaming describes). */ +const sse = (chunks: readonly unknown[]): Response => + new Response(chunks.map((c) => `data: ${JSON.stringify(c)}\n\n`).join('') + 'data: [DONE]\n\n', { + status: 200, + headers: { 'content-type': 'text/event-stream' }, + }); + +/** A single chat-completion stream chunk wrapping the given choices. */ +const streamChunk = (choices: readonly unknown[]): Record => ({ + id: 's', + object: 'chat.completion.chunk', + created: 0, + model: 'gpt-4o', + choices, +}); + +/** A one-choice stream chunk with the given delta + finish reason. */ +const dchunk = (delta: unknown, finish: string | null = null): Record => ({ + id: 's', + object: 'chat.completion.chunk', + created: 0, + model: 'gpt-4o', + choices: [{ index: 0, delta, finish_reason: finish }], +}); + +describe('OpenAI-compatible adapter', () => { + it('exposes openai + deepseek ids with their capability surfaces', () => { + expect(openaiAdapter.id).toBe('openai'); + expect(deepseekAdapter.id).toBe('deepseek'); + expect(openaiAdapter.supports.vision).toBe(true); + expect(openaiAdapter.supports.reasoning).toBe(false); + expect(deepseekAdapter.supports.reasoning).toBe(true); + expect(deepseekAdapter.supports.vision).toBe(false); + }); + + it('maps finish reasons to the canonical enum (incl. graceful unknown → stop)', () => { + expect(mapStopReason('stop')).toBe('stop'); + expect(mapStopReason('length')).toBe('length'); + expect(mapStopReason('tool_calls')).toBe('tool_use'); + expect(mapStopReason('function_call')).toBe('tool_use'); + expect(mapStopReason('content_filter')).toBe('content_filter'); + expect(mapStopReason(null)).toBe('stop'); + expect(mapStopReason(undefined)).toBe('stop'); + expect(mapStopReason('future_reason')).toBe('stop'); + }); + + it('maps usage to NET, subtracting cache from gross prompt_tokens', () => { + // OpenAI: prompt_tokens_details.cached_tokens + expect( + mapUsage({ + prompt_tokens: 100, + completion_tokens: 20, + prompt_tokens_details: { cached_tokens: 30 }, + }), + ).toEqual({ inputTokens: 70, outputTokens: 20, cacheReadTokens: 30 }); + // DeepSeek: top-level prompt_cache_hit_tokens + expect( + mapUsage({ prompt_tokens: 50, completion_tokens: 5, prompt_cache_hit_tokens: 10 }), + ).toEqual({ + inputTokens: 40, + outputTokens: 5, + cacheReadTokens: 10, + }); + // No cache → no cacheReadTokens key; clamps at 0. + expect(mapUsage({ prompt_tokens: 10, completion_tokens: 5 })).toEqual({ + inputTokens: 10, + outputTokens: 5, + }); + expect(mapUsage({})).toEqual({ inputTokens: 0, outputTokens: 0 }); + }); + + it('mapContent keeps text + function tool_calls and skips custom (non-function) tool calls', () => { + const parts = mapContent( + { + content: 'hi', + tool_calls: [ + { id: 't1', function: { name: 'f', arguments: '{"a":1}' } }, + { id: 'c1' }, // a custom tool call (no function) — skipped + ], + }, + 'openai', + ); + expect(parts).toEqual([ + { type: 'text', text: 'hi' }, + { type: 'tool_call', id: 't1', name: 'f', args: { a: 1 } }, + ]); + }); + + it('mapContent treats empty tool arguments as {}', () => { + const parts = mapContent( + { content: null, tool_calls: [{ id: 't1', function: { name: 'f', arguments: '' } }] }, + 'openai', + ); + expect(parts).toEqual([{ type: 'tool_call', id: 't1', name: 'f', args: {} }]); + }); +}); + +describe('openaiErrorToLlmError — classification', () => { + it('classifies the connection/abort error classes', () => { + expect(openaiErrorToLlmError(new APIUserAbortError(), 'openai')).toMatchObject({ + kind: 'cancelled', + retryable: false, + provider: 'openai', + }); + expect(openaiErrorToLlmError(new APIConnectionTimeoutError(), 'openai')).toMatchObject({ + kind: 'timeout', + retryable: true, + }); + expect( + openaiErrorToLlmError(new APIConnectionError({ message: 'down' }), 'deepseek'), + ).toMatchObject({ kind: 'transport', retryable: true, provider: 'deepseek' }); + }); + + it('classifies an APIError by HTTP status; status-less → unknown', () => { + expect( + openaiErrorToLlmError(new APIError(429, undefined, 'rate limited', undefined), 'openai'), + ).toMatchObject({ kind: 'rate_limit', retryable: true, status: 429 }); + expect( + openaiErrorToLlmError(new APIError(401, undefined, 'unauthorized', undefined), 'openai'), + ).toMatchObject({ kind: 'auth', retryable: false, status: 401 }); + expect( + openaiErrorToLlmError(new APIError(undefined, undefined, 'mystery', undefined), 'openai'), + ).toMatchObject({ kind: 'unknown', retryable: false }); + }); + + it('falls back to unknown for a non-Error throwable', () => { + expect(openaiErrorToLlmError('boom', 'openai')).toMatchObject({ + kind: 'unknown', + retryable: false, + }); + }); +}); + +describe('OpenAI-compatible adapter — request building + secret safety', () => { + it('prepends system, splits tool results, and maps tool_choice + tools onto the body', async () => { + let sent: Record = {}; + const adapter = createOpenAiAdapter({ + providerId: 'openai', + fetch: (_input, init) => { + sent = parseJsonBody(init); + return Promise.resolve(okResponse()); + }, + maxRetries: 0, + }); + await adapter.generate( + { + model: 'gpt-4o', + system: 'be terse', + toolChoice: 'required', + tools: [{ name: 'get_weather', parameters: { type: 'object' } }], + messages: [ + { + role: 'assistant', + content: [ + { type: 'tool_call', id: 'c1', name: 'get_weather', args: { city: 'Paris' } }, + ], + }, + { + role: 'tool', + content: [{ type: 'tool_result', toolCallId: 'c1', result: { tempC: 18 } }], + }, + ], + }, + 'k', + ); + const messages = sent['messages'] as Array<{ + role: string; + content?: unknown; + tool_calls?: unknown[]; + tool_call_id?: string; + }>; + expect(messages[0]).toMatchObject({ role: 'system', content: 'be terse' }); + expect(messages[1]).toMatchObject({ role: 'assistant' }); + expect((messages[1]?.tool_calls as Array<{ id: string }>)[0]).toMatchObject({ + id: 'c1', + type: 'function', + }); + expect(messages[2]).toMatchObject({ + role: 'tool', + tool_call_id: 'c1', + content: JSON.stringify({ tempC: 18 }), + }); + expect(sent['tool_choice']).toBe('required'); + expect(sent['tools']).toMatchObject([{ type: 'function', function: { name: 'get_weather' } }]); + }); + + it('forwards temperature/stopSequences and lets providerOptions only ADD (mapped fields win)', async () => { + let sent: Record = {}; + const adapter = createOpenAiAdapter({ + fetch: (_input, init) => { + sent = parseJsonBody(init); + return Promise.resolve(okResponse()); + }, + maxRetries: 0, + }); + await adapter.generate( + { + model: 'gpt-4o', + temperature: 0.5, + stopSequences: ['STOP'], + providerOptions: { seed: 42, model: 'attacker-override' }, + messages: [{ role: 'user', content: [{ type: 'text', text: 'hi' }] }], + }, + 'k', + ); + expect(sent['temperature']).toBe(0.5); + expect(sent['stop']).toEqual(['STOP']); + expect(sent['seed']).toBe(42); // escape-hatch field reached the wire + expect(sent['model']).toBe('gpt-4o'); // mapped field wins over providerOptions + }); + + it('maps tool_choice {name} to a named function choice', async () => { + let sent: Record = {}; + const adapter = createOpenAiAdapter({ + fetch: (_input, init) => { + sent = parseJsonBody(init); + return Promise.resolve(okResponse()); + }, + maxRetries: 0, + }); + await adapter.generate( + { + model: 'gpt-4o', + toolChoice: { name: 'get_weather' }, + messages: [{ role: 'user', content: [{ type: 'text', text: 'hi' }] }], + }, + 'k', + ); + expect(sent['tool_choice']).toEqual({ type: 'function', function: { name: 'get_weather' } }); + }); + + it('never leaks the API key into the surfaced LlmError', async () => { + const adapter = createOpenAiAdapter({ + fetch: () => + Promise.resolve( + new Response( + JSON.stringify({ error: { message: 'unauthorized', type: 'invalid_request_error' } }), + { + status: 401, + headers: { 'content-type': 'application/json' }, + }, + ), + ), + maxRetries: 0, + }); + let caught: unknown; + try { + await adapter.generate( + { model: 'gpt-4o', messages: [{ role: 'user', content: [{ type: 'text', text: 'hi' }] }] }, + 'sk-SECRET-KEY-123', + ); + } catch (err) { + caught = err; + } + expect(caught).toBeInstanceOf(Error); + expect(JSON.stringify(caught)).not.toContain('SECRET'); + }); +}); + +describe('OpenAI-compatible adapter — stream edge cases', () => { + const REQ = { + model: 'gpt-4o', + messages: [{ role: 'user' as const, content: [{ type: 'text' as const, text: 'hi' }] }], + }; + + it('yields a single error chunk when the stream fails to start (429)', async () => { + const adapter = createOpenAiAdapter({ + fetch: () => + Promise.resolve( + new Response(JSON.stringify({ error: { message: 'rl', type: 'rate_limit_exceeded' } }), { + status: 429, + headers: { 'content-type': 'application/json' }, + }), + ), + maxRetries: 0, + }); + const chunks = await collect(adapter.stream(REQ, 'k')); + expect(chunks).toHaveLength(1); + expect(chunks[0]?.type).toBe('error'); + if (chunks[0]?.type === 'error') { + expect(chunks[0].error.kind).toBe('rate_limit'); + } + }); + + it('ignores a tool_calls delta with no id/name on first delta (defensive)', async () => { + const adapter = createOpenAiAdapter({ + fetch: () => + Promise.resolve( + sse([ + { + id: 's', + object: 'chat.completion.chunk', + created: 0, + model: 'gpt-4o', + // a fragment with no preceding id+name for index 0 — can't start a tool, skipped + choices: [ + { + index: 0, + delta: { tool_calls: [{ index: 0, function: { arguments: '{}' } }] }, + finish_reason: null, + }, + ], + }, + { + id: 's', + object: 'chat.completion.chunk', + created: 0, + model: 'gpt-4o', + choices: [{ index: 0, delta: { content: 'hi' }, finish_reason: 'stop' }], + }, + ]), + ), + maxRetries: 0, + }); + const chunks = await collect(adapter.stream(REQ, 'k')); + expect(chunks.some((c) => c.type === 'tool_call_start')).toBe(false); + expect(chunks.some((c) => c.type === 'tool_call_delta')).toBe(false); + expect(chunks.some((c) => c.type === 'text_delta')).toBe(true); + expect(chunks.at(-1)?.type).toBe('stop'); + }); + + it('threads an AbortSignal to the request options', async () => { + let sawSignal = false; + const adapter = createOpenAiAdapter({ + fetch: (_input, init) => { + sawSignal = init?.signal instanceof AbortSignal; + return Promise.resolve(okResponse()); + }, + maxRetries: 0, + }); + const controller = new AbortController(); + await adapter.generate({ ...REQ, signal: controller.signal }, 'k'); + expect(sawSignal).toBe(true); + }); +}); + +describe('OpenAI-compatible adapter — additional fold + generate branches', () => { + const REQ = { + model: 'gpt-4o', + messages: [{ role: 'user' as const, content: [{ type: 'text' as const, text: 'hi' }] }], + }; + + it('emits a tool_call_delta when the first tool delta already carries arguments', async () => { + const adapter = createOpenAiAdapter({ + fetch: () => + Promise.resolve( + sse([ + streamChunk([ + { + index: 0, + delta: { + tool_calls: [ + { + index: 0, + id: 'call_1', + type: 'function', + function: { name: 'f', arguments: '{"a":1}' }, + }, + ], + }, + finish_reason: null, + }, + ]), + streamChunk([{ index: 0, delta: {}, finish_reason: 'tool_calls' }]), + ]), + ), + maxRetries: 0, + }); + const chunks = await collect(adapter.stream(REQ, 'k')); + const start = chunks.find((c) => c.type === 'tool_call_start'); + const delta = chunks.find((c) => c.type === 'tool_call_delta'); + const end = chunks.find((c) => c.type === 'tool_call_end'); + expect(start).toMatchObject({ type: 'tool_call_start', id: 'call_1', name: 'f' }); + expect(delta).toMatchObject({ + type: 'tool_call_delta', + id: 'call_1', + argsJsonDelta: '{"a":1}', + }); + expect(end).toMatchObject({ type: 'tool_call_end', id: 'call_1' }); + }); + + it('folds a mid-stream error into an error chunk', async () => { + const adapter = createOpenAiAdapter({ + fetch: () => + Promise.resolve( + sse([ + streamChunk([{ index: 0, delta: { content: 'partial' }, finish_reason: null }]), + { error: { message: 'mid-stream failure', type: 'server_error', code: null } }, + ]), + ), + maxRetries: 0, + }); + const chunks = await collect(adapter.stream(REQ, 'k')); + expect(chunks.some((c) => c.type === 'text_delta')).toBe(true); + expect(chunks.at(-1)?.type).toBe('error'); + }); + + it('generate tolerates an empty-choices completion (no content, zero usage)', async () => { + const adapter = createOpenAiAdapter({ + fetch: () => + Promise.resolve( + new Response( + JSON.stringify({ + id: 'c', + object: 'chat.completion', + created: 0, + model: 'gpt-4o', + choices: [], + }), + { status: 200, headers: { 'content-type': 'application/json' } }, + ), + ), + maxRetries: 0, + }); + const result = await adapter.generate(REQ, 'k'); + expect(result.content).toEqual([]); + expect(result.stopReason).toBe('stop'); + expect(result.usage).toEqual({ inputTokens: 0, outputTokens: 0 }); + }); +}); + +describe('OpenAI-compatible adapter — reasoning + structured output (ADR-0030)', () => { + const REQ = { + model: 'deepseek-chat', + messages: [{ role: 'user' as const, content: [{ type: 'text' as const, text: 'hi' }] }], + }; + + it('folds DeepSeek reasoning_content into reasoning_start/delta/end before the text', async () => { + const adapter = createOpenAiAdapter({ + providerId: 'deepseek', + fetch: () => + Promise.resolve( + sse([ + dchunk({ role: 'assistant', reasoning_content: 'let me think' }), + dchunk({ reasoning_content: ' more' }), + dchunk({ content: 'answer' }, 'stop'), + ]), + ), + maxRetries: 0, + }); + const chunks = await collect(adapter.stream(REQ, 'k')); + const types = chunks.map((c) => c.type); + expect(types.indexOf('reasoning_start')).toBeGreaterThanOrEqual(0); + expect(types.indexOf('reasoning_end')).toBeLessThan(types.indexOf('text_delta')); // reasoning closes before text + expect(chunks.filter((c) => c.type === 'reasoning_delta')).toHaveLength(2); + }); + + it('mapContent emits a reasoning part from reasoning_content', () => { + const parts = mapContent({ content: 'answer', reasoning_content: 'because' }, 'deepseek'); + expect(parts[0]).toEqual({ type: 'reasoning', text: 'because' }); + expect(parts[1]).toEqual({ type: 'text', text: 'answer' }); + }); + + it('mapUsage surfaces reasoning_tokens as reasoningTokens', () => { + expect( + mapUsage({ + prompt_tokens: 10, + completion_tokens: 20, + completion_tokens_details: { reasoning_tokens: 12 }, + }), + ).toEqual({ inputTokens: 10, outputTokens: 20, reasoningTokens: 12 }); + }); + + it('lowers responseFormat json to response_format json_schema', async () => { + let sent: Record = {}; + const adapter = createOpenAiAdapter({ + fetch: (_i, init) => { + sent = parseJsonBody(init); + return Promise.resolve(okResponse()); + }, + maxRetries: 0, + }); + await adapter.generate( + { + model: 'gpt-4o', + responseFormat: { type: 'json', schema: { type: 'object' }, name: 'out' }, + messages: REQ.messages, + }, + 'k', + ); + expect(sent['response_format']).toEqual({ + type: 'json_schema', + json_schema: { name: 'out', schema: { type: 'object' }, strict: true }, + }); + }); + + it('lowers responseFormat json to json_object for DeepSeek (json_schema 400s there)', async () => { + let sent: Record = {}; + const adapter = createOpenAiAdapter({ + providerId: 'deepseek', + fetch: (_i, init) => { + sent = parseJsonBody(init); + return Promise.resolve(okResponse()); + }, + maxRetries: 0, + }); + await adapter.generate( + { + model: 'deepseek-chat', + responseFormat: { type: 'json', schema: { type: 'object' }, name: 'out' }, + messages: REQ.messages, + }, + 'k', + ); + expect(sent['response_format']).toEqual({ type: 'json_object' }); + }); +}); + +describe('OpenAI-compatible adapter — reasoning close edges', () => { + const REQ = { + model: 'deepseek-chat', + messages: [{ role: 'user' as const, content: [{ type: 'text' as const, text: 'hi' }] }], + }; + + it('closes reasoning before a tool call', async () => { + const adapter = createOpenAiAdapter({ + providerId: 'deepseek', + fetch: () => + Promise.resolve( + sse([ + dchunk({ reasoning_content: 'think' }), + dchunk({ + tool_calls: [ + { index: 0, id: 't1', type: 'function', function: { name: 'f', arguments: '{}' } }, + ], + }), + dchunk({}, 'tool_calls'), + ]), + ), + maxRetries: 0, + }); + const types = (await collect(adapter.stream(REQ, 'k'))).map((c) => c.type); + expect(types.indexOf('reasoning_end')).toBeLessThan(types.indexOf('tool_call_start')); + }); + + it('closes reasoning at finish when no content follows', async () => { + const adapter = createOpenAiAdapter({ + providerId: 'deepseek', + fetch: () => + Promise.resolve(sse([dchunk({ reasoning_content: 'think' }), dchunk({}, 'stop')])), + maxRetries: 0, + }); + const chunks = await collect(adapter.stream(REQ, 'k')); + expect(chunks.some((c) => c.type === 'reasoning_end')).toBe(true); + expect(chunks.at(-1)?.type).toBe('stop'); + }); +}); + +describe('OpenAI-compatible adapter — robustness (review fixes)', () => { + it('parseToolArgs degrades malformed tool arguments to {} (via mapContent)', () => { + const parts = mapContent( + { + content: null, + tool_calls: [{ id: 't1', function: { name: 'f', arguments: '{not json' } }], + }, + 'openai', + ); + expect(parts).toEqual([{ type: 'tool_call', id: 't1', name: 'f', args: {} }]); + }); + + it('sanitizes an invalid json_schema name to OpenAI rules', async () => { + let sent: Record = {}; + const adapter = createOpenAiAdapter({ + fetch: (_i, init) => { + sent = parseJsonBody(init); + return Promise.resolve(okResponse()); + }, + maxRetries: 0, + }); + await adapter.generate( + { + model: 'gpt-4o', + responseFormat: { type: 'json', schema: { type: 'object' }, name: 'my schema!' }, + messages: [{ role: 'user', content: [{ type: 'text', text: 'hi' }] }], + }, + 'k', + ); + const rf = sent['response_format'] as { json_schema: { name: string } }; + expect(rf.json_schema.name).toBe('my_schema_'); // spaces/'!' → '_' + }); + + it('classifies an APIError code via firstNonEmptyString', () => { + const err = new APIError(400, undefined, 'bad', undefined); + Object.assign(err, { code: 'invalid_request' }); + expect(openaiErrorToLlmError(err, 'openai')).toMatchObject({ + kind: 'bad_request', + code: 'invalid_request', + }); + }); +}); + +describe('OpenAI-compatible adapter — baseURL SSRF guard', () => { + it('accepts a public HTTPS base URL', () => { + expect(() => createOpenAiAdapter({ baseURL: 'https://api.openai.com/v1' })).not.toThrow(); + }); + + it('rejects a non-HTTPS base URL', () => { + const insecure = 'http://api.openai.com'; // NOSONAR — cleartext URL is the exact input under test + expect(() => createOpenAiAdapter({ baseURL: insecure })).toThrow(InvalidBaseUrlError); + }); + + it('rejects the cloud-metadata link-local address', () => { + expect(() => createOpenAiAdapter({ baseURL: 'https://169.254.169.254/latest' })).toThrow( + InvalidBaseUrlError, + ); + }); + + it('rejects loopback and RFC-1918 private ranges', () => { + for (const url of [ + 'https://localhost:8080', + 'https://127.0.0.1', + 'https://10.0.0.5', + 'https://192.168.1.1', + 'https://172.16.0.1', + 'https://172.31.255.255', + 'https://service.internal', + 'https://0.0.0.0', + ]) { + expect(() => createOpenAiAdapter({ baseURL: url })).toThrow(InvalidBaseUrlError); + } + }); + + it('rejects evasions the URL parser normalizes (userinfo, decimal IP, trailing dot, IPv6)', () => { + for (const url of [ + 'https://evil.com@169.254.169.254/latest', // userinfo trick — real host is the metadata IP + 'https://2130706433/', // decimal-encoded 127.0.0.1 + 'https://0x7f000001/', // hex-encoded 127.0.0.1 + 'https://0177.0.0.1/', // octal-encoded 127.0.0.1 + 'https://127.0.0.1./', // trailing-dot loopback + 'https://LOCALHOST/', // case-variant localhost + 'https://[::1]/', // IPv6 loopback + 'https://[::ffff:127.0.0.1]/', // IPv4-mapped IPv6 loopback + 'https://[::ffff:169.254.169.254]/', // IPv4-mapped IPv6 → cloud metadata + 'https://[::ffff:10.0.0.1]/', // IPv4-mapped IPv6 → private 10/8 + 'https://[::ffff:192.168.1.1]/', // IPv4-mapped IPv6 → private 192.168/16 + 'https://[::ffff:172.16.0.1]/', // IPv4-mapped IPv6 → private 172.16/12 + 'https://[64:ff9b::169.254.169.254]/', // NAT64 → cloud metadata + 'https://[fd00::1]/', // IPv6 unique-local + 'https://[fe80::1]/', // IPv6 link-local + 'https://0.0.0.0/', // unspecified 0.0.0.0/8 + 'https://100.64.0.1/', // CGNAT 100.64.0.0/10 + 'https://100.127.255.255/', // CGNAT upper bound + ]) { + expect(() => createOpenAiAdapter({ baseURL: url })).toThrow(InvalidBaseUrlError); + } + }); + + it('accepts an uppercase HTTPS scheme (normalized) and a public host', () => { + expect(() => createOpenAiAdapter({ baseURL: 'HTTPS://API.OPENAI.COM/v1' })).not.toThrow(); + }); + + it('does not reject the safe public 172.x range outside 16–31', () => { + expect(() => createOpenAiAdapter({ baseURL: 'https://172.32.0.1' })).not.toThrow(); + }); + + it('does not validate the built-in DeepSeek default (no caller baseURL)', () => { + expect(() => createOpenAiAdapter({ providerId: 'deepseek' })).not.toThrow(); + }); +}); + +describe('OpenAI-compatible adapter — truncation + refusal normalization', () => { + const REQ = { + model: 'gpt-4o', + messages: [{ role: 'user' as const, content: [{ type: 'text' as const, text: 'hi' }] }], + }; + + it('emits a transport error (not a clean stop) when a stream ends without finish_reason', async () => { + const adapter = createOpenAiAdapter({ + fetch: () => Promise.resolve(sse([dchunk({ content: 'partial' }, null)])), + maxRetries: 0, + }); + const chunks = await collect(adapter.stream(REQ, 'k')); + expect(chunks.some((c) => c.type === 'text_delta')).toBe(true); + const last = chunks.at(-1); + expect(last?.type).toBe('error'); + if (last?.type === 'error') { + expect(last.error.kind).toBe('transport'); + expect(last.error.retryable).toBe(true); + } + }); + + it('normalizes a streamed refusal to a content_filter stop', async () => { + const adapter = createOpenAiAdapter({ + fetch: () => + Promise.resolve( + sse([dchunk({ role: 'assistant', refusal: "I can't help with that" }, 'stop')]), + ), + maxRetries: 0, + }); + const chunks = await collect(adapter.stream(REQ, 'k')); + const stop = chunks.at(-1); + expect(stop?.type).toBe('stop'); + if (stop?.type === 'stop') { + expect(stop.stopReason).toBe('content_filter'); + } + }); + + it('drops an empty-string content delta (no zero-length text_delta)', async () => { + const adapter = createOpenAiAdapter({ + fetch: () => + Promise.resolve(sse([dchunk({ content: '' }), dchunk({ content: 'real' }, 'stop')])), + maxRetries: 0, + }); + const chunks = await collect(adapter.stream(REQ, 'k')); + const textDeltas = chunks.filter((c) => c.type === 'text_delta'); + expect(textDeltas).toHaveLength(1); + expect(textDeltas[0]).toMatchObject({ text: 'real' }); + }); + + it('normalizes a non-streaming refusal to a content_filter stop', async () => { + const adapter = createOpenAiAdapter({ + fetch: () => + Promise.resolve( + new Response( + JSON.stringify({ + id: 'c', + object: 'chat.completion', + created: 0, + model: 'gpt-4o', + choices: [ + { + index: 0, + message: { role: 'assistant', content: null, refusal: "I won't do that" }, + finish_reason: 'stop', + logprobs: null, + }, + ], + usage: { prompt_tokens: 5, completion_tokens: 2, total_tokens: 7 }, + }), + { status: 200, headers: { 'content-type': 'application/json' } }, + ), + ), + maxRetries: 0, + }); + const result = await adapter.generate(REQ, 'k'); + expect(result.content).toEqual([]); + expect(result.stopReason).toBe('content_filter'); + }); +}); diff --git a/packages/llm/src/adapters/openai.ts b/packages/llm/src/adapters/openai.ts new file mode 100644 index 00000000..c86f6ce6 --- /dev/null +++ b/packages/llm/src/adapters/openai.ts @@ -0,0 +1,669 @@ +import OpenAI, { + APIConnectionError, + APIConnectionTimeoutError, + APIError, + APIUserAbortError, +} from 'openai'; + +import type { ContentPart, StopReason } from '@relavium/shared'; + +import { assertStreamable, assertSupported } from '../capabilities.js'; +import { InvalidBaseUrlError } from '../errors.js'; +import { LlmProviderError, kindFromHttpStatus, makeLlmError } from '../llm-error.js'; +import { normalizeToolCall, toWire } from '../tool-normalizer.js'; +import type { + CapabilityFlags, + LlmError, + LlmMessage, + LlmProvider, + LlmRequest, + LlmResult, + ProviderId, + StreamChunk, + ToolChoice, + ToolDef, + Usage, +} from '../types.js'; + +import { REASONING_ID, isAbortSignal } from './shared.js'; + +/** + * The shared OpenAI-compatible adapter (1.G) — one implementation over the `openai` SDK serving both + * **OpenAI** and **DeepSeek** (DeepSeek via a custom `baseURL`, no separate dependency). Like the + * Anthropic adapter it lives behind the `@relavium/llm` seam: the SDK is imported only here, and no + * vendor type crosses back out (`generate` → `LlmResult`, `stream` → `StreamChunk`s, failures → + * `LlmError`). The provider id (`openai` | `deepseek`) selects cost pricing + capabilities while one + * fold/normalization path is shared. See + * [llm-provider-seam.md](../../../../docs/reference/shared-core/llm-provider-seam.md). + */ + +const DEEPSEEK_BASE_URL = 'https://api.deepseek.com'; + +/** OpenAI's common-path capability surface (reasoning models are a separate, non-common path). */ +const OPENAI_SUPPORTS: CapabilityFlags = { + tools: true, + streaming: true, + parallelToolCalls: true, + vision: true, + promptCache: true, // automatic prompt caching; no separate write charge + reasoning: false, +}; + +/** DeepSeek's capability surface (deepseek-reasoner exposes reasoning; no vision). */ +const DEEPSEEK_SUPPORTS: CapabilityFlags = { + tools: true, + streaming: true, + parallelToolCalls: true, + vision: false, + promptCache: true, // cache-hit input is discounted + reasoning: true, +}; + +const ZERO_USAGE: Usage = { inputTokens: 0, outputTokens: 0 }; + +// --- Normalization: OpenAI wire → canonical --------------------------------------------------- + +/** Map an OpenAI/DeepSeek finish reason to the canonical 5-value enum. */ +export function mapStopReason(reason: string | null | undefined): StopReason { + switch (reason) { + case 'length': + return 'length'; + case 'tool_calls': + case 'function_call': + return 'tool_use'; + case 'content_filter': + return 'content_filter'; + default: + // 'stop' / null / a future reason the SDK doesn't type → graceful 'stop' (matches Anthropic). + return 'stop'; + } +} + +/** + * Map OpenAI/DeepSeek usage to the canonical **NET** `Usage`. `prompt_tokens` is **gross** (it + * includes cache reads), so net input = `prompt_tokens − cached`. The cache count comes from + * OpenAI's `prompt_tokens_details.cached_tokens` or DeepSeek's top-level `prompt_cache_hit_tokens`. + */ +export function mapUsage(usage: { + prompt_tokens?: number | null; + completion_tokens?: number | null; + prompt_tokens_details?: { cached_tokens?: number | null } | null; + prompt_cache_hit_tokens?: number | null; + completion_tokens_details?: { reasoning_tokens?: number | null } | null; +}): Usage { + const cached = usage.prompt_tokens_details?.cached_tokens ?? usage.prompt_cache_hit_tokens ?? 0; + const gross = usage.prompt_tokens ?? 0; + const out: Usage = { + inputTokens: Math.max(0, gross - cached), + outputTokens: usage.completion_tokens ?? 0, + }; + if (cached > 0) { + out.cacheReadTokens = cached; + } + // Reasoning tokens are already counted inside completion_tokens (billing unchanged); surface for + // observability only (ADR-0030). + const reasoning = usage.completion_tokens_details?.reasoning_tokens ?? 0; + if (reasoning > 0) { + out.reasoningTokens = reasoning; + } + return out; +} + +/** OpenAI tool-call arguments arrive as a JSON string; parse to the canonical `args` (empty → `{}`). */ +function parseToolArgs(raw: string): unknown { + try { + return JSON.parse(raw.length > 0 ? raw : '{}'); + } catch { + // Deliberate (prior review decision, locked by a unit test): a malformed provider tool-arg + // payload degrades to {} rather than throwing and failing the whole result. A stricter + // surface-as-fatal alternative was raised in PR #9 review and intentionally not adopted. + return {}; + } +} + +/** Fold a non-streaming assistant message into canonical content parts (text + tool_call). */ +export function mapContent( + message: { + content: string | null; + // DeepSeek-R1 / Kimi return reasoning as a top-level field the OpenAI SDK does not type; the SDK + // passes unknown response fields through, so it is present at runtime when the model emits it. + reasoning_content?: string | null; + tool_calls?: + | ReadonlyArray<{ id: string; function?: { name: string; arguments: string } }> + | undefined; + }, + provider: ProviderId, +): ContentPart[] { + const parts: ContentPart[] = []; + if ( + message.reasoning_content !== null && + message.reasoning_content !== undefined && + message.reasoning_content.length > 0 + ) { + parts.push({ type: 'reasoning', text: message.reasoning_content }); + } + if (message.content !== null && message.content.length > 0) { + parts.push({ type: 'text', text: message.content }); + } + for (const call of message.tool_calls ?? []) { + if (call.function === undefined) { + continue; // custom (non-function) tool calls are off the common path + } + parts.push( + normalizeToolCall(provider, { + id: call.id, + name: call.function.name, + args: parseToolArgs(call.function.arguments), + }), + ); + } + return parts; +} + +/** The first non-empty string of the candidates, else undefined — the normalized error `code`. */ +function firstNonEmptyString(...values: readonly unknown[]): string | undefined { + for (const value of values) { + if (typeof value === 'string' && value.length > 0) { + return value; + } + } + return undefined; +} + +/** Normalize an SDK `APIError` into an `LlmError`, typed by the structural subset it reads. */ +function mapOpenAiApiError( + err: { status?: unknown; code?: unknown; type?: unknown; message: string }, + provider: ProviderId, +): LlmError { + const status = typeof err.status === 'number' ? err.status : undefined; + const code = firstNonEmptyString(err.code, err.type); + const kind = status === undefined ? 'unknown' : kindFromHttpStatus(status); + return makeLlmError({ + provider, + kind, + message: err.message, + ...(status === undefined ? {} : { status }), + ...(code === undefined ? {} : { code }), + }); +} + +/** Classify any SDK throwable into a normalized `LlmError` — no vendor error shape escapes. */ +export function openaiErrorToLlmError(err: unknown, provider: ProviderId): LlmError { + if (err instanceof APIUserAbortError) { + return makeLlmError({ provider, kind: 'cancelled', message: 'request aborted' }); + } + if (err instanceof APIConnectionTimeoutError) { + return makeLlmError({ provider, kind: 'timeout', message: err.message }); + } + if (err instanceof APIConnectionError) { + return makeLlmError({ provider, kind: 'transport', message: err.message }); + } + if (err instanceof APIError) { + return mapOpenAiApiError(err, provider); + } + return makeLlmError({ + provider, + kind: 'unknown', + message: err instanceof Error ? err.message : 'unknown provider error', + }); +} + +// --- Request building: canonical → OpenAI wire ----------------------------------------------- + +type ToolCallPart = Extract; +type ToolResultPart = Extract; + +/** Concatenate the text parts of a message into one string (OpenAI content is a plain string here). */ +function textOf(content: readonly ContentPart[]): string { + return content.map((part) => (part.type === 'text' ? part.text : '')).join(''); +} + +/** Map one canonical message to one or more OpenAI message params (tool results split out). */ +function toOpenAiMessages(message: LlmMessage): OpenAI.ChatCompletionMessageParam[] { + switch (message.role) { + case 'user': + return [{ role: 'user', content: textOf(message.content) }]; + case 'assistant': { + const toolCalls = message.content + .filter((part): part is ToolCallPart => part.type === 'tool_call') + .map((part) => ({ + id: part.id, + type: 'function' as const, + function: { name: part.name, arguments: JSON.stringify(part.args) ?? '{}' }, + })); + const msg: OpenAI.ChatCompletionAssistantMessageParam = { role: 'assistant' }; + const text = textOf(message.content); + if (text.length > 0) { + msg.content = text; + } + if (toolCalls.length > 0) { + msg.tool_calls = toolCalls; + } + // An assistant message that lowered to neither text nor tool calls (e.g. reasoning-only — reasoning + // is ephemeral and never replayed, ADR-0030) would be wire-invalid; emit empty content instead. + if (msg.content === undefined && msg.tool_calls === undefined) { + msg.content = ''; + } + return [msg]; + } + case 'tool': + // Each tool_result rides in its own {role:'tool'} message keyed by the tool-call id. + return message.content + .filter((part): part is ToolResultPart => part.type === 'tool_result') + .map((part) => ({ + role: 'tool', + tool_call_id: part.toolCallId, + content: + typeof part.result === 'string' ? part.result : (JSON.stringify(part.result) ?? ''), + })); + } +} + +function toOpenAiTool(toolDef: ToolDef, provider: ProviderId): OpenAI.ChatCompletionTool { + const wire = toWire(toolDef, provider); + if (!('function' in wire)) { + throw new Error('unreachable: the OpenAI wire shape always carries a function'); + } + const fn: OpenAI.ChatCompletionFunctionTool['function'] = { + name: wire.function.name, + // The canonical JSON-Schema is a valid OpenAI function-parameters object; bridge at the boundary. + parameters: wire.function.parameters as Record, + }; + if (wire.function.description !== undefined) { + fn.description = wire.function.description; + } + return { type: 'function', function: fn }; +} + +function toOpenAiToolChoice(choice: ToolChoice): OpenAI.ChatCompletionToolChoiceOption { + if (choice === 'auto') { + return 'auto'; + } + if (choice === 'none') { + return 'none'; + } + if (choice === 'required') { + return 'required'; + } + return { type: 'function', function: { name: choice.name } }; +} + +/** OpenAI requires the json_schema `name` to match `^[a-zA-Z0-9_-]{1,64}$`; sanitize a caller's name. */ +function toJsonSchemaName(name: string | undefined): string { + if (name === undefined) { + return 'response'; + } + const sanitized = name.replace(/[^a-zA-Z0-9_-]/g, '_').slice(0, 64); + return sanitized.length > 0 ? sanitized : 'response'; +} + +/** The shared request body (everything except the `stream` discriminant each method sets). */ +function buildCommonBody( + req: LlmRequest, + provider: ProviderId, +): Omit { + const messages: OpenAI.ChatCompletionMessageParam[] = []; + if (req.system !== undefined) { + messages.push({ role: 'system', content: req.system }); + } + for (const message of req.messages) { + messages.push(...toOpenAiMessages(message)); + } + const body: Omit = { + model: req.model, + messages, + }; + if (req.tools !== undefined) { + body.tools = req.tools.map((tool) => toOpenAiTool(tool, provider)); + } + if (req.toolChoice !== undefined) { + body.tool_choice = toOpenAiToolChoice(req.toolChoice); + } + if (req.responseFormat?.type === 'json') { + if (provider === 'deepseek') { + // DeepSeek only supports json_object; json_schema returns 400 (ADR-0030). + // Note: DeepSeek json_object also requires the word "json" to appear in the prompt. + body.response_format = { type: 'json_object' }; + } else { + // Native structured output for OpenAI (ADR-0030). The canonical JSON-Schema bridges here. + body.response_format = { + type: 'json_schema', + json_schema: { + name: toJsonSchemaName(req.responseFormat.name), + schema: req.responseFormat.schema as Record, + strict: req.responseFormat.strict ?? true, + }, + }; + } + } + if (req.temperature !== undefined) { + body.temperature = req.temperature; + } + if (req.maxTokens !== undefined) { + body.max_tokens = req.maxTokens; + } + if (req.stopSequences !== undefined) { + body.stop = req.stopSequences; + } + if (req.providerOptions === undefined) { + return body; + } + // The typed escape hatch (1.D): `body` is spread LAST so mapped common-path fields always win. + return { ...req.providerOptions, ...body }; +} + +function buildRequestOptions(req: LlmRequest): { signal?: AbortSignal } { + return isAbortSignal(req.signal) ? { signal: req.signal } : {}; +} + +/** + * True for a hostname that resolves to a loopback, private (RFC-1918 / ULA), link-local, or + * cloud-metadata address — the literal forms an SSRF payload would use. `host` is the already- + * normalized `URL.hostname` (lowercased, IPv6 brackets stripped, decimal/hex IPs canonicalized). + */ +function isPrivateOrLocalHost(host: string): boolean { + if (host.includes(':')) { + // IPv6 literal. An embedded-IPv4 form (IPv4-mapped `::ffff:a.b.c.d` and well-known NAT64 + // `64:ff9b::a.b.c.d`) routes to its embedded IPv4 on a dual-stack host — decode it and re-check + // the IPv4, so e.g. `::ffff:169.254.169.254` (which Node normalizes to `::ffff:a9fe:a9fe`) + // cannot slip past as a "non-loopback" IPv6. + const embeddedHex = /^(?:::ffff:|64:ff9b::)([0-9a-f]{1,4}):([0-9a-f]{1,4})$/.exec(host); + if (embeddedHex !== null) { + const hi = Number.parseInt(embeddedHex[1] ?? '', 16); + const lo = Number.parseInt(embeddedHex[2] ?? '', 16); + return isPrivateOrLocalHost(`${hi >> 8}.${hi & 0xff}.${lo >> 8}.${lo & 0xff}`); + } + const embeddedDotted = /^(?:::ffff:|64:ff9b::)(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})$/.exec(host); + if (embeddedDotted !== null) { + return isPrivateOrLocalHost(embeddedDotted[1] ?? ''); + } + return ( + host === '::1' || // loopback + host === '::' || // unspecified + host.startsWith('fe80:') || // link-local fe80::/10 + host.startsWith('fc') || // unique-local fc00::/7 + host.startsWith('fd') + ); + } + if ( + host === 'localhost' || + host.endsWith('.localhost') || + host.endsWith('.internal') || + host.endsWith('.local') || + host.startsWith('0.') || // 0.0.0.0/8 — "this host" / unspecified + host.startsWith('127.') || // loopback 127/8 + host.startsWith('10.') || // private 10/8 + host.startsWith('192.168.') || // private 192.168/16 + host.startsWith('169.254.') // IPv4 link-local — cloud metadata (169.254.169.254) + ) { + return true; + } + // 172.16.0.0–172.31.255.255 (private) and 100.64.0.0–100.127.255.255 (CGNAT). + const m172 = /^172\.(\d{1,3})\./.exec(host); + if (m172 !== null) { + const octet = Number(m172[1]); + if (octet >= 16 && octet <= 31) return true; + } + const m100 = /^100\.(\d{1,3})\./.exec(host); + if (m100 !== null) { + const octet = Number(m100[1]); + if (octet >= 64 && octet <= 127) return true; + } + return false; +} + +/** + * Throw if a caller-supplied `baseURL` is not a safe public HTTPS endpoint — a construction-time SSRF + * guard so a hostile base URL can't redirect egress (with the real key) to an internal/metadata host. + * The `URL` parser normalizes the evasions string-matching misses (userinfo `@`, decimal/hex IPs, + * trailing dots, case, IPv6 brackets). This is a best-effort literal block; the *complete* SSRF guard + * (DNS resolution to catch a public name pointing at a private IP, redirect re-validation) is the + * shared security primitive's job (security-review.md) — a forward obligation, not duplicated here. + */ +function assertHttpsBaseUrl(url: string): void { + let parsed: URL; + try { + parsed = new URL(url); + } catch { + throw new InvalidBaseUrlError(url, 'not a valid URL'); + } + if (parsed.protocol !== 'https:') { + throw new InvalidBaseUrlError(url, `must use HTTPS, got '${parsed.protocol}'`); + } + const host = parsed.hostname.toLowerCase().replace(/^\[/, '').replace(/\]$/, ''); + if (isPrivateOrLocalHost(host)) { + throw new InvalidBaseUrlError(url, 'resolves to a private, loopback, or link-local address'); + } +} + +// --- Streaming fold -------------------------------------------------------------------------- + +/** + * Fold one streamed `tool_calls` delta into chunks, tracking the tool id by its stream index. OpenAI + * sends `id`/`name` only on the first delta for an index; subsequent deltas carry argument fragments. + */ +function foldToolCallDelta( + delta: { index: number; id?: string; function?: { name?: string; arguments?: string } }, + toolIdByIndex: Map, +): StreamChunk[] { + const chunks: StreamChunk[] = []; + const args = delta.function?.arguments ?? ''; + const existing = toolIdByIndex.get(delta.index); + if (existing === undefined) { + const id = delta.id; + const name = delta.function?.name; + if (id === undefined || name === undefined) { + return chunks; // defensive: can't start a tool call without an id + name + } + toolIdByIndex.set(delta.index, id); + chunks.push({ type: 'tool_call_start', id, name }); + if (args.length > 0) { + chunks.push({ type: 'tool_call_delta', id, argsJsonDelta: args }); + } + return chunks; + } + if (args.length > 0) { + chunks.push({ type: 'tool_call_delta', id: existing, argsJsonDelta: args }); + } + return chunks; +} + +/** Read a DeepSeek/Kimi `reasoning_content` delta the OpenAI SDK does not type (present at runtime). */ +function readReasoningContent(delta: unknown): string | undefined { + if (typeof delta === 'object' && delta !== null && 'reasoning_content' in delta) { + const value = (delta as { reasoning_content?: unknown }).reasoning_content; + return typeof value === 'string' ? value : undefined; + } + return undefined; +} + +/** Mutable fold state threaded across the streamed chunks. */ +interface OpenAiStreamState { + reasoningOpen: boolean; + stopReason: StopReason; + /** True once a terminal `finish_reason` is seen — a stream that ends without one was truncated. */ + sawTerminal: boolean; + /** True once the provider streamed a refusal (delta.refusal) — the stop normalizes to content_filter. */ + refused: boolean; + readonly toolIdByIndex: Map; +} + +/** Emit a `reasoning_end` (closing the ephemeral reasoning channel) if one is open. */ +function closeReasoning(state: OpenAiStreamState, out: StreamChunk[]): void { + if (state.reasoningOpen) { + out.push({ type: 'reasoning_end', id: REASONING_ID }); + state.reasoningOpen = false; + } +} + +/** Fold one chat-completion chunk into the chunks to emit, mutating the streamed fold state. */ +function foldChatChunk(chunk: OpenAI.ChatCompletionChunk, state: OpenAiStreamState): StreamChunk[] { + const out: StreamChunk[] = []; + const choice = chunk.choices[0]; + if (choice === undefined) { + return out; + } + // A streamed refusal (delta.refusal) is a safety decline, not an answer — record it so the terminal + // stop normalizes to content_filter rather than masking the refusal as a successful stop. + const refusal = choice.delta.refusal; + if (typeof refusal === 'string' && refusal.length > 0) { + state.refused = true; + } + // DeepSeek-R1 / Kimi stream reasoning first (content null) — open the ephemeral reasoning channel. + const reasoning = readReasoningContent(choice.delta); + if (reasoning !== undefined && reasoning.length > 0) { + if (!state.reasoningOpen) { + out.push({ type: 'reasoning_start', id: REASONING_ID }); + state.reasoningOpen = true; + } + out.push({ type: 'reasoning_delta', id: REASONING_ID, text: reasoning }); + } + // Gate on length: an empty-string content delta is not real text, and emitting it would also close + // the reasoning channel prematurely. + if (choice.delta.content != null && choice.delta.content.length > 0) { + closeReasoning(state, out); + out.push({ type: 'text_delta', text: choice.delta.content }); + } + for (const toolCall of choice.delta.tool_calls ?? []) { + closeReasoning(state, out); + out.push(...foldToolCallDelta(toolCall, state.toolIdByIndex)); + } + if (choice.finish_reason != null) { + closeReasoning(state, out); + state.sawTerminal = true; + state.stopReason = state.refused ? 'content_filter' : mapStopReason(choice.finish_reason); + // OpenAI has no per-tool end event — every tracked tool finalizes at finish_reason. + for (const id of state.toolIdByIndex.values()) { + out.push({ type: 'tool_call_end', id }); + } + state.toolIdByIndex.clear(); + } + return out; +} + +/** Fold the OpenAI chat-completion event stream into the canonical `StreamChunk` sequence. */ +async function* streamChunks( + client: OpenAI, + req: LlmRequest, + provider: ProviderId, +): AsyncIterable { + const state: OpenAiStreamState = { + reasoningOpen: false, + stopReason: 'stop', + sawTerminal: false, + refused: false, + toolIdByIndex: new Map(), + }; + let usage: Usage = ZERO_USAGE; + let sdkStream: AsyncIterable; + try { + sdkStream = await client.chat.completions.create( + { ...buildCommonBody(req, provider), stream: true, stream_options: { include_usage: true } }, + buildRequestOptions(req), + ); + } catch (err) { + yield { type: 'error', error: openaiErrorToLlmError(err, provider) }; + return; + } + try { + for await (const chunk of sdkStream) { + if (chunk.usage) { + usage = mapUsage(chunk.usage); // the include_usage chunk arrives last, with empty choices + } + for (const out of foldChatChunk(chunk, state)) { + yield out; + } + } + } catch (err) { + yield { type: 'error', error: openaiErrorToLlmError(err, provider) }; + return; + } + // A stream that ends without a terminal finish_reason was truncated (dropped connection, partial + // body) — surface it as a retryable transport error, never a clean stop that hides lost content. + if (!state.sawTerminal) { + yield { + type: 'error', + error: makeLlmError({ + provider, + kind: 'transport', + message: 'stream ended before a terminal finish_reason (truncated response)', + }), + }; + return; + } + yield { type: 'stop', stopReason: state.stopReason, usage }; +} + +// --- The adapter ----------------------------------------------------------------------------- + +/** The two provider ids the OpenAI-compatible adapter can serve (a strict subset of `ProviderId`). */ +type OpenAiProviderId = Extract; + +/** Dependencies the conformance replayer / tests inject (and the provider id + base URL selector). */ +export interface OpenAiAdapterDeps { + /** Which provider this instance serves — selects capabilities, cost pricing, and the default base URL. */ + readonly providerId?: OpenAiProviderId; + /** Override the API base URL (DeepSeek defaults to `api.deepseek.com`). Validated HTTPS-only. */ + readonly baseURL?: string; + /** Inject a `fetch` (the replayer/recorder) in place of the network. */ + readonly fetch?: (input: string | URL | Request, init?: RequestInit) => Promise; + /** Override the SDK retry count (the replayer sets 0 for deterministic, fast tests). */ + readonly maxRetries?: number; +} + +/** Build an OpenAI-compatible `LlmProvider`. Exposed as `openaiAdapter` / `deepseekAdapter`. */ +export function createOpenAiAdapter(deps: OpenAiAdapterDeps = {}): LlmProvider { + const providerId: OpenAiProviderId = deps.providerId ?? 'openai'; + const supports = providerId === 'deepseek' ? DEEPSEEK_SUPPORTS : OPENAI_SUPPORTS; + const baseURL = deps.baseURL ?? (providerId === 'deepseek' ? DEEPSEEK_BASE_URL : undefined); + // Validate caller-supplied base URLs at construction time: HTTPS-only, no internal addresses. + if (deps.baseURL !== undefined) { + assertHttpsBaseUrl(deps.baseURL); + } + const createClient = (key: string): OpenAI => + new OpenAI({ + apiKey: key, + ...(baseURL === undefined ? {} : { baseURL }), + ...(deps.fetch === undefined ? {} : { fetch: deps.fetch }), + ...(deps.maxRetries === undefined ? {} : { maxRetries: deps.maxRetries }), + }); + + return { + id: providerId, + supports, + async generate(req: LlmRequest, key: string): Promise { + assertSupported(providerId, supports, req); // fail fast, never silently drop an unsupported feature + const client = createClient(key); + try { + const completion = await client.chat.completions.create( + { ...buildCommonBody(req, providerId), stream: false }, + buildRequestOptions(req), + ); + const choice = completion.choices[0]; + // A non-null refusal is a safety decline — normalize to content_filter, not a clean stop. + const refused = + typeof choice?.message.refusal === 'string' && choice.message.refusal.length > 0; + return { + // An empty `choices` array is a complete-but-empty 200 — a clean empty stop, not an error. + content: choice === undefined ? [] : mapContent(choice.message, providerId), + stopReason: refused ? 'content_filter' : mapStopReason(choice?.finish_reason), + usage: completion.usage ? mapUsage(completion.usage) : ZERO_USAGE, + raw: completion, + }; + } catch (err) { + throw new LlmProviderError(openaiErrorToLlmError(err, providerId)); + } + }, + stream(req: LlmRequest, key: string): AsyncIterable { + assertSupported(providerId, supports, req); // fail fast on an unsupported feature or no streaming + assertStreamable(providerId, supports); + return streamChunks(createClient(key), req, providerId); + }, + }; +} + +/** The production OpenAI adapter. */ +export const openaiAdapter: LlmProvider = createOpenAiAdapter(); + +/** The production DeepSeek adapter (the shared OpenAI-compatible impl pointed at DeepSeek). */ +export const deepseekAdapter: LlmProvider = createOpenAiAdapter({ providerId: 'deepseek' }); diff --git a/packages/llm/src/adapters/shared.ts b/packages/llm/src/adapters/shared.ts new file mode 100644 index 00000000..52e9157b --- /dev/null +++ b/packages/llm/src/adapters/shared.ts @@ -0,0 +1,16 @@ +/** + * Shared helpers for the provider adapters — the platform-coupled zone (`src/adapters/*`) that may + * reference host globals. Kept here so AbortSignal handling lives in one place across the adapters. + */ + +/** True for a real `AbortSignal` (the host passes one; it structurally satisfies AbortSignalLike). */ +export function isAbortSignal(value: unknown): value is AbortSignal { + return typeof AbortSignal !== 'undefined' && value instanceof AbortSignal; +} + +/** + * The single-track reasoning-channel id used by the OpenAI/DeepSeek and Gemini streaming folds — those + * providers emit one reasoning block per response (no concurrent tracks). A provider that interleaves + * multiple reasoning streams must move to an index-keyed id like the Anthropic adapter (`reasoning-${index}`). + */ +export const REASONING_ID = 'reasoning-0'; diff --git a/packages/llm/src/conformance/deepseek.conformance.test.ts b/packages/llm/src/conformance/deepseek.conformance.test.ts new file mode 100644 index 00000000..9a34250a --- /dev/null +++ b/packages/llm/src/conformance/deepseek.conformance.test.ts @@ -0,0 +1,30 @@ +import { describe, expect, it } from 'vitest'; + +import { createOpenAiAdapter, deepseekAdapter } from '../adapters/openai.js'; +import { DEEPSEEK_FIXTURES } from './fixtures/deepseek.js'; +import { replayFetch } from './replay.js'; +import { defineConformanceSuite, type MakeReplayAdapter } from './spec.js'; + +// DeepSeek is served by the SAME OpenAI-compatible adapter pointed at api.deepseek.com — same fold, +// distinct provider id + cache field. The fetch override keeps the SDK inside src/adapters/*. +const makeReplayAdapter: MakeReplayAdapter = (recorded) => + createOpenAiAdapter({ providerId: 'deepseek', fetch: replayFetch(recorded), maxRetries: 0 }); + +defineConformanceSuite('deepseek', makeReplayAdapter, DEEPSEEK_FIXTURES); + +// Live nightly — runs only when DEEPSEEK_API_KEY is set; skipped in PR mode (testing.md). +const liveKey = process.env['DEEPSEEK_API_KEY'] ?? ''; +describe('deepseek — conformance (live, nightly)', () => { + it.skipIf(liveKey === '')('generate hits the real API and returns canonical text', async () => { + const result = await deepseekAdapter.generate( + { + model: 'deepseek-chat', + maxTokens: 16, + messages: [{ role: 'user', content: [{ type: 'text', text: 'Reply with one word.' }] }], + }, + liveKey, + ); + expect(result.content.some((part) => part.type === 'text')).toBe(true); + expect(result.usage.outputTokens).toBeGreaterThan(0); + }); +}); diff --git a/packages/llm/src/conformance/fixtures/anthropic.ts b/packages/llm/src/conformance/fixtures/anthropic.ts index e9db82b2..488f83ce 100644 --- a/packages/llm/src/conformance/fixtures/anthropic.ts +++ b/packages/llm/src/conformance/fixtures/anthropic.ts @@ -164,6 +164,84 @@ const streamError = sse([ ['error', { type: 'error', error: { type: 'overloaded_error', message: 'Overloaded' } }], ]); +// A stream with a thinking block (thinking + signature deltas) then text — exercises the reasoning +// channel (ADR-0030): thinking_delta → reasoning_delta, signature_delta → reasoning_end signature. +const reasoningStream = sse([ + [ + 'message_start', + { + type: 'message_start', + message: { + id: 'msg_reasoning', + type: 'message', + role: 'assistant', + model: 'claude-opus-4-8', + content: [], + stop_reason: null, + stop_sequence: null, + usage: { input_tokens: 6, output_tokens: 1 }, + }, + }, + ], + [ + 'content_block_start', + { + type: 'content_block_start', + index: 0, + content_block: { type: 'thinking', thinking: '', signature: '' }, + }, + ], + [ + 'content_block_delta', + { + type: 'content_block_delta', + index: 0, + delta: { type: 'thinking_delta', thinking: 'let me think' }, + }, + ], + [ + 'content_block_delta', + { + type: 'content_block_delta', + index: 0, + delta: { type: 'signature_delta', signature: 'sig-xyz' }, + }, + ], + ['content_block_stop', { type: 'content_block_stop', index: 0 }], + [ + 'content_block_start', + { type: 'content_block_start', index: 1, content_block: { type: 'text', text: '' } }, + ], + [ + 'content_block_delta', + { type: 'content_block_delta', index: 1, delta: { type: 'text_delta', text: 'Done.' } }, + ], + ['content_block_stop', { type: 'content_block_stop', index: 1 }], + [ + 'message_delta', + { + type: 'message_delta', + delta: { stop_reason: 'end_turn', stop_sequence: null }, + // The terminal message_delta carries the authoritative cumulative thinking count in + // output_tokens_details — the real wire shape the streaming usage merge must read (ADR-0030). + usage: { output_tokens: 9, output_tokens_details: { thinking_tokens: 4 } }, + }, + ], + ['message_stop', { type: 'message_stop' }], +]); + +// A non-streaming reply produced under responseFormat: json — the content is a JSON document. +const structuredOutput = JSON.stringify({ + id: 'msg_json', + type: 'message', + role: 'assistant', + model: 'claude-opus-4-8', + content: [{ type: 'text', text: '{"ok":true}' }], + stop_reason: 'end_turn', + stop_sequence: null, + usage: { input_tokens: 8, output_tokens: 4 }, +}); + export const ANTHROPIC_FIXTURES: ConformanceFixtures = { textGenerate: { status: 200, body: textMessage }, toolGenerate: { status: 200, body: toolMessage }, @@ -171,11 +249,15 @@ export const ANTHROPIC_FIXTURES: ConformanceFixtures = { toolStream: { status: 200, contentType: 'text/event-stream', body: toolStream }, rateLimit: { status: 429, body: rateLimitError }, streamError: { status: 200, contentType: 'text/event-stream', body: streamError }, + reasoningStream: { status: 200, contentType: 'text/event-stream', body: reasoningStream }, + structuredOutput: { status: 200, body: structuredOutput }, expected: { - textGenerate: { stopReason: 'stop', inputTokens: 12, outputTokens: 7 }, + textGenerate: { stopReason: 'stop', text: 'Hello, world!', inputTokens: 12, outputTokens: 7 }, toolGenerate: { toolName: 'get_weather', stopReason: 'tool_use' }, textStream: { stopReason: 'stop', inputTokens: 12, outputTokens: 7 }, toolStream: { toolName: 'get_weather', stopReason: 'tool_use' }, streamErrorKind: 'overloaded', + reasoningStream: { text: 'let me think', reasoningTokens: 4 }, + structuredOutput: { text: '{"ok":true}' }, }, }; diff --git a/packages/llm/src/conformance/fixtures/deepseek.ts b/packages/llm/src/conformance/fixtures/deepseek.ts new file mode 100644 index 00000000..b46d75b1 --- /dev/null +++ b/packages/llm/src/conformance/fixtures/deepseek.ts @@ -0,0 +1,186 @@ +import type { ConformanceFixtures } from '../spec.js'; + +/** + * Hand-authored DeepSeek conformance fixtures (1.G) — DeepSeek speaks the OpenAI-compatible wire, so + * these mirror the OpenAI shapes with the `deepseek-chat` model id and DeepSeek's own cache field + * (`prompt_cache_hit_tokens`), which exercises the gross→net usage path distinctly from OpenAI's + * `prompt_tokens_details.cached_tokens`. **Regenerate, don't hand-edit, once a live key records + * fresh ones** (testing.md). + */ + +const textMessage = JSON.stringify({ + id: 'chatcmpl_ds_text', + object: 'chat.completion', + created: 0, + model: 'deepseek-chat', + choices: [ + { + index: 0, + message: { role: 'assistant', content: 'Hello, world!' }, + finish_reason: 'stop', + logprobs: null, + }, + ], + // DeepSeek reports the cache hit at the top level — 4 of the 12 prompt tokens were cached. + usage: { + prompt_tokens: 12, + completion_tokens: 7, + total_tokens: 19, + prompt_cache_hit_tokens: 4, + prompt_cache_miss_tokens: 8, + }, +}); + +const toolMessage = JSON.stringify({ + id: 'chatcmpl_ds_tool', + object: 'chat.completion', + created: 0, + model: 'deepseek-chat', + choices: [ + { + index: 0, + message: { + role: 'assistant', + content: null, + tool_calls: [ + { + id: 'call_ds_weather', + type: 'function', + function: { name: 'get_weather', arguments: '{"city":"Paris"}' }, + }, + ], + }, + finish_reason: 'tool_calls', + logprobs: null, + }, + ], + usage: { prompt_tokens: 20, completion_tokens: 15, total_tokens: 35 }, +}); + +const rateLimitError = JSON.stringify({ + error: { + message: 'Rate limit reached', + type: 'rate_limit_exceeded', + code: 'rate_limit_exceeded', + }, +}); + +function sse(chunks: readonly unknown[]): string { + return chunks.map((chunk) => `data: ${JSON.stringify(chunk)}\n\n`).join('') + 'data: [DONE]\n\n'; +} + +const chunk = ( + choices: readonly unknown[], + usage?: Record, +): Record => ({ + id: 'chatcmpl_ds_stream', + object: 'chat.completion.chunk', + created: 0, + model: 'deepseek-chat', + choices, + ...(usage === undefined ? {} : { usage }), +}); + +const textStream = sse([ + chunk([{ index: 0, delta: { role: 'assistant', content: '' }, finish_reason: null }]), + chunk([{ index: 0, delta: { content: 'Hello, ' }, finish_reason: null }]), + chunk([{ index: 0, delta: { content: 'world!' }, finish_reason: null }]), + chunk([{ index: 0, delta: {}, finish_reason: 'stop' }]), + chunk([], { + prompt_tokens: 12, + completion_tokens: 7, + total_tokens: 19, + prompt_cache_hit_tokens: 4, + }), +]); + +const toolStream = sse([ + chunk([ + { + index: 0, + delta: { + role: 'assistant', + tool_calls: [ + { + index: 0, + id: 'call_ds_weather', + type: 'function', + function: { name: 'get_weather', arguments: '' }, + }, + ], + }, + finish_reason: null, + }, + ]), + chunk([ + { + index: 0, + delta: { tool_calls: [{ index: 0, function: { arguments: '{"city":' } }] }, + finish_reason: null, + }, + ]), + chunk([ + { + index: 0, + delta: { tool_calls: [{ index: 0, function: { arguments: '"Paris"}' } }] }, + finish_reason: null, + }, + ]), + chunk([{ index: 0, delta: {}, finish_reason: 'tool_calls' }]), + chunk([], { prompt_tokens: 20, completion_tokens: 15, total_tokens: 35 }), +]); + +const streamError = JSON.stringify({ + error: { message: 'Service overloaded', type: 'server_error', code: null }, +}); + +// DeepSeek-R1 streams reasoning_content first (content null), then the answer — exercises the +// reasoning channel over the OpenAI-compatible wire (ADR-0030). +const reasoningStream = sse([ + chunk([ + { + index: 0, + delta: { role: 'assistant', reasoning_content: 'let me think' }, + finish_reason: null, + }, + ]), + chunk([{ index: 0, delta: { content: 'Done.' }, finish_reason: 'stop' }]), + chunk([], { prompt_tokens: 6, completion_tokens: 5, total_tokens: 11 }), +]); + +const structuredOutput = JSON.stringify({ + id: 'chatcmpl_ds_json', + object: 'chat.completion', + created: 0, + model: 'deepseek-chat', + choices: [ + { + index: 0, + message: { role: 'assistant', content: '{"ok":true}' }, + finish_reason: 'stop', + logprobs: null, + }, + ], + usage: { prompt_tokens: 8, completion_tokens: 4, total_tokens: 12 }, +}); + +export const DEEPSEEK_FIXTURES: ConformanceFixtures = { + textGenerate: { status: 200, body: textMessage }, + toolGenerate: { status: 200, body: toolMessage }, + textStream: { status: 200, contentType: 'text/event-stream', body: textStream }, + toolStream: { status: 200, contentType: 'text/event-stream', body: toolStream }, + rateLimit: { status: 429, body: rateLimitError }, + streamError: { status: 503, body: streamError }, + reasoningStream: { status: 200, contentType: 'text/event-stream', body: reasoningStream }, + structuredOutput: { status: 200, body: structuredOutput }, + expected: { + // 4 of 12 prompt tokens cached → net input 8, cacheRead 4. + textGenerate: { stopReason: 'stop', text: 'Hello, world!', inputTokens: 8, outputTokens: 7 }, + toolGenerate: { toolName: 'get_weather', stopReason: 'tool_use' }, + textStream: { stopReason: 'stop', inputTokens: 8, outputTokens: 7 }, + toolStream: { toolName: 'get_weather', stopReason: 'tool_use' }, + streamErrorKind: 'overloaded', + reasoningStream: { text: 'let me think' }, + structuredOutput: { text: '{"ok":true}' }, + }, +}; diff --git a/packages/llm/src/conformance/fixtures/gemini.ts b/packages/llm/src/conformance/fixtures/gemini.ts new file mode 100644 index 00000000..b4e3b08c --- /dev/null +++ b/packages/llm/src/conformance/fixtures/gemini.ts @@ -0,0 +1,106 @@ +import type { ConformanceFixtures } from '../spec.js'; + +/** + * Hand-authored Gemini conformance fixtures (1.H). Unlike the Anthropic/OpenAI fixtures (raw HTTP + * bodies the SDK parses), `@google/genai` has no `fetch` hook, so these are recorded at the + * **SDK-output level** — `body` is the JSON of a `GenerateContentResponse` (non-streaming) or a JSON + * array of streamed responses — and the Gemini conformance test injects them through a fake + * `GeminiTransport`. That still exercises the full fold/normalization (the part conformance proves) + * with no vendor import. **Regenerate, don't hand-edit, once a live key records fresh ones.** + */ + +const textResponse = JSON.stringify({ + candidates: [ + { content: { role: 'model', parts: [{ text: 'Hello, world!' }] }, finishReason: 'STOP' }, + ], + usageMetadata: { promptTokenCount: 12, candidatesTokenCount: 7, totalTokenCount: 19 }, +}); + +const toolResponse = JSON.stringify({ + candidates: [ + { + content: { + role: 'model', + parts: [{ functionCall: { name: 'get_weather', args: { city: 'Paris' } } }], + }, + finishReason: 'STOP', // STOP + a tool call normalizes to tool_use + }, + ], + usageMetadata: { promptTokenCount: 20, candidatesTokenCount: 15, totalTokenCount: 35 }, +}); + +const textStream = JSON.stringify([ + { candidates: [{ content: { role: 'model', parts: [{ text: 'Hello, ' }] } }] }, + { + candidates: [{ content: { role: 'model', parts: [{ text: 'world!' }] }, finishReason: 'STOP' }], + usageMetadata: { promptTokenCount: 12, candidatesTokenCount: 7, totalTokenCount: 19 }, + }, +]); + +const toolStream = JSON.stringify([ + { + candidates: [ + { + content: { + role: 'model', + parts: [{ functionCall: { name: 'get_weather', args: { city: 'Paris' } } }], + }, + finishReason: 'STOP', + }, + ], + usageMetadata: { promptTokenCount: 20, candidatesTokenCount: 15, totalTokenCount: 35 }, + }, +]); + +const rateLimitError = JSON.stringify({ + error: { code: 429, message: 'Resource has been exhausted', status: 'RESOURCE_EXHAUSTED' }, +}); + +const overloadedError = JSON.stringify({ + error: { code: 503, message: 'The model is overloaded', status: 'UNAVAILABLE' }, +}); + +// Streamed thought parts (thought: true) then the answer — exercises the reasoning channel (ADR-0030). +const reasoningStream = JSON.stringify([ + { + candidates: [ + { + content: { + role: 'model', + parts: [{ text: 'let me think', thought: true, thoughtSignature: 'sig-g' }], + }, + }, + ], + }, + { + candidates: [{ content: { role: 'model', parts: [{ text: 'Done.' }] }, finishReason: 'STOP' }], + usageMetadata: { promptTokenCount: 6, candidatesTokenCount: 5, thoughtsTokenCount: 2 }, + }, +]); + +const structuredOutput = JSON.stringify({ + candidates: [ + { content: { role: 'model', parts: [{ text: '{"ok":true}' }] }, finishReason: 'STOP' }, + ], + usageMetadata: { promptTokenCount: 8, candidatesTokenCount: 4, totalTokenCount: 12 }, +}); + +export const GEMINI_FIXTURES: ConformanceFixtures = { + textGenerate: { status: 200, body: textResponse }, + toolGenerate: { status: 200, body: toolResponse }, + textStream: { status: 200, body: textStream }, + toolStream: { status: 200, body: toolStream }, + rateLimit: { status: 429, body: rateLimitError }, + streamError: { status: 503, body: overloadedError }, + reasoningStream: { status: 200, body: reasoningStream }, + structuredOutput: { status: 200, body: structuredOutput }, + expected: { + textGenerate: { stopReason: 'stop', text: 'Hello, world!', inputTokens: 12, outputTokens: 7 }, + toolGenerate: { toolName: 'get_weather', stopReason: 'tool_use' }, + textStream: { stopReason: 'stop', inputTokens: 12, outputTokens: 7 }, + toolStream: { toolName: 'get_weather', stopReason: 'tool_use' }, + streamErrorKind: 'overloaded', + reasoningStream: { text: 'let me think', reasoningTokens: 2 }, + structuredOutput: { text: '{"ok":true}' }, + }, +}; diff --git a/packages/llm/src/conformance/fixtures/openai.ts b/packages/llm/src/conformance/fixtures/openai.ts new file mode 100644 index 00000000..64421698 --- /dev/null +++ b/packages/llm/src/conformance/fixtures/openai.ts @@ -0,0 +1,169 @@ +import type { ConformanceFixtures } from '../spec.js'; + +/** + * Hand-authored OpenAI conformance fixtures (1.G) — recorded-shape Chat Completions responses for + * each canonical scenario (JSON for non-streaming, an SSE `data:` transcript for streams). They drive + * the real `openai` SDK parser + our normalization offline. **Regenerate, don't hand-edit, once a + * live key records fresh ones** (testing.md); these seed PR mode until then. + */ + +const textMessage = JSON.stringify({ + id: 'chatcmpl_text', + object: 'chat.completion', + created: 0, + model: 'gpt-4o', + choices: [ + { + index: 0, + message: { role: 'assistant', content: 'Hello, world!', refusal: null }, + finish_reason: 'stop', + logprobs: null, + }, + ], + usage: { + prompt_tokens: 12, + completion_tokens: 7, + total_tokens: 19, + prompt_tokens_details: { cached_tokens: 0 }, + }, +}); + +const toolMessage = JSON.stringify({ + id: 'chatcmpl_tool', + object: 'chat.completion', + created: 0, + model: 'gpt-4o', + choices: [ + { + index: 0, + message: { + role: 'assistant', + content: null, + tool_calls: [ + { + id: 'call_weather', + type: 'function', + function: { name: 'get_weather', arguments: '{"city":"Paris"}' }, + }, + ], + refusal: null, + }, + finish_reason: 'tool_calls', + logprobs: null, + }, + ], + usage: { prompt_tokens: 20, completion_tokens: 15, total_tokens: 35 }, +}); + +const rateLimitError = JSON.stringify({ + error: { + message: 'Rate limit reached for requests', + type: 'rate_limit_exceeded', + code: 'rate_limit_exceeded', + }, +}); + +/** Build an OpenAI SSE transcript from chunk objects: `data: {json}` frames + a `[DONE]` sentinel. */ +function sse(chunks: readonly unknown[]): string { + return chunks.map((chunk) => `data: ${JSON.stringify(chunk)}\n\n`).join('') + 'data: [DONE]\n\n'; +} + +const chunk = ( + choices: readonly unknown[], + usage?: { prompt_tokens: number; completion_tokens: number; total_tokens: number }, +): Record => ({ + id: 'chatcmpl_stream', + object: 'chat.completion.chunk', + created: 0, + model: 'gpt-4o', + choices, + ...(usage === undefined ? {} : { usage }), +}); + +const textStream = sse([ + chunk([{ index: 0, delta: { role: 'assistant', content: '' }, finish_reason: null }]), + chunk([{ index: 0, delta: { content: 'Hello, ' }, finish_reason: null }]), + chunk([{ index: 0, delta: { content: 'world!' }, finish_reason: null }]), + chunk([{ index: 0, delta: {}, finish_reason: 'stop' }]), + // include_usage tail: a final chunk with empty choices carrying the usage. + chunk([], { prompt_tokens: 12, completion_tokens: 7, total_tokens: 19 }), +]); + +const toolStream = sse([ + chunk([ + { + index: 0, + delta: { + role: 'assistant', + tool_calls: [ + { + index: 0, + id: 'call_weather', + type: 'function', + function: { name: 'get_weather', arguments: '' }, + }, + ], + }, + finish_reason: null, + }, + ]), + chunk([ + { + index: 0, + delta: { tool_calls: [{ index: 0, function: { arguments: '{"city":' } }] }, + finish_reason: null, + }, + ]), + chunk([ + { + index: 0, + delta: { tool_calls: [{ index: 0, function: { arguments: '"Paris"}' } }] }, + finish_reason: null, + }, + ]), + chunk([{ index: 0, delta: {}, finish_reason: 'tool_calls' }]), + chunk([], { prompt_tokens: 20, completion_tokens: 15, total_tokens: 35 }), +]); + +// A 503 on the streaming request — the SDK raises before streaming starts; the adapter's init catch +// folds it into a classified `error` StreamChunk (overloaded → retryable). +const streamError = JSON.stringify({ + error: { message: 'The server is overloaded', type: 'server_error', code: null }, +}); + +// A non-streaming reply produced under responseFormat: json — the content is a JSON document. +const structuredOutput = JSON.stringify({ + id: 'chatcmpl_json', + object: 'chat.completion', + created: 0, + model: 'gpt-4o', + choices: [ + { + index: 0, + message: { role: 'assistant', content: '{"ok":true}', refusal: null }, + finish_reason: 'stop', + logprobs: null, + }, + ], + usage: { prompt_tokens: 8, completion_tokens: 4, total_tokens: 12 }, +}); + +// No reasoningStream fixture: OpenAI chat.completions emits no reasoning output (the conformance +// reasoning scenario is skipped for this provider). +export const OPENAI_FIXTURES: ConformanceFixtures = { + textGenerate: { status: 200, body: textMessage }, + toolGenerate: { status: 200, body: toolMessage }, + textStream: { status: 200, contentType: 'text/event-stream', body: textStream }, + toolStream: { status: 200, contentType: 'text/event-stream', body: toolStream }, + rateLimit: { status: 429, body: rateLimitError }, + streamError: { status: 503, body: streamError }, + structuredOutput: { status: 200, body: structuredOutput }, + expected: { + textGenerate: { stopReason: 'stop', text: 'Hello, world!', inputTokens: 12, outputTokens: 7 }, + toolGenerate: { toolName: 'get_weather', stopReason: 'tool_use' }, + textStream: { stopReason: 'stop', inputTokens: 12, outputTokens: 7 }, + toolStream: { toolName: 'get_weather', stopReason: 'tool_use' }, + streamErrorKind: 'overloaded', + structuredOutput: { text: '{"ok":true}' }, + }, +}; diff --git a/packages/llm/src/conformance/gemini.conformance.test.ts b/packages/llm/src/conformance/gemini.conformance.test.ts new file mode 100644 index 00000000..674fa5a1 --- /dev/null +++ b/packages/llm/src/conformance/gemini.conformance.test.ts @@ -0,0 +1,70 @@ +import { describe, expect, it } from 'vitest'; + +import { + createGeminiAdapter, + geminiAdapter, + type GeminiResponse, + type GeminiTransport, +} from '../adapters/gemini.js'; +import { GEMINI_FIXTURES } from './fixtures/gemini.js'; +import { defineConformanceSuite, type MakeReplayAdapter } from './spec.js'; + +async function* toAsyncIterable(items: readonly GeminiResponse[]): AsyncIterable { + await Promise.resolve(); // a streamed transport is async; this fake yields a recorded sequence + for (const item of items) { + yield item; + } +} + +// Validate the parsed fixture with a type guard rather than an unsafe `as` (CLAUDE.md): the fold reads +// every field defensively, so a structural object/array check is sufficient and fails loud on a +// malformed fixture. +const isGeminiResponse = (value: unknown): value is GeminiResponse => + typeof value === 'object' && value !== null && !Array.isArray(value); +const isGeminiResponseArray = (value: unknown): value is GeminiResponse[] => + Array.isArray(value) && value.every(isGeminiResponse); + +// Gemini has no `fetch` hook, so the conformance harness replays at the transport level: a recorded +// SDK-output JSON (single response or an array of streamed responses) is parsed and served through a +// fake GeminiTransport — no vendor SDK is imported here. +const makeReplayAdapter: MakeReplayAdapter = (recorded) => { + const failure = recorded.status >= 400; + const rejection = (): Promise => + Promise.reject(Object.assign(new Error('replayed gemini error'), { status: recorded.status })); + const transport: GeminiTransport = { + generate: () => { + if (failure) return rejection(); + const parsed: unknown = JSON.parse(recorded.body); + return isGeminiResponse(parsed) + ? Promise.resolve(parsed) + : Promise.reject(new Error('replay fixture is not a GeminiResponse object')); + }, + stream: () => { + if (failure) return rejection(); + const parsed: unknown = JSON.parse(recorded.body); + return isGeminiResponseArray(parsed) + ? Promise.resolve(toAsyncIterable(parsed)) + : Promise.reject(new Error('replay fixture is not a GeminiResponse[] array')); + }, + }; + return createGeminiAdapter({ transport }); +}; + +defineConformanceSuite('gemini', makeReplayAdapter, GEMINI_FIXTURES); + +// Live nightly — runs only when GEMINI_API_KEY is set; skipped in PR mode (testing.md). +const liveKey = process.env['GEMINI_API_KEY'] ?? ''; +describe('gemini — conformance (live, nightly)', () => { + it.skipIf(liveKey === '')('generate hits the real API and returns canonical text', async () => { + const result = await geminiAdapter.generate( + { + model: 'gemini-2.0-flash', + maxTokens: 16, + messages: [{ role: 'user', content: [{ type: 'text', text: 'Reply with one word.' }] }], + }, + liveKey, + ); + expect(result.content.some((part) => part.type === 'text')).toBe(true); + expect(result.usage.outputTokens).toBeGreaterThan(0); + }); +}); diff --git a/packages/llm/src/conformance/openai.conformance.test.ts b/packages/llm/src/conformance/openai.conformance.test.ts new file mode 100644 index 00000000..440f4c12 --- /dev/null +++ b/packages/llm/src/conformance/openai.conformance.test.ts @@ -0,0 +1,30 @@ +import { describe, expect, it } from 'vitest'; + +import { createOpenAiAdapter, openaiAdapter } from '../adapters/openai.js'; +import { OPENAI_FIXTURES } from './fixtures/openai.js'; +import { replayFetch } from './replay.js'; +import { defineConformanceSuite, type MakeReplayAdapter } from './spec.js'; + +// Wire the OpenAI adapter to replay a recorded response — no vendor SDK is imported here; the adapter +// takes a `fetch` override, so the SDK stays inside src/adapters/* (the seam fence). +const makeReplayAdapter: MakeReplayAdapter = (recorded) => + createOpenAiAdapter({ providerId: 'openai', fetch: replayFetch(recorded), maxRetries: 0 }); + +defineConformanceSuite('openai', makeReplayAdapter, OPENAI_FIXTURES); + +// Live nightly — runs only when OPENAI_API_KEY is set; skipped in PR mode (testing.md). +const liveKey = process.env['OPENAI_API_KEY'] ?? ''; +describe('openai — conformance (live, nightly)', () => { + it.skipIf(liveKey === '')('generate hits the real API and returns canonical text', async () => { + const result = await openaiAdapter.generate( + { + model: 'gpt-4o-mini', + maxTokens: 16, + messages: [{ role: 'user', content: [{ type: 'text', text: 'Reply with one word.' }] }], + }, + liveKey, + ); + expect(result.content.some((part) => part.type === 'text')).toBe(true); + expect(result.usage.outputTokens).toBeGreaterThan(0); + }); +}); diff --git a/packages/llm/src/conformance/spec.ts b/packages/llm/src/conformance/spec.ts index 493bf237..81e36017 100644 --- a/packages/llm/src/conformance/spec.ts +++ b/packages/llm/src/conformance/spec.ts @@ -3,6 +3,7 @@ import { describe, expect, it } from 'vitest'; import type { StopReason } from '@relavium/shared'; import { LlmProviderError } from '../llm-error.js'; +import { LlmResultSchema, StreamChunkSchema } from '../types.js'; import type { LlmErrorKind, LlmProvider, LlmRequest, StreamChunk } from '../types.js'; import type { RecordedResponse } from './replay.js'; @@ -18,12 +19,22 @@ import type { RecordedResponse } from './replay.js'; /** The canonical values a provider's fixtures should normalize to — asserted concretely. */ export interface ConformanceExpectations { - readonly textGenerate: { stopReason: StopReason; inputTokens: number; outputTokens: number }; + readonly textGenerate: { + stopReason: StopReason; + text: string; + inputTokens: number; + outputTokens: number; + }; readonly toolGenerate: { toolName: string; stopReason: StopReason }; readonly textStream: { stopReason: StopReason; inputTokens: number; outputTokens: number }; readonly toolStream: { toolName: string; stopReason: StopReason }; /** The classified kind a mid-stream `error` event should yield. */ readonly streamErrorKind: LlmErrorKind; + /** Reasoning a thinking model streams (ADR-0030) — only providers that emit reasoning supply this. + * `reasoningTokens` is the count the terminal `stop` chunk must surface (observability; ADR-0030). */ + readonly reasoningStream?: { text: string; reasoningTokens?: number }; + /** The JSON text a model returns under `responseFormat: json` (ADR-0030) — providers that support it. */ + readonly structuredOutput?: { text: string }; } /** The recorded provider responses a conformance run needs — one per canonical scenario. */ @@ -40,6 +51,10 @@ export interface ConformanceFixtures { readonly rateLimit: RecordedResponse; /** A stream that emits a mid-stream `error` event after starting. */ readonly streamError: RecordedResponse; + /** A streamed reply that includes reasoning (ADR-0030) — omit for providers that emit no reasoning. */ + readonly reasoningStream?: RecordedResponse; + /** A non-streaming reply produced under `responseFormat: json` (ADR-0030) — omit if unsupported. */ + readonly structuredOutput?: RecordedResponse; /** The canonical values the above should normalize to. */ readonly expected: ConformanceExpectations; } @@ -70,9 +85,21 @@ const TOOL_REQUEST: LlmRequest = { toolChoice: 'auto', }; +const JSON_REQUEST: LlmRequest = { + model: 'conformance-model', + messages: [{ role: 'user', content: [{ type: 'text', text: 'Return JSON.' }] }], + responseFormat: { + type: 'json', + schema: { type: 'object', properties: { ok: { type: 'boolean' } }, required: ['ok'] }, + }, +}; + async function collect(stream: AsyncIterable): Promise { const chunks: StreamChunk[] = []; for await (const chunk of stream) { + // Defense-in-depth: every streamed chunk must satisfy the canonical StreamChunk schema (throws + // loud on a non-conforming shape, incl. the Usage subset invariant on the terminal stop). + StreamChunkSchema.parse(chunk); chunks.push(chunk); } return chunks; @@ -89,7 +116,10 @@ export function defineConformanceSuite( describe(`${name} — conformance (replay)`, () => { it('generate: returns text content with the exact usage and canonical stop reason', async () => { const result = await makeReplayAdapter(fixtures.textGenerate).generate(TEXT_REQUEST, KEY); - expect(result.content.some((part) => part.type === 'text')).toBe(true); + // Defense-in-depth: the whole result must satisfy the canonical LlmResult schema. + expect(LlmResultSchema.safeParse(result).success).toBe(true); + const text = result.content.map((part) => (part.type === 'text' ? part.text : '')).join(''); + expect(text).toBe(expected.textGenerate.text); // exact value, not just presence expect(result.usage.inputTokens).toBe(expected.textGenerate.inputTokens); expect(result.usage.outputTokens).toBe(expected.textGenerate.outputTokens); expect(result.stopReason).toBe(expected.textGenerate.stopReason); @@ -176,5 +206,56 @@ export function defineConformanceSuite( expect(errorChunk.error.provider).toBe(name); } }); + + it.skipIf(fixtures.reasoningStream === undefined)( + 'reasoning: reasoning_start/delta(s)/end arrive and close before the terminal stop (ADR-0030)', + async () => { + const recorded = fixtures.reasoningStream; + if (recorded === undefined) { + return; // narrow for skipIf + } + const chunks = await collect(makeReplayAdapter(recorded).stream(TEXT_REQUEST, KEY)); + expect(chunks.some((chunk) => chunk.type === 'reasoning_start')).toBe(true); + expect(chunks.some((chunk) => chunk.type === 'reasoning_delta')).toBe(true); + const types = chunks.map((chunk) => chunk.type); + expect(types.lastIndexOf('reasoning_end')).toBeGreaterThanOrEqual(0); + expect(types.lastIndexOf('reasoning_end')).toBeLessThan(types.indexOf('stop')); + if (expected.reasoningStream !== undefined) { + const text = chunks + .map((chunk) => (chunk.type === 'reasoning_delta' ? chunk.text : '')) + .join(''); + expect(text).toBe(expected.reasoningStream.text); + } + // The terminal stop must surface the reasoning-token count (ADR-0030 observability) — this is + // what catches a streaming-usage merge that drops reasoningTokens (e.g. the Anthropic message_delta). + if (expected.reasoningStream?.reasoningTokens !== undefined) { + const stop = chunks.at(-1); + expect(stop?.type).toBe('stop'); + if (stop?.type === 'stop') { + expect(stop.usage.reasoningTokens).toBe(expected.reasoningStream.reasoningTokens); + } + } + }, + ); + + it.skipIf(fixtures.structuredOutput === undefined)( + 'structured output: responseFormat json returns parseable JSON text (ADR-0030)', + async () => { + const recorded = fixtures.structuredOutput; + if (recorded === undefined) { + return; // narrow for skipIf + } + const result = await makeReplayAdapter(recorded).generate(JSON_REQUEST, KEY); + const text = result.content.map((part) => (part.type === 'text' ? part.text : '')).join(''); + expect(() => JSON.parse(text) as unknown).not.toThrow(); + // responseFormat: json must yield text, not a tool call, and a canonical terminal stop reason — + // surfacing any adapter that routes structured output through a forced tool or mis-maps the stop. + expect(result.stopReason).toBe('stop'); + expect(result.content.every((part) => part.type !== 'tool_call')).toBe(true); + if (expected.structuredOutput !== undefined) { + expect(text).toBe(expected.structuredOutput.text); + } + }, + ); }); } diff --git a/packages/llm/src/errors.ts b/packages/llm/src/errors.ts index f92b71ed..fb3bdce2 100644 --- a/packages/llm/src/errors.ts +++ b/packages/llm/src/errors.ts @@ -11,7 +11,8 @@ import type { CapabilityFlags, ProviderId } from './types.js'; export type LlmConfigErrorCode = | 'unknown_model' | 'unsupported_tool_schema' - | 'unsupported_capability'; + | 'unsupported_capability' + | 'invalid_base_url'; /** Base for the seam's thrown config errors — narrow on `code`, never on `message`. */ export abstract class LlmConfigError extends Error { @@ -55,6 +56,22 @@ export class ToolSchemaError extends LlmConfigError { } } +/** + * The factory was given a `baseURL` that is not a safe HTTPS endpoint (e.g. an HTTP URL, a + * loopback/link-local address, or a cloud-metadata service) — the adapter refuses to construct + * rather than silently enabling an SSRF path that forwards the real API key. + */ +export class InvalidBaseUrlError extends LlmConfigError { + readonly code = 'invalid_base_url'; + readonly url: string; + + constructor(url: string, reason: string) { + super(`invalid base URL '${url}': ${reason}`); + this.name = 'InvalidBaseUrlError'; + this.url = url; + } +} + /** * A request needs a capability the chosen provider lacks (e.g. tools on a tools-less provider) — * surfaced rather than silently dropping the feature (1.D). diff --git a/packages/llm/src/index.ts b/packages/llm/src/index.ts index e3430849..6431494d 100644 --- a/packages/llm/src/index.ts +++ b/packages/llm/src/index.ts @@ -14,6 +14,7 @@ export { CapabilityFlagsSchema, LlmErrorKindSchema, LlmErrorSchema, + ResponseFormatSchema, LlmRequestSchema, LlmResultSchema, StreamChunkSchema, @@ -29,6 +30,7 @@ export type { CapabilityFlags, LlmErrorKind, LlmError, + ResponseFormat, LlmRequest, LlmResult, StreamChunk, @@ -46,6 +48,7 @@ export { UnknownModelError, ToolSchemaError, UnsupportedCapabilityError, + InvalidBaseUrlError, } from './errors.js'; export type { LlmConfigErrorCode } from './errors.js'; diff --git a/packages/llm/src/types.test.ts b/packages/llm/src/types.test.ts index ab568399..ba475fb4 100644 --- a/packages/llm/src/types.test.ts +++ b/packages/llm/src/types.test.ts @@ -7,6 +7,7 @@ import { LlmMessageSchema, LlmRequestSchema, LlmResultSchema, + ResponseFormatSchema, StreamChunkSchema, ToolChoiceSchema, ToolDefSchema, @@ -216,3 +217,67 @@ describe('seam types are pure Relavium types (no vendor SDK type crosses the sea expectTypeOf().returns.resolves.toEqualTypeOf(); }); }); + +describe('seam shape amendment (ADR-0030)', () => { + it('ResponseFormatSchema accepts text and json{schema}', () => { + expect(ResponseFormatSchema.safeParse({ type: 'text' }).success).toBe(true); + expect( + ResponseFormatSchema.safeParse({ + type: 'json', + schema: { type: 'object' }, + name: 'out', + strict: true, + }).success, + ).toBe(true); + expect(ResponseFormatSchema.safeParse({ type: 'json', schema: [] }).success).toBe(false); // not an object + }); + + it('LlmRequestSchema accepts an optional responseFormat', () => { + const req = { + model: 'm', + messages: [{ role: 'user', content: [{ type: 'text', text: 'hi' }] }], + responseFormat: { type: 'json', schema: { type: 'object' } }, + }; + expect(LlmRequestSchema.safeParse(req).success).toBe(true); + }); + + it('StreamChunkSchema accepts the reasoning triad and a provider-executed tool_result', () => { + expect(StreamChunkSchema.safeParse({ type: 'reasoning_start', id: 'r0' }).success).toBe(true); + expect( + StreamChunkSchema.safeParse({ type: 'reasoning_delta', id: 'r0', text: 'x' }).success, + ).toBe(true); + expect( + StreamChunkSchema.safeParse({ + type: 'reasoning_end', + id: 'r0', + signature: 's', + redacted: true, + }).success, + ).toBe(true); + expect( + StreamChunkSchema.safeParse({ + type: 'tool_result', + id: 't1', + name: 'web_search', + result: { hits: [] }, + providerExecuted: true, + }).success, + ).toBe(true); + }); + + it('UsageSchema accepts an optional reasoningTokens', () => { + expect( + UsageSchema.safeParse({ inputTokens: 1, outputTokens: 2, reasoningTokens: 1 }).success, + ).toBe(true); + // boundary: equal is allowed + expect( + UsageSchema.safeParse({ inputTokens: 1, outputTokens: 2, reasoningTokens: 2 }).success, + ).toBe(true); + }); + + it('UsageSchema rejects reasoningTokens > outputTokens (ADR-0030 subset invariant)', () => { + expect( + UsageSchema.safeParse({ inputTokens: 1, outputTokens: 2, reasoningTokens: 3 }).success, + ).toBe(false); + }); +}); diff --git a/packages/llm/src/types.ts b/packages/llm/src/types.ts index 10536ab2..81fef6cd 100644 --- a/packages/llm/src/types.ts +++ b/packages/llm/src/types.ts @@ -58,14 +58,45 @@ export const ToolChoiceSchema = z.union([ ]); export type ToolChoice = z.infer; +/** + * How the model should shape its output (ADR-0030). `json` carries one canonical JSON-Schema; each + * adapter lowers it to the provider's **native** structured-output mode (OpenAI `response_format`, + * Gemini `responseJsonSchema`, Anthropic `output_config`/forced tool) — native-vs-forced is the + * adapter's concern. This is the seam mechanism that realizes a node's `output_schema`. + */ +export const ResponseFormatSchema = z.discriminatedUnion('type', [ + z.object({ type: z.literal('text') }), + z.object({ + type: z.literal('json'), + schema: z.custom( + (value) => typeof value === 'object' && value !== null && !Array.isArray(value), + { message: 'responseFormat.schema must be a JSON-Schema object' }, + ), + name: nonEmptyString.optional(), // schema name some providers require (OpenAI); adapters default it + strict: z.boolean().optional(), // strict/exact-schema adherence where the provider supports it + }), +]); +export type ResponseFormat = z.infer; + /** Normalized token usage. `costMicrocents` is Relavium's, computed from the pricing table. */ -export const UsageSchema = z.object({ - inputTokens: nonNegativeInt, - outputTokens: nonNegativeInt, - cacheReadTokens: nonNegativeInt.optional(), - cacheWriteTokens: nonNegativeInt.optional(), - costMicrocents: nonNegativeInt.optional(), -}); +export const UsageSchema = z + .object({ + inputTokens: nonNegativeInt, + outputTokens: nonNegativeInt, + cacheReadTokens: nonNegativeInt.optional(), + cacheWriteTokens: nonNegativeInt.optional(), + // Reasoning ("thinking") tokens — OBSERVABILITY only (ADR-0030). Already counted inside + // `outputTokens` for billing, so the CostTracker bills `outputTokens` whole; this is not a new + // cost class, just visibility into how much of the output was reasoning. + reasoningTokens: nonNegativeInt.optional(), + costMicrocents: nonNegativeInt.optional(), + }) + // Enforce the ADR-0030 invariant: reasoning is a SUBSET of output, never larger. Catches an adapter + // that mis-maps the reasoning count (the seam's contract, not a billing input). + .refine((u) => u.reasoningTokens === undefined || u.reasoningTokens <= u.outputTokens, { + message: 'reasoningTokens must be ≤ outputTokens (reasoning is counted inside output)', + path: ['reasoningTokens'], + }); export type Usage = z.infer; /** What a provider supports; features off the common path are reached via `providerOptions`. */ @@ -116,6 +147,7 @@ export const LlmRequestSchema = z.object({ messages: z.array(LlmMessageSchema), tools: z.array(ToolDefSchema).optional(), toolChoice: ToolChoiceSchema.optional(), + responseFormat: ResponseFormatSchema.optional(), // structured-output request (ADR-0030) temperature: z.number().optional(), maxTokens: z.number().int().positive().optional(), // required downstream for Anthropic — adapters default it stopSequences: z.array(z.string()).optional(), @@ -157,6 +189,26 @@ export const StreamChunkSchema = z.discriminatedUnion('type', [ z.object({ type: z.literal('tool_call_start'), id: nonEmptyString, name: nonEmptyString }), z.object({ type: z.literal('tool_call_delta'), id: nonEmptyString, argsJsonDelta: z.string() }), z.object({ type: z.literal('tool_call_end'), id: nonEmptyString }), + // Reasoning channel (ADR-0030) — mirrors the tool_call_* triad; `id` correlates the deltas to the + // terminating reasoning_end, which carries the optional ephemeral provider signature. + z.object({ type: z.literal('reasoning_start'), id: nonEmptyString }), + z.object({ type: z.literal('reasoning_delta'), id: nonEmptyString, text: z.string() }), + z.object({ + type: z.literal('reasoning_end'), + id: nonEmptyString, + signature: z.string().optional(), + redacted: z.boolean().optional(), + }), + // A provider-executed (server-side) tool result carried inline (ADR-0030) — distinct from the + // engine-executed tool_call_* triad. Reserved shape; the engine dispatcher records it, never runs it. + z.object({ + type: z.literal('tool_result'), + id: nonEmptyString, + name: nonEmptyString, + result: z.unknown(), + isError: z.boolean().optional(), + providerExecuted: z.literal(true), + }), z.object({ type: z.literal('stop'), stopReason: StopReasonSchema, usage: UsageSchema }), z.object({ type: z.literal('error'), error: LlmErrorSchema }), ]); diff --git a/packages/shared/src/content.test.ts b/packages/shared/src/content.test.ts index cc194eaa..cad31f8c 100644 --- a/packages/shared/src/content.test.ts +++ b/packages/shared/src/content.test.ts @@ -60,3 +60,33 @@ describe('AbortSignalLike', () => { expect(fired).toBe(true); }); }); + +describe('ContentPart amendment (ADR-0030)', () => { + it('accepts a reasoning part with an optional signature/redacted', () => { + expect(ContentPartSchema.safeParse({ type: 'reasoning', text: 'thinking' }).success).toBe(true); + expect( + ContentPartSchema.safeParse({ type: 'reasoning', text: 't', signature: 's', redacted: true }) + .success, + ).toBe(true); + }); + + it('accepts providerExecuted on tool_call and tool_result', () => { + expect( + ContentPartSchema.safeParse({ + type: 'tool_call', + id: 'c1', + name: 'f', + args: {}, + providerExecuted: true, + }).success, + ).toBe(true); + expect( + ContentPartSchema.safeParse({ + type: 'tool_result', + toolCallId: 'c1', + result: {}, + providerExecuted: true, + }).success, + ).toBe(true); + }); +}); diff --git a/packages/shared/src/content.ts b/packages/shared/src/content.ts index 866cba18..a08dab27 100644 --- a/packages/shared/src/content.ts +++ b/packages/shared/src/content.ts @@ -23,12 +23,28 @@ export const ContentPartSchema = z.discriminatedUnion('type', [ id: nonEmptyString, name: nonEmptyString, args: z.unknown(), + // The provider ran this tool on its own side (server-side / built-in tool, e.g. web search). The + // engine's ToolDispatcher does NOT execute it and does not apply its allowlist to it — it only + // records/forwards. Omitted (or false) means an engine-executed call. See ADR-0030 / ADR-0029. + providerExecuted: z.boolean().optional(), }), z.object({ type: z.literal('tool_result'), toolCallId: nonEmptyString, result: z.unknown(), isError: z.boolean().optional(), + // The provider produced this result (the counterpart of a provider-executed tool_call). ADR-0030. + providerExecuted: z.boolean().optional(), + }), + // Reasoning / "thinking" content (ADR-0030). EPHEMERAL: `signature` is a same-provider, same-turn + // continuity token — it is never persisted to a session, never replayed across a provider boundary + // on fallback, and never written to a run event or log. The engine does not interpret it; only the + // originating adapter feeds it back. `redacted` marks a provider-withheld block (data, no text). + z.object({ + type: z.literal('reasoning'), + text: z.string(), + signature: z.string().optional(), + redacted: z.boolean().optional(), }), ]); export type ContentPart = z.infer; diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 6e90bdca..348584e2 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -12,6 +12,9 @@ catalogs: '@eslint/js': specifier: ^9.17.0 version: 9.39.4 + '@google/genai': + specifier: ^2.8.0 + version: 2.8.0 '@types/better-sqlite3': specifier: ^7.6.13 version: 7.6.13 @@ -39,6 +42,9 @@ catalogs: eslint-config-prettier: specifier: ^9.1.0 version: 9.1.2 + openai: + specifier: ^6.42.0 + version: 6.42.0 prettier: specifier: ^3.4.2 version: 3.8.3 @@ -126,9 +132,15 @@ importers: '@anthropic-ai/sdk': specifier: 'catalog:' version: 0.101.0(zod@3.25.76) + '@google/genai': + specifier: 'catalog:' + version: 2.8.0 '@relavium/shared': specifier: workspace:* version: link:../shared + openai: + specifier: 'catalog:' + version: 6.42.0(ws@8.21.0)(zod@3.25.76) zod: specifier: 'catalog:' version: 3.25.76 @@ -854,6 +866,15 @@ packages: resolution: {integrity: sha512-43/qtrDUokr7LJqoF2c3+RInu/t4zfrpYdoSDfYyhg52rwLV6TnOvdG4fXm7IkSB3wErkcmJS9iEhjVtOSEjjA==} engines: {node: ^18.18.0 || ^20.9.0 || >=21.1.0} + '@google/genai@2.8.0': + resolution: {integrity: sha512-pc2ayxqO5+O7AvnHBqpNHIk7PAZkHZgL31tbyx0gJZBSS9qPYiQoqwK7oYOw/ePmG6QY4EMSu+304vD5QlhXAw==} + engines: {node: '>=20.0.0'} + peerDependencies: + '@modelcontextprotocol/sdk': ^1.25.2 + peerDependenciesMeta: + '@modelcontextprotocol/sdk': + optional: true + '@humanfs/core@0.19.2': resolution: {integrity: sha512-UhXNm+CFMWcbChXywFwkmhqjs3PRCmcSa/hfBgLIb7oQ5HNb1wS0icWsGtSAUNgefHeI+eBrA8I1fxmbHsGdvA==} engines: {node: '>=18.18.0'} @@ -899,6 +920,36 @@ packages: resolution: {integrity: sha512-+1VkjdD0QBLPodGrJUeqarH8VAIvQODIbwh9XpP5Syisf7YoQgsJKPNFoqqLQlu+VQ/tVSshMR6loPMn8U+dPg==} engines: {node: '>=14'} + '@protobufjs/aspromise@1.1.2': + resolution: {integrity: sha512-j+gKExEuLmKwvz3OgROXtrJ2UG2x8Ch2YZUxahh+s1F2HZ+wAceUNLkvy6zKCPVRkU++ZWQrdxsUeQXmcg4uoQ==} + + '@protobufjs/base64@1.1.2': + resolution: {integrity: sha512-AZkcAA5vnN/v4PDqKyMR5lx7hZttPDgClv83E//FMNhR2TMcLUhfRUBHCmSl0oi9zMgDDqRUJkSxO3wm85+XLg==} + + '@protobufjs/codegen@2.0.5': + resolution: {integrity: sha512-zgXFLzW3Ap33e6d0Wlj4MGIm6Ce8O89n/apUaGNB/jx+hw+ruWEp7EwGUshdLKVRCxZW12fp9r40E1mQrf/34g==} + + '@protobufjs/eventemitter@1.1.1': + resolution: {integrity: sha512-vW1GmwMZNnL+gMRaovlh9yZX74kc+TTU3FObkkurpMaRtBfLP3ldjS9KQWlwZgraRE0+dheEEoAxdzcJQ8eXZg==} + + '@protobufjs/fetch@1.1.1': + resolution: {integrity: sha512-GpptLrs57adMSuHi3VNj0mAF8dwh36LMaYF6XyJ6JMWlVsc+t42tm1HSEDmOs3A8fC9yyeisgLhsTVQokOZ0zw==} + + '@protobufjs/float@1.0.2': + resolution: {integrity: sha512-Ddb+kVXlXst9d+R9PfTIxh1EdNkgoRe5tOX6t01f1lYWOvJnSPDBlG241QLzcyPdoNTsblLUdujGSE4RzrTZGQ==} + + '@protobufjs/inquire@1.1.2': + resolution: {integrity: sha512-pa0vFRuws4wkvaXKK1uXZMAwAX4/t8ANaJo45iw/oQHNQ9q5xUzwgFmVJGXiga2BeN+zpX7Vf9vmsiIa2J+MUw==} + + '@protobufjs/path@1.1.2': + resolution: {integrity: sha512-6JOcJ5Tm08dOHAbdR3GrvP+yUUfkjG5ePsHYczMFLq3ZmMkAD98cDgcT2iA1lJ9NVwFd4tH/iSSoe44YWkltEA==} + + '@protobufjs/pool@1.1.0': + resolution: {integrity: sha512-0kELaGSIDBKvcgS4zkjz1PeddatrjYcmMWOlAuAPwAeccUrPHdUqo/J6LiymHHEiJT5NrF1UVwxY14f+fy4WQw==} + + '@protobufjs/utf8@1.1.1': + resolution: {integrity: sha512-oOAWABowe8EAbMyWKM0tYDKi8Yaox52D+HWZhAIJqQXbqe0xI/GV7FhLWqlEKreMkfDjshR5FKgi3mnle0h6Eg==} + '@rollup/rollup-android-arm-eabi@4.61.1': resolution: {integrity: sha512-JnBB8MdXj45cajvTuO5FmPlvFVJRQgvrz1uSEl3NwqFnReAPGwb8EanbGi4z2nRaqLzjJSv5/JmycoTKlRZxHA==} cpu: [arm] @@ -1078,6 +1129,9 @@ packages: '@types/node@22.19.19': resolution: {integrity: sha512-dyh/xO2Fh5bYrfWaaqGrRQQGkNdmYw6AmaAUvYeUMNTWQtvb796ikLdmTchRmOlOiIJ1TDXfWgVx1QkUlQ6Hew==} + '@types/retry@0.12.0': + resolution: {integrity: sha512-wWKOClTTiizcZhXnPY4wikVAwmdYHp8q6DmC+EJUzAMsycb7HB32Kh9RN4+0gExjmPmZSAQjgURXIGATPegAvA==} + '@typescript-eslint/eslint-plugin@8.60.1': resolution: {integrity: sha512-JQ4S5GB0tfjO8BuJ4fcX+HodkzJjYBV+7OJ+wLygaX7OGQ7FudyHL4NSCA6ob+w3Yn+5MkKIozOwQhXeM7opVg==} engines: {node: ^18.18.0 || ^20.9.0 || >=21.1.0} @@ -1185,6 +1239,10 @@ packages: engines: {node: '>=0.4.0'} hasBin: true + agent-base@7.1.4: + resolution: {integrity: sha512-MnA+YT8fwfJPgBx3m60MNqakm30XOkyIoH1y6huTQvC0PwZG7ki8NacLBcrPbNoo8vEZy7Jpuk7+jMO+CUovTQ==} + engines: {node: '>= 14'} + ajv@6.15.0: resolution: {integrity: sha512-fgFx7Hfoq60ytK2c7DhnF8jIvzYgOMxfugjLOSMHjLIPgenqa7S7oaagATUq99mV6IYvN2tRmC0wnTYX6iPbMw==} @@ -1228,6 +1286,9 @@ packages: resolution: {integrity: sha512-CyzaZRQKyHkB2ZInfTTl2nvT33EbDpjkLEbE8/Zck3Ll6O0qqvuGdrJ45HgtH+HykRg88ITY3AdreBGN70aBSQ==} engines: {node: 20.x || 22.x || 23.x || 24.x || 25.x || 26.x} + bignumber.js@9.3.1: + resolution: {integrity: sha512-Ko0uX15oIUS7wJ3Rb30Fs6SkVbLmPBAKdlm7q9+ak9bbIeFf0MwuBsQV6z7+X768/cHsfg+WlysDWJcmthjsjQ==} + bindings@1.5.0: resolution: {integrity: sha512-p2q/t/mhvuOj/UeLlV6566GD/guowlr0hHxClI0W9m7MWYkL1F0hLo+0Aexs9HSPCtR1SXQ0TD3MMKrXZajbiQ==} @@ -1244,6 +1305,9 @@ packages: resolution: {integrity: sha512-kLpxurY4Z4r9sgMsyG0Z9uzsBlgiU/EFKhj/h91/8yHu0edo7XuixOIH3VcJ8kkxs6/jPzoI6U9Vj3WqbMQ94g==} engines: {node: 18 || 20 || >=22} + buffer-equal-constant-time@1.0.1: + resolution: {integrity: sha512-zRpUiDwd/xk6ADqPMATG8vc9VPrkck7T07OIx0gnjmJAnHnTVXNQG3vfvWNuiZIkwu9KrKdA1iJKfsfTVxE6NA==} + buffer-from@1.1.2: resolution: {integrity: sha512-E+XQCRwSbaaiChtv6k6Dwgc+bx+Bs6vuKJHHl5kox/BaKbhiXzqQOwK4cO22yElGp2OCmjwVhT3HmxgyPGnJfQ==} @@ -1287,6 +1351,10 @@ packages: resolution: {integrity: sha512-uV2QOWP2nWzsy2aMp8aRibhi9dlzF5Hgh5SHaB9OiTGEyDTiJJyx0uy51QXdyWbtAHNua4XJzUKca3OzKUd3vA==} engines: {node: '>= 8'} + data-uri-to-buffer@4.0.1: + resolution: {integrity: sha512-0R9ikRb668HB7QDxT1vkpuUBtqc53YyAwMwGeUFKRojY/NWKvdZ+9UYtRfGmhqNbRkTSVpMbmyhXipFFv2cb/A==} + engines: {node: '>= 12'} + debug@4.4.3: resolution: {integrity: sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==} engines: {node: '>=6.0'} @@ -1414,6 +1482,9 @@ packages: eastasianwidth@0.2.0: resolution: {integrity: sha512-I88TYZWc9XiYHRQ4/3c5rjjfgkjhLyW2luGIheGERbNQ6OY7yTybanSpDXZa8y7VUP9YmDcYa+eyq4ca7iLqWA==} + ecdsa-sig-formatter@1.0.11: + resolution: {integrity: sha512-nagl3RYrbNv6kQkeJIpt6NJZy8twLB/2vtz6yN9Z4vRKHN4/QZJIEbqohALSgwKdnksuY3k5Addp5lg8sVoVcQ==} + emoji-regex@8.0.0: resolution: {integrity: sha512-MSjYzcWNOA0ewAHpz0MxpYFvwg6yjy1NG3xteoqz644VCo/RPgnr1/GGt+ic3iJTzQ8Eu3TdM14SawnVUmGE6A==} @@ -1513,6 +1584,9 @@ packages: resolution: {integrity: sha512-knvyeauYhqjOYvQ66MznSMs83wmHrCycNEN6Ao+2AeYEfxUIkuiVxdEa1qlGEPK+We3n0THiDciYSsCcgW/DoA==} engines: {node: '>=12.0.0'} + extend@3.0.2: + resolution: {integrity: sha512-fjquC59cD7CyW6urNXK0FBufkZcoiGG80wTuPujX590cB5Ttln20E2UB4S/WARVqhXffZl2LNgS+gQdPIIim/g==} + fast-deep-equal@3.1.3: resolution: {integrity: sha512-f3qQ9oQy9j2AhBe/H9VC91wLmKBCCU/gDOnKNAYG5hswO7BLKj09Hc5HYNz9cGI++xlpDCIgDaitVs03ATR84Q==} @@ -1534,6 +1608,10 @@ packages: picomatch: optional: true + fetch-blob@3.2.0: + resolution: {integrity: sha512-7yAQpD2UMJzLi1Dqv7qFYnPbaPx7ZfFK6PiIxQ4PfkGPyNyl2Ugx+a/umUonmKqjhM4DnfbMvdX6otXq83soQQ==} + engines: {node: ^12.20 || >= 14.13} + file-entry-cache@8.0.0: resolution: {integrity: sha512-XXTUwCvisa5oacNGRP9SfNtYBNAMi+RPwBFmblZEF7N7swHYQS6/Zfk7SRwx4D5j3CH211YNRco1DEMNVfZCnQ==} engines: {node: '>=16.0.0'} @@ -1556,6 +1634,10 @@ packages: resolution: {integrity: sha512-gIXjKqtFuWEgzFRJA9WCQeSJLZDjgJUOMCMzxtvFq/37KojM1BFGufqsCy0r4qSQmYLsZYMeyRqzIWOMup03sw==} engines: {node: '>=14'} + formdata-polyfill@4.0.10: + resolution: {integrity: sha512-buewHzMvYL29jdeQTVILecSaZKnt/RJWjoZCF5OW60Z67/GmSLBkOFM7qh1PI3zFNtJbaZL5eQu1vLfazOwj4g==} + engines: {node: '>=12.20.0'} + fs-constants@1.0.0: resolution: {integrity: sha512-y6OAwoSIf7FyjMIv94u+b5rdheZEjzR63GTyZJm5qh4Bi+2YgwLCcI/fPFZkL5PSixOt6ZNKm+w+Hfp/Bciwow==} @@ -1564,6 +1646,14 @@ packages: engines: {node: ^8.16.0 || ^10.6.0 || >=11.0.0} os: [darwin] + gaxios@7.1.5: + resolution: {integrity: sha512-5FZy72Rh8LhtjmvDrKkI+lVhrsQrVKVsItxMoDm5mNQE+xR0WVIIs+jzPSJgBvKVsLi24fZhXJIsNI0bihDzFg==} + engines: {node: '>=18'} + + gcp-metadata@8.1.2: + resolution: {integrity: sha512-zV/5HKTfCeKWnxG0Dmrw51hEWFGfcF2xiXqcA3+J90WDuP0SvoiSO5ORvcBsifmx/FoIjgQN3oNOGaQ5PhLFkg==} + engines: {node: '>=18'} + get-tsconfig@4.14.0: resolution: {integrity: sha512-yTb+8DXzDREzgvYmh6s9vHsSVCHeC0G3PI5bEXNBHtmshPnO+S5O7qgLEOn0I5QvMy6kpZN8K1NKGyilLb93wA==} @@ -1583,6 +1673,14 @@ packages: resolution: {integrity: sha512-oahGvuMGQlPw/ivIYBjVSrWAfWLBeku5tpPE2fOPLi+WHffIWbuh2tCjhyQhTBPMf5E9jDEH4FOmTYgYwbKwtQ==} engines: {node: '>=18'} + google-auth-library@10.7.0: + resolution: {integrity: sha512-QpTAbNJ36TliZLx3TTtahR8HG0hN9RllL1e3FymOvQSIKK8JmgV58H924ub2wa2DsS3ANjjP1Aw1N+Ramc8hqQ==} + engines: {node: '>=18'} + + google-logging-utils@1.1.3: + resolution: {integrity: sha512-eAmLkjDjAFCVXg7A1unxHsLf961m6y17QFqXqAXGj/gVkKFrEICfStRfwUlGNfeCEjNRa32JEWOUTlYXPyyKvA==} + engines: {node: '>=14'} + has-flag@4.0.0: resolution: {integrity: sha512-EykJT/Q1KjTWctppgIAgfSO0tKVuZUjhgMr17kqTumMl6Afv3EISleU7qZUzoXDFTAHTDC4NOoG/ZxU3EvlMPQ==} engines: {node: '>=8'} @@ -1590,6 +1688,10 @@ packages: html-escaper@2.0.2: resolution: {integrity: sha512-H2iMtd0I4Mt5eYiapRdIDjp+XzelXQ0tFE4JS7YFwFevXXMmOp9myNrUvCg0D6ws8iqkRPBfKHgbwig1SmlLfg==} + https-proxy-agent@7.0.6: + resolution: {integrity: sha512-vK9P5/iUfdl95AI+JVyUuIcVtd4ofvtrOr3HNtM2yxC9bnMbEdp3x01OhQNnjb8IJYi38VlTE3mBXwcfvywuSw==} + engines: {node: '>= 14'} + ieee754@1.2.1: resolution: {integrity: sha512-dcyqhDvX1C46lXZcVqCpK+FtMRQVdIMN6/Df5js2zouUsqG7I6sFxitIC+7KYK29KdXOLHdu9zL4sFnoVQnqaA==} @@ -1659,6 +1761,9 @@ packages: resolution: {integrity: sha512-ePWsvanv0DWuDRsW8dnt+R4jQ31SCRCQ7hhNcPXZPsoBZiemuZNYGf7adZdqX2D86j6rvKp3RpCxVTSb8WQlOw==} hasBin: true + json-bigint@1.0.0: + resolution: {integrity: sha512-SiPv/8VpZuWbvLSMtTDU8hEfrZWg/mH/nV/b4o0CYbSxu1UIQPLdwKOCIyLQX+VIPO5vrLX3i8qtqFyhdPSUSQ==} + json-buffer@3.0.1: resolution: {integrity: sha512-4bV5BfR2mqfQTJm+V5tPPdf+ZpuhiIvTuAB5g8kcrXOZpTT/QwwVRWBywX1ozr6lEuPdbHxwaJlm9G6mI2sfSQ==} @@ -1672,6 +1777,12 @@ packages: json-stable-stringify-without-jsonify@1.0.1: resolution: {integrity: sha512-Bdboy+l7tA3OGW6FjyFHWkP5LuByj1Tk33Ljyq0axyzdk9//JSi2u3fP1QSmd1KNwq6VOKYGlAu87CisVir6Pw==} + jwa@2.0.1: + resolution: {integrity: sha512-hRF04fqJIP8Abbkq5NKGN0Bbr3JxlQ+qhZufXVr0DvujKy93ZCbXZMHDL4EOtodSbCWxOqR8MS1tXA5hwqCXDg==} + + jws@4.0.1: + resolution: {integrity: sha512-EKI/M/yqPncGUUh44xz0PxSidXFr/+r0pA70+gIYhjv+et7yxM+s29Y+VGDkovRofQem0fs7Uvf4+YmAdyRduA==} + keyv@4.5.4: resolution: {integrity: sha512-oxVHkHR/EJf2CNXnWxRLW6mg7JyCCUcG0DtEGmL2ctUo1PNTin1PUil+r/+4r5MpVgC/fn1kjsx7mjSujKqIpw==} @@ -1686,6 +1797,9 @@ packages: lodash.merge@4.6.2: resolution: {integrity: sha512-0KpjqXRVvrYyCsX1swR/XTK0va6VQkQM6MNo7PqW77ByjAhoARA8EfrP1N4+KlKj8YS0ZUCtRT/YUuhyYDujIQ==} + long@5.3.2: + resolution: {integrity: sha512-mNAgZ1GmyNhD7AuqnTG3/VQ26o760+ZYBPKjPvugO8+nLbYfX6TVpJPseBvopbdY+qpZ/lKUnmEc1LeZYS3QAA==} + loupe@3.2.1: resolution: {integrity: sha512-CdzqowRJCeLU72bHvWqwRBBlLcMEtIvGrlvef74kMnV2AolS9Y8xUv1I0U/MNAWMhBlKIoyuEgoJ0t/bbwHbLQ==} @@ -1745,9 +1859,29 @@ packages: resolution: {integrity: sha512-KdHvFWZjEKDf0cakgFjebl371GPsISX2oZHcuyKqM7DtogIsHrqKeLTo8wBHxaXRAQlY2PsPlZmfo+9ZCxEREQ==} engines: {node: '>=10'} + node-domexception@1.0.0: + resolution: {integrity: sha512-/jKZoMpw0F8GRwl4/eLROPA3cfcXtLApP0QzLmUT/HuPCZWyB7IY9ZrMeKw2O/nFIqPQB3PVM9aYm0F312AXDQ==} + engines: {node: '>=10.5.0'} + deprecated: Use your platform's native DOMException instead + + node-fetch@3.3.2: + resolution: {integrity: sha512-dRB78srN/l6gqWulah9SrxeYnxeddIG30+GOqK/9OlLVyLg3HPnr6SqOWTWOXKRwC2eGYCkZ59NNuSgvSrpgOA==} + engines: {node: ^12.20.0 || ^14.13.1 || >=16.0.0} + once@1.4.0: resolution: {integrity: sha512-lNaJgI+2Q5URQBkccEKHTQOPaXdUxnZZElQTZY0MFUAuaEqe1E+Nyvgdz/aIyNi6Z9MzO5dv1H8n58/GELp3+w==} + openai@6.42.0: + resolution: {integrity: sha512-1WFEt/uXMXOLhYRNkgJWo08Y2YNvNwpVU72K7ibrWgWpNOXd4VojXLbe6SQ4bLiUQ3Y8jz4IiyVkylJCL1DtZg==} + peerDependencies: + ws: ^8.18.0 + zod: ^3.25 || ^4.0 + peerDependenciesMeta: + ws: + optional: true + zod: + optional: true + optionator@0.9.4: resolution: {integrity: sha512-6IpQ7mKUxRcZNLIObR0hz7lxsapSSIYNZJwXPGeF0mTVqGKFIXj1DQcMoT22S3ROcLyY/rz0PWaWZ9ayWmad9g==} engines: {node: '>= 0.8.0'} @@ -1760,6 +1894,10 @@ packages: resolution: {integrity: sha512-LaNjtRWUBY++zB5nE/NwcaoMylSPk+S+ZHNB1TzdbMJMny6dynpAGt7X/tl/QYq3TIeE6nxHppbo2LGymrG5Pw==} engines: {node: '>=10'} + p-retry@4.6.2: + resolution: {integrity: sha512-312Id396EbJdvRONlngUx0NydfrIQ5lsYu0znKVUzVvArzEIt08V1qhtyESbGVd1FGX7UKtiFp5uwKZdM8wIuQ==} + engines: {node: '>=8'} + package-json-from-dist@1.0.1: resolution: {integrity: sha512-UEZIS3/by4OC8vL3P2dTXRETpebLI2NiI5vIrjaD/5UtrkFX/tNbwjTSRAGC/+7CAo2pIcBaRgWmcBBHcsaCIw==} @@ -1812,6 +1950,10 @@ packages: engines: {node: '>=14'} hasBin: true + protobufjs@7.6.2: + resolution: {integrity: sha512-N9EiLovGEQOJSPF26Ij7qUGvahfEnq0eeYZ02aigIedkmz1qZSwjnP9SBITHJuF/6MYbIW4HDN8zdYjsjqJKXQ==} + engines: {node: '>=12.0.0'} + pump@3.0.4: resolution: {integrity: sha512-VS7sjc6KR7e1ukRFhQSY5LM2uBWAUPiOPa/A3mkKmiMwSmRFUITt0xuj+/lesgnCv+dPIEYlkzrcyXgquIHMcA==} @@ -1834,6 +1976,10 @@ packages: resolve-pkg-maps@1.0.0: resolution: {integrity: sha512-seS2Tj26TBVOC2NIc2rOe2y2ZO7efxITtLZcGSOnHHNOQ7CkiUBfw0Iw2ck6xkIhPwLhKNLS8BO+hEpngQlqzw==} + retry@0.13.1: + resolution: {integrity: sha512-XQBQ3I8W1Cge0Seh+6gjj03LbmRFWuoszgK9ooCpwYIrhhoO80pfq4cUkU5DkknwfOfFteRwlZ56PYOGYyFWdg==} + engines: {node: '>= 4'} + rollup@4.61.1: resolution: {integrity: sha512-I4KW6iuRpuu2uHBLraZ1wNZe0DP7lnRha+VJ9tNaYVaVgKhW0aI3h4RYnoRPeql0flHm/Co55b7snEDcOfOJrA==} engines: {node: '>=18.0.0', npm: '>=8.0.0'} @@ -2074,6 +2220,10 @@ packages: jsdom: optional: true + web-streams-polyfill@3.3.3: + resolution: {integrity: sha512-d2JWLCivmZYTSIoge9MsgFCZrt571BikcWGYkjC1khllbTeDlGqZ2D8vD8E/lJa8WGWbb7Plm8/XJYV7IJHZZw==} + engines: {node: '>= 8'} + which@2.0.2: resolution: {integrity: sha512-BLI3Tl1TW3Pvl70l3yq3Y64i+awpwXqsGBYWkkqMtnbXgrMD+yj7rhW0kuEDxzJaYXGjEW5ogapKNMEKNMjibA==} engines: {node: '>= 8'} @@ -2099,6 +2249,18 @@ packages: wrappy@1.0.2: resolution: {integrity: sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==} + ws@8.21.0: + resolution: {integrity: sha512-Vsp28b7DRcimFQvrqu2Wek3z1iYxDCWqHYB8Qsnk/S4RfaCQzPGPyBNuVjJV3cd6UiKtUtp6sNM77gWvzcCH+g==} + engines: {node: '>=10.0.0'} + peerDependencies: + bufferutil: ^4.0.1 + utf-8-validate: '>=5.0.2' + peerDependenciesMeta: + bufferutil: + optional: true + utf-8-validate: + optional: true + yocto-queue@0.1.0: resolution: {integrity: sha512-rVksvsnNCdJ/ohGc6xgPwyN8eheCxsiLM8mxuE/t/mOVqJewPuO1miLpTHQiRgTKCLexL4MeAFVagts7HmNZ2Q==} engines: {node: '>=10'} @@ -2495,6 +2657,17 @@ snapshots: '@eslint/core': 0.17.0 levn: 0.4.1 + '@google/genai@2.8.0': + dependencies: + google-auth-library: 10.7.0 + p-retry: 4.6.2 + protobufjs: 7.6.2 + ws: 8.21.0 + transitivePeerDependencies: + - bufferutil + - supports-color + - utf-8-validate + '@humanfs/core@0.19.2': dependencies: '@humanfs/types': 0.15.0 @@ -2539,6 +2712,28 @@ snapshots: '@pkgjs/parseargs@0.11.0': optional: true + '@protobufjs/aspromise@1.1.2': {} + + '@protobufjs/base64@1.1.2': {} + + '@protobufjs/codegen@2.0.5': {} + + '@protobufjs/eventemitter@1.1.1': {} + + '@protobufjs/fetch@1.1.1': + dependencies: + '@protobufjs/aspromise': 1.1.2 + + '@protobufjs/float@1.0.2': {} + + '@protobufjs/inquire@1.1.2': {} + + '@protobufjs/path@1.1.2': {} + + '@protobufjs/pool@1.1.0': {} + + '@protobufjs/utf8@1.1.1': {} + '@rollup/rollup-android-arm-eabi@4.61.1': optional: true @@ -2656,7 +2851,8 @@ snapshots: '@types/node@22.19.19': dependencies: undici-types: 6.21.0 - optional: true + + '@types/retry@0.12.0': {} '@typescript-eslint/eslint-plugin@8.60.1(@typescript-eslint/parser@8.60.1(eslint@9.39.4)(typescript@5.9.3))(eslint@9.39.4)(typescript@5.9.3)': dependencies: @@ -2824,6 +3020,8 @@ snapshots: acorn@8.16.0: {} + agent-base@7.1.4: {} + ajv@6.15.0: dependencies: fast-deep-equal: 3.1.3 @@ -2862,6 +3060,8 @@ snapshots: bindings: 1.5.0 prebuild-install: 7.1.3 + bignumber.js@9.3.1: {} + bindings@1.5.0: dependencies: file-uri-to-path: 1.0.0 @@ -2885,6 +3085,8 @@ snapshots: dependencies: balanced-match: 4.0.4 + buffer-equal-constant-time@1.0.1: {} + buffer-from@1.1.2: {} buffer@5.7.1: @@ -2927,6 +3129,8 @@ snapshots: shebang-command: 2.0.0 which: 2.0.2 + data-uri-to-buffer@4.0.1: {} + debug@4.4.3: dependencies: ms: 2.1.3 @@ -2957,6 +3161,10 @@ snapshots: eastasianwidth@0.2.0: {} + ecdsa-sig-formatter@1.0.11: + dependencies: + safe-buffer: 5.2.1 + emoji-regex@8.0.0: {} emoji-regex@9.2.2: {} @@ -3161,6 +3369,8 @@ snapshots: expect-type@1.3.0: {} + extend@3.0.2: {} + fast-deep-equal@3.1.3: {} fast-json-stable-stringify@2.1.0: {} @@ -3173,6 +3383,11 @@ snapshots: optionalDependencies: picomatch: 4.0.4 + fetch-blob@3.2.0: + dependencies: + node-domexception: 1.0.0 + web-streams-polyfill: 3.3.3 + file-entry-cache@8.0.0: dependencies: flat-cache: 4.0.1 @@ -3196,11 +3411,31 @@ snapshots: cross-spawn: 7.0.6 signal-exit: 4.1.0 + formdata-polyfill@4.0.10: + dependencies: + fetch-blob: 3.2.0 + fs-constants@1.0.0: {} fsevents@2.3.3: optional: true + gaxios@7.1.5: + dependencies: + extend: 3.0.2 + https-proxy-agent: 7.0.6 + node-fetch: 3.3.2 + transitivePeerDependencies: + - supports-color + + gcp-metadata@8.1.2: + dependencies: + gaxios: 7.1.5 + google-logging-utils: 1.1.3 + json-bigint: 1.0.0 + transitivePeerDependencies: + - supports-color + get-tsconfig@4.14.0: dependencies: resolve-pkg-maps: 1.0.0 @@ -3222,10 +3457,30 @@ snapshots: globals@14.0.0: {} + google-auth-library@10.7.0: + dependencies: + base64-js: 1.5.1 + ecdsa-sig-formatter: 1.0.11 + gaxios: 7.1.5 + gcp-metadata: 8.1.2 + google-logging-utils: 1.1.3 + jws: 4.0.1 + transitivePeerDependencies: + - supports-color + + google-logging-utils@1.1.3: {} + has-flag@4.0.0: {} html-escaper@2.0.2: {} + https-proxy-agent@7.0.6: + dependencies: + agent-base: 7.1.4 + debug: 4.4.3 + transitivePeerDependencies: + - supports-color + ieee754@1.2.1: {} ignore@5.3.2: {} @@ -3288,6 +3543,10 @@ snapshots: dependencies: argparse: 2.0.1 + json-bigint@1.0.0: + dependencies: + bignumber.js: 9.3.1 + json-buffer@3.0.1: {} json-schema-to-ts@3.1.1: @@ -3299,6 +3558,17 @@ snapshots: json-stable-stringify-without-jsonify@1.0.1: {} + jwa@2.0.1: + dependencies: + buffer-equal-constant-time: 1.0.1 + ecdsa-sig-formatter: 1.0.11 + safe-buffer: 5.2.1 + + jws@4.0.1: + dependencies: + jwa: 2.0.1 + safe-buffer: 5.2.1 + keyv@4.5.4: dependencies: json-buffer: 3.0.1 @@ -3314,6 +3584,8 @@ snapshots: lodash.merge@4.6.2: {} + long@5.3.2: {} + loupe@3.2.1: {} lru-cache@10.4.3: {} @@ -3364,10 +3636,23 @@ snapshots: dependencies: semver: 7.8.1 + node-domexception@1.0.0: {} + + node-fetch@3.3.2: + dependencies: + data-uri-to-buffer: 4.0.1 + fetch-blob: 3.2.0 + formdata-polyfill: 4.0.10 + once@1.4.0: dependencies: wrappy: 1.0.2 + openai@6.42.0(ws@8.21.0)(zod@3.25.76): + optionalDependencies: + ws: 8.21.0 + zod: 3.25.76 + optionator@0.9.4: dependencies: deep-is: 0.1.4 @@ -3385,6 +3670,11 @@ snapshots: dependencies: p-limit: 3.1.0 + p-retry@4.6.2: + dependencies: + '@types/retry': 0.12.0 + retry: 0.13.1 + package-json-from-dist@1.0.1: {} parent-module@1.0.1: @@ -3433,6 +3723,21 @@ snapshots: prettier@3.8.3: {} + protobufjs@7.6.2: + dependencies: + '@protobufjs/aspromise': 1.1.2 + '@protobufjs/base64': 1.1.2 + '@protobufjs/codegen': 2.0.5 + '@protobufjs/eventemitter': 1.1.1 + '@protobufjs/fetch': 1.1.1 + '@protobufjs/float': 1.0.2 + '@protobufjs/inquire': 1.1.2 + '@protobufjs/path': 1.1.2 + '@protobufjs/pool': 1.1.0 + '@protobufjs/utf8': 1.1.1 + '@types/node': 22.19.19 + long: 5.3.2 + pump@3.0.4: dependencies: end-of-stream: 1.4.5 @@ -3457,6 +3762,8 @@ snapshots: resolve-pkg-maps@1.0.0: {} + retry@0.13.1: {} + rollup@4.61.1: dependencies: '@types/estree': 1.0.9 @@ -3800,6 +4107,8 @@ snapshots: - tsx - yaml + web-streams-polyfill@3.3.3: {} + which@2.0.2: dependencies: isexe: 2.0.0 @@ -3825,6 +4134,8 @@ snapshots: wrappy@1.0.2: {} + ws@8.21.0: {} + yocto-queue@0.1.0: {} zod@3.25.76: {} diff --git a/pnpm-workspace.yaml b/pnpm-workspace.yaml index e6b5c86b..dec51d92 100644 --- a/pnpm-workspace.yaml +++ b/pnpm-workspace.yaml @@ -33,6 +33,8 @@ catalog: better-sqlite3: ^12.10.0 # Provider SDKs — imported ONLY inside packages/llm/src/adapters/* (the seam fence, 0.F). # The official SDKs chosen by ADR-0011 + tech-stack.md; adding them here realizes that - # decision (no new ADR). The OpenAI-compatible (`openai`) and Gemini (`@google/genai`) - # SDKs land with 1.G / 1.H. + # decision (no new ADR). The OpenAI-compatible (`openai`) adapter serves OpenAI + DeepSeek + # (DeepSeek via a custom baseURL); `@google/genai` backs the Gemini adapter (1.G / 1.H). '@anthropic-ai/sdk': ^0.101.0 + openai: ^6.42.0 + '@google/genai': ^2.8.0