Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/agents-models-ai-sdk.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"agents": patch
---

Add experimental `agents/models/ai-sdk`, an AI SDK provider for Workers AI and AI Gateway over the `AI` binding. See [Models](https://github.com/cloudflare/agents/blob/main/docs/agents/models.md).
1 change: 1 addition & 0 deletions docs/agents/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,7 @@ The differentiator is not "we have durable state" — it is what happens when a
- [Agent Tools](./agent-tools.md) - Run chat-capable sub-agents as tools with streaming child timelines
- [Server-Driven Messages](./server-driven-messages.md) - Autonomous agent workflows: scheduled follow-ups, queue processing, webhooks, chained reasoning
- TODO: [Using AI Models](./using-ai-models.md) - OpenAI, Anthropic, Workers AI, and other providers
- [Models (Experimental)](./models.md) - `createAI` — one AI SDK provider for Workers AI and third-party catalog models, same string space
- [Models for pi-ai (Experimental)](./models-pi-ai.md) - `createAI` for pi-ai: Workers AI ids and third-party models through AI Gateway, as a pi-ai provider
- TODO: [RAG (Retrieval Augmented Generation)](./rag.md) - Vector search with Vectorize
- [Sessions (Experimental)](./sessions.md) - Durable message trees, streamed history, compaction, search, and lossless attachment offload
Expand Down
7 changes: 4 additions & 3 deletions docs/agents/models-pi-ai.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Models for pi-ai (Experimental)

`agents/models/pi-ai` is the [pi-ai](https://github.com/earendil-works/pi/tree/main/packages/ai)
twin of `agents/models/ai-sdk`: the same `createAI` factory, the same options,
twin of [`agents/models/ai-sdk`](./models.md): the same `createAI` factory, the same options,
the same gateway, returning pi-ai `Model` objects instead of AI SDK ones. Use it with pi-ai's
`stream`/`complete`, with [pi-durable](https://github.com/earendil-works/pi/tree/main/packages/durable)'s
`Harness`, or with any framework built on a pi-ai `Models` registry.
Expand Down Expand Up @@ -227,7 +227,7 @@ are not routable today, among them `mistral` (`mistral-conversations`), `google`

Workers AI stays on `env.AI.run`. Its departures from strict OpenAI chat completions (native
events, heartbeats, per-delta usage, per-family request quirks) are absorbed by the same
compat layer the AI SDK provider uses, so the parser only ever sees strict
compat layer the [AI SDK provider](./models.md) uses, so the parser only ever sees strict
OpenAI chunks. Anything the layer had to drop is recorded on the message as a
`cloudflare-compat` diagnostic.

Expand Down Expand Up @@ -289,7 +289,7 @@ diagnostic listing the abandoned attempts, and `message.model` names the model t
answered.

A leg travels with the chain's gateway options — the chain's `id`, `cacheTtl` and the rest win
over a leg's own, and `metadata` merges — as they do in the AI SDK provider. A
over a leg's own, and `metadata` merges — as they do in the [AI SDK provider](./models.md). A
leg built as `ai(model, options)` keeps everything else it was built with. A leg named by a
`@cf/` id has no options of its own, so it also inherits the chain's `headers`, session
affinity and Workers AI knobs; it never inherits the chain's model metadata (`name`, `cost`,
Expand Down Expand Up @@ -320,5 +320,6 @@ and may change in a minor release. An unlisted `@cf/` id resolves with default m

## See also

- [Models for the AI SDK](./models.md) — the same factory for the Vercel AI SDK
- [pi-ai](https://github.com/earendil-works/pi/tree/main/packages/ai) and
[pi-durable](https://github.com/earendil-works/pi/tree/main/packages/durable)
504 changes: 504 additions & 0 deletions docs/agents/models.md

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions examples/next/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ lands.
| [`harnesses/codex`](./harnesses/codex) | This PR | A static Codex Rust/Wasm loop composed as a Lifecycle capability, using LanguageModelV4 and Shell Workspace |
| [`harnesses/pi`](./harnesses/pi) | This PR | Experimental: pi `AgentHarness` on a pinned pi dev build, composed as an example-local Lifecycle capability |
| [`harnesses/self-modifying`](./harnesses/self-modifying) | This PR | A Lifecycle capability runs editable harness revisions in fresh Dynamic Workers with trusted System tools and auto-discovered Custom tools |
| [`models`](./models) | This PR | One `createAI` per framework (AI SDK and pi-ai): `@cf/` ids for Workers AI, vendor models routed through AI Gateway |

Each example is an independent workspace package and should stay focused on one
capability. Once the APIs are stable, move the examples into the main examples
Expand Down
146 changes: 146 additions & 0 deletions examples/next/models/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
# Next: models

The experimental `createAI` factories, `agents/models/ai-sdk` and
`agents/models/pi-ai`, driven from a plain Worker. Same rule, same routes, one
per framework.

## The rule in five lines

1. Workers AI is the one catalog this package keeps up with, so a string is a
`@cf/<author>/<model>` id and nothing else.
2. Every other vendor brings its own code — an `@ai-sdk/*` provider you install,
or a pi-ai registry you import — and `ai(model)` only routes it.
3. AI Gateway holds the vendor credential (unified billing, or a BYOK key you
store on the gateway), so no vendor key lives in the Worker.
4. The gateway is an option, not a product: `{ id, cacheTtl, ... }` flat or
nested under `gateway`, and `default` auto-creates on first use.
5. `createAI({ binding: env.AI })` is the whole configuration: Workers AI goes
through `env.AI.run`, vendor models through `env.AI.gateway(id).run`.

```ts
import { createAnthropic } from "@ai-sdk/anthropic";
import { createAI } from "agents/models/ai-sdk";

const ai = createAI({ binding: env.AI });
const anthropic = createAnthropic({ apiKey: "cloudflare" }); // placeholder

ai("@cf/zai-org/glm-4.7-flash");
ai(anthropic("claude-opus-4-8"), {
fallback: [ai("@cf/zai-org/glm-4.7-flash")]
});
ai.embedding("@cf/baai/bge-base-en-v1.5");
ai.image("@cf/black-forest-labs/flux-1-schnell");
ai.speech("@cf/deepgram/aura-1");
ai.transcription("@cf/openai/whisper-large-v3-turbo");
ai.reranking("@cf/baai/bge-reranker-base");
```

## Routes

Every text route takes a model two ways: `?model=@cf/<author>/<model>` for
Workers AI, or `?vendor=anthropic:claude-opus-4-8` / `?vendor=openai:gpt-5-mini`,
which the Worker builds with that vendor's own provider and hands to `ai()`.

| Route | Query | Does |
| ------------- | ----------------------------- | ---------------------------------------------------------------------- |
| `/` | `model` or `vendor`, `prompt` | `generateText` — returns text, usage and `providerMetadata.cloudflare` |
| `/stream` | `model` or `vendor`, `prompt` | `streamText` as a plain text stream |
| `/tools` | `model` or `vendor`, `prompt` | `generateText` with a `getWeather` tool and `stopWhen: isStepCount(3)` |
| `/json` | `model` or `vendor`, `prompt` | `generateText` with `Output.object` over a zod schema |
| `/embed` | `text` (repeatable) | `embedMany` with `@cf/baai/bge-base-en-v1.5` |
| `/image` | `model`, `prompt` | `generateImage` — answers with the image itself (JPEG or PNG) |
| `/speech` | `model`, `text`, `voice` | `generateSpeech` — answers with the MP3 |
| `/transcribe` | `model`, `text` | `transcribe`; POST audio, or GET and the Worker speaks a line first |
| `/rerank` | `query`, `doc` (repeatable) | `rerank` with `@cf/baai/bge-reranker-base` |
| `/fallback` | `prompt` | `ai(anthropic(...), { fallback: [ai("@cf/...")] })` |

`model` defaults to `@cf/zai-org/glm-4.7-flash`. The modality routes are Workers
AI only — that is where the catalog is.

The pi-ai twin of each text route lives under `/pi/*`, built with `createAI`
from `agents/models/pi-ai`. A vendor there is a pi `Model` object taken from
pi-ai's own registry import:

| Route | Query | Does |
| -------------- | ----------------------------- | ------------------------------------------------------------------------- |
| `/pi` | `model` or `vendor`, `prompt` | `ai.completeSimple` — returns text, usage and the `cloudflare` diagnostic |
| `/pi/stream` | `model` or `vendor`, `prompt` | `ai.streamSimple` as a plain text stream of `text_delta` events |
| `/pi/tools` | `model` or `vendor`, `prompt` | pi-agent-core `Agent` with `getWeather`, driven by `ai.streamFn` |
| `/pi/fallback` | `prompt` | a vendor model with a Workers AI fallback leg |

```ts
import { anthropicProvider } from "@earendil-works/pi-ai/providers/anthropic";
import { createAI } from "agents/models/pi-ai";

const ai = createAI({ binding: env.AI });
const claude = anthropicProvider()
.getModels()
.find((model) => model.id === "claude-opus-4-8")!;

await ai.complete(ai(claude), { messages });
new Agent({
streamFn: ai.streamFn,
initialState: { model: ai("@cf/zai-org/glm-4.7-flash") }
});
```

pi ships a generated Cloudflare AI Gateway registry, so that module also takes
`ai("anthropic/claude-opus-4.8")` as a string — pi's catalog knowledge, not
ours. The AI SDK module has no such registry and takes `@cf/` strings only.

## Run it

```sh
pnpm install
pnpm dev
```

## curl

Workers AI:

```sh
curl 'http://localhost:8787/?model=@cf/zai-org/glm-4.7-flash&prompt=hello'
curl 'http://localhost:8787/stream?model=@cf/zai-org/glm-4.7-flash&prompt=explain+durable+objects'
curl 'http://localhost:8787/tools?model=@cf/zai-org/glm-4.7-flash&prompt=weather+in+Lisbon'
curl 'http://localhost:8787/json?model=@cf/zai-org/glm-4.7-flash&prompt=a+fact+about+octopuses'
curl 'http://localhost:8787/embed?text=hello&text=world'
curl 'http://localhost:8787/image?prompt=a+cyberpunk+lizard' -o image.jpg
curl 'http://localhost:8787/speech?text=hello+from+Cloudflare' -o speech.mp3
curl 'http://localhost:8787/transcribe'
curl --data-binary @speech.mp3 'http://localhost:8787/transcribe'
curl 'http://localhost:8787/rerank?query=which+is+cooler&doc=a+lizard&doc=a+cat'
```

`@cf/deepgram/nova-3` and `@cf/deepgram/flux` want the audio as an object body
that the JSON run path cannot carry, so `/transcribe` answers a clear error for
them; use a whisper id.

Anthropic and OpenAI, the same routes with `vendor` instead of `model`:

```sh
curl 'http://localhost:8787/?vendor=anthropic:claude-opus-4-8&prompt=hello'
curl 'http://localhost:8787/tools?vendor=anthropic:claude-opus-4-8&prompt=weather+in+Lisbon'
curl 'http://localhost:8787/?vendor=openai:gpt-5-mini&prompt=hello'
curl 'http://localhost:8787/fallback'
```

pi-ai, the same two forms under `/pi`:

```sh
curl 'http://localhost:8787/pi?model=@cf/zai-org/glm-4.7-flash&prompt=hello'
curl 'http://localhost:8787/pi?vendor=anthropic:claude-opus-4-8&prompt=hello'
curl 'http://localhost:8787/pi/stream?vendor=anthropic:claude-opus-4-8&prompt=explain+durable+objects'
curl 'http://localhost:8787/pi/tools?vendor=openai:gpt-5-mini&prompt=weather+in+Lisbon'
curl 'http://localhost:8787/pi/fallback'
```

Vendor models bill through the account's unified billing, or through a key you
store on the gateway. An account with neither answers with a gateway error
rather than a completion — `/fallback` is the route that shows what to do about
it. On `/stream` the response status is already 200 by the time the model is
reached, so a failure before the first token shows up as an empty body with the
error in the `wrangler dev` log.

`agents/models/ai-sdk` and `agents/models/pi-ai` are experimental and their
surfaces may change.
12 changes: 12 additions & 0 deletions examples/next/models/env.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
/* eslint-disable */
// Generated by Wrangler by running `wrangler types --include-runtime=false env.d.ts` (hash: 9c7b605a3c68793cf554d9938d5868e2)
interface __BaseEnv_Env {
AI: Ai;
}
declare namespace Cloudflare {
interface GlobalProps {
mainModule: typeof import("./src/index");
}
interface Env extends __BaseEnv_Env {}
}
interface Env extends __BaseEnv_Env {}
28 changes: 28 additions & 0 deletions examples/next/models/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
{
"name": "@cloudflare/agents-next-models-example",
"description": "Next: one createAI per framework (AI SDK and pi-ai) over Workers AI and third-party catalog models, gateway as an option",
"private": true,
"type": "module",
"version": "0.0.0",
"scripts": {
"dev": "wrangler dev",
"deploy": "wrangler deploy",
"typecheck": "tsc --noEmit",
"types": "wrangler types env.d.ts --include-runtime false"
},
"dependencies": {
"@ai-sdk/anthropic": "^4.0.0",
"@ai-sdk/openai": "^4.0.0",
"@ai-sdk/provider": "^4.0.0",
"@earendil-works/pi-agent-core": "^0.80.2",
"@earendil-works/pi-ai": "^0.99.2",
"agents": "*",
"ai": "^7.0.0",
"zod": "^4.5.4"
},
"devDependencies": {
"@cloudflare/workers-types": "^5.20260729.1",
"typescript": "^6.0.3",
"wrangler": "^4.115.0"
}
}
Loading
Loading