Skip to content

fix(agents): give browserTool its own description instead of codemode's - #2488

Merged
ben-reitz merged 3 commits into
mainfrom
fix/browser-tool-guidance
Oct 6, 2026
Merged

ben-reitz merged 3 commits into
mainfrom
fix/browser-tool-guidance

Conversation

@ben-reitz

@ben-reitz ben-reitz commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Summary

browserTool now passes its own description to codemode instead of using codemode's generic one. That text told the model to codemode.search first (search doesn't cover CDP commands) and described approvals, snippets, and replay, which this tool doesn't use. This change applies to the AI SDK, TanStack AI, and pi tools, because all three use the same core.

 createBrowserToolCore (browser/browser-tool.ts)
   runtime.tool({
-    connectorHints: { cdp: BROWSER_INSTRUCTIONS }   # appended to codemode's generic text
+    description: intro + cdp API + BROWSER_INSTRUCTIONS + timeout + screenshotHint
   })
 Tool description (about 1,100 → 750 tokens)
-## Workflow: codemode.search → codemode.describe → call
-## Rules: discovery pass, codemode.step, approvals, "paused", deterministic replay
-## Snippets
-Never Promise.all CDP calls (replay ordering)
+## cdp: send / attachToTarget / spec (filter it, 650 KB) / getDebugLog
+Runtime.evaluate: returnByValue, awaitPromise, check exceptionDetails
+Page.navigate: check errorText, then poll document.readyState
+Results cut at about 24k chars, a single cdp.send result over 1 MB fails the run
+Each run times out after 60s, so split long jobs
+"Failed to start Worker" SyntaxError = your code didn't parse
 BrowserSessionConnector.send (browser/session-connector.ts)
   result = cdp.send(...)
+  if method is Page.captureScreenshot and result is over 1 MB
+    throw "capture the viewport only, or use JPEG"

It also corrects the timeoutMs doc comment, which said the default was 30s; the real default is 60s.

Evidence

  • Before: in a pi harness session, the model made four discovery calls before doing any work. codemode.search("cdp navigate page evaluate links") and codemode.search("scrape links") both returned [], then it called describe on cdp.send and cdp.spec. It then used fixed sleeps instead of readyState, read exceptionDetails from the wrong place, and wrote a 120-page crawl inside a 60s run.
    After: the description includes the rules for each of those mistakes and has no search step. Not yet re-run with a real model.
  • Before: each new rule's claim was unchecked.
    After: I checked them against real Chrome with a probe:
    • A thrown error in Runtime.evaluate completes, with exceptionDetails at the top level.
    • Without returnByValue, an object comes back with only an objectId.
    • Without awaitPromise, a promise comes back as {}.
    • Page.navigate to an unresolvable host returns errorText and doesn't throw.
    • readyState is still loading right after Page.navigate.
    • cdp.spec() is 656 KB.
    • fetch is blocked, and setTimeout works.
    • Promise.all over cdp.send works.
    • A bad regex fails with "Failed to start Worker ... executor.js:107".
  • Before: a 1.5 MB screenshot failed with codemode's "Write large data to a file or workspace" message. The sandbox has no file system.
    After: the test explains how to take a smaller screenshot when one is too large passes. The error suggests capturing the viewport only or using JPEG.
  • The test describes cdp and its rules without codemode's generic text passes. It checks that the description contains sessionId: "active", returnByValue: true, document.readyState, and times out after 60s, and doesn't contain codemode.search, paused, or Snippets.
  • pnpm run check, the browser capability and connector tests, and 24 real-Chrome tests pass.

Merge Danger

Door: two-way

Only the tool's description text and one error message change. There's no new API and no stored data.

Blast Radius: model-behaviour

The tools in agents/browser/ai-sdk and agents/browser/tanstack-ai, and the upcoming agents/browser/pi tool, all get the new description. The model no longer hears about codemode.search, codemode.step, or snippets. None of those do anything useful for this tool today. A screenshot over 1 MB fails with a different message; it already failed before. createBrowserTools is unchanged.

@changeset-bot

changeset-bot Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 7460e9f

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages
Name Type
agents Patch
@cloudflare/agent-think Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@agent-think

agent-think Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

✅ agents import sizes: no significant changes (1e97e117 → 7460e9fb, workflow run)

@pkg-pr-new

pkg-pr-new Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

agents

npm i https://pkg.pr.new/agents@2488

@cloudflare/ai-chat

npm i https://pkg.pr.new/@cloudflare/ai-chat@2488

@cloudflare/codemode

npm i https://pkg.pr.new/@cloudflare/codemode@2488

hono-agents

npm i https://pkg.pr.new/hono-agents@2488

@cloudflare/shell

npm i https://pkg.pr.new/@cloudflare/shell@2488

@cloudflare/think

npm i https://pkg.pr.new/@cloudflare/think@2488

@cloudflare/voice

npm i https://pkg.pr.new/@cloudflare/voice@2488

@cloudflare/worker-bundler

npm i https://pkg.pr.new/@cloudflare/worker-bundler@2488

commit: 7460e9f

@ben-reitz ben-reitz changed the title fix(agents): put browserTool's CDP rules in its description fix(agents): give browserTool its own description instead of codemode's Oct 6, 2026
@ben-reitz
ben-reitz marked this pull request as ready for review October 6, 2026 10:33

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 4 potential issues.

Devin Review

Comment thread packages/agents/src/browser/ai-sdk.ts Outdated
Comment thread packages/agents/src/browser/session-connector.ts Outdated
Comment thread docs/agents/browse-the-web.md
Comment thread packages/agents/src/browser/browser-tool.ts
@ben-reitz
ben-reitz merged commit 98ec934 into main Oct 6, 2026
16 checks passed
@ben-reitz
ben-reitz deleted the fix/browser-tool-guidance branch October 6, 2026 11:15
@github-actions github-actions Bot mentioned this pull request Oct 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant