[Bug]: Injected browser rule forbids falling back after preview_open succeeds, so agents stop and ask the user which browser to use #16404

Filed by Claude (Opus 5.5, running inside T3 Code) on behalf of @janpilu, at their request. The investigation was done by agents on their machine. We reviewed the last week of our T3 threads.

Before submitting

• I searched existing issues and did not find a duplicate. The failures that set this off are tracked in [Bug]: After a preview automation timeout, the session fails over to a client on another machine that doesn't have the thread open #13540 (failover after a timeout), [Bug]: A timed-out preview_evaluate keeps running in the renderer and makes all preview tools report "No preview automation host" until it finishes #16264 (a timed-out evaluate keeps running), [Bug]: Preview tabs sometimes log "sandboxed_renderer.bundle.js script failed to run" (Electron #54149, fixed in 44.4.4) #13990 (sandboxed_renderer.bundle.js failed to run) and [Bug]: Desktop Browser automation errors lose their type over IPC and report as 'failed on client' #15336 (errors reduced to "failed on client"). This report is about how agents react to those failures. It isn't about the failures themselves.
• I included enough detail to reproduce or investigate the problem.

Area

apps/server (src/provider/T3OrchestrationInstructions.ts)

Summary

T3 injects this browser rule into every provider session:

Use an alternative browser system only when the T3 preview tools are absent, the user explicitly requests another browser, or preview_open returns an explicit unsupported/unavailable error.

None of the three conditions covers the common failure: preview_open works, then later preview calls keep failing. Examples are calls that time out, a tab that lands on chrome-error://chromewebdata/, recordingStop failed on client …, or a different client ID answering after the session moved to another host (#13540). In that state the agent may not use Playwright or any other browser unless the user "explicitly requests" it. So unattended agents stop and ask the user which browser to use.

The person being asked usually can't answer. Our verification agents asked four times in three days, with questions like "Close the extra client?" and "Leave only the development-machine client connected?". Their answers were "what does this mean" and "no idea what you are talking about". The same thing happened to a coworker on a different machine, because the rule is injected for everyone. A project's own AGENTS.md or skill text that allows Playwright doesn't help: agents read the injected developer instruction as taking priority, and project text doesn't count as the user explicitly asking.

Steps to reproduce

1. Give an agent a task that requires browser verification. Allow "T3 preview tools or Playwright" in the project instructions, and don't mention a browser in the prompt.
2. Make preview calls fail after a successful preview_open. The simplest way is [Bug]: After a preview automation timeout, the session fails over to a client on another machine that doesn't have the thread open #13540 with two desktop clients connected: preview_evaluate with new Promise(() => {}) times out, and later calls go to the other host.
3. Watch the agent.

Expected behavior

After a bounded number of failed preview calls (for example, two failures on the same step), the agent may switch to another browser tool. It says so and quotes the raw preview error, and it doesn't need to ask. Alternatively, the rule says plainly that project or skill instructions can authorise a fallback.

Actual behavior

The agent stops the turn and asks the user for permission to switch browsers. Unattended pipelines stall until someone answers a question they don't have the context for. In one case, on 2026-10-05, the agent asked to switch because it thought its recordings were frozen. They weren't: its own frame-sampling script was returning frame 0 for every timestamp. The same rule meant even a mistaken diagnosis ended up as a question to the user.

Suggested change

Add a fourth condition along these lines: "…or after preview calls on an already-open tab have failed twice for the same step (timeout, chrome-error://, a failed recording stop, or a different client answering). In that case, state the raw error and switch without asking." Keep the general preference for T3 preview tools as it is.

Impact

Major degradation or frequent failure. Our verification pipeline stalls with an unanswerable question every few runs. Until #13540 and #16264 ship, this rule is what turns their failures into blocked turns.

Version

Nightly 0.0.46-nightly.20261005.2667 (37de6cbde65c), macOS 26.6.2 arm64. The instruction text is in apps/server/src/provider/T3OrchestrationInstructions.ts. In the shipped bundle it's binCli-*.mjs.