Repository navigation
feat(agent): pause a turn at its step or cost limit, show its estimated cost, and undo its file changes - #2873
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. 🗂️ Base branches to auto review: 2
Please check the settings in the CodeRabbit UI or the
|
🤖 Internal: Discord sync markerAuto-managed by the Discord notification workflow. Stores the linked Discord message ID and forum thread ID. Do not edit or delete. |
dylan-savage
left a comment
There was a problem hiding this comment.
Approving. I found no bugs in the limits, the meter or Undo, and the new suites pass locally.
Follow-up: the estimate leaves out thinking. The stand-in sends no thinking param, so Sonnet 5 thinks by default and bills it as output the browser never sees. That makes "~$X est." and the $2 cap lower bounds; worth saying so in the meta line and the setting text.
4a7d660 to
8074fb3
Compare
a31f363 to
9b087fd
Compare
…th the rr-verify-agent skill rr-verify-agent, in .claude/skills/ beside #2870's verify-rocketride, drives Rocket Builder's Agent tab with Playwright against a real engine and model, one journey per behavior this stack adds, and records proof: a screenshot and an ARIA snapshot after each action, the files read back through the engine, the chat, the console and the WebSocket frames the page sent. It changes no product code. - run takes the machine's port lock, starts the engine, rocket-ui and the shell, drives each journey, and stops only what it started. Its exit code is the worst result. launch, drive, doctor, engine and cleanup are the single steps. - --engine source, the default, starts dist/server through #2870's launcher, with a store of the run's own. build-engine fills dist/server from the nearest server release and syncs the branch's engine Python. Without the launcher, launch falls back to the pinned image, says so, and names the checks that cannot pass there. - It takes an engine as its own only by what its launch recorded, so stale state cannot reset or remove another checkout's engine or signal a process that reused a pid. - Each journey resets the workspace, the sessions and the layout, seeds its fixtures through the engine, and signs in as a user does. Evidence goes to .verify/agent/ (git-ignored), with key names only, never values. - Each check failed alone on a narrow mutation of the product code it guards. Checks no such mutation could fail alone were cut. - The cost check matches #2873's meta line, which now says the estimate leaves out thinking. - lint:scripts lets the skill's scripts import playwright from the skill's own package.json, which CI does not install. Every other import is still checked. The eval runner, its data and shared:eval are in their own PR on top of this one.
|
Done. The meta line reads "~$X est., not counting thinking", and the Cost Limit setting says the limit leaves out the model's thinking, so a turn can cost more. I checked first: the stand-in's config has no |
8074fb3 to
791c40f
Compare
9b087fd to
545b01f
Compare
…th the rr-verify-agent skill rr-verify-agent, in .claude/skills/ beside #2870's verify-rocketride, drives Rocket Builder's Agent tab with Playwright against a real engine and model, one journey per behavior this stack adds, and records proof: a screenshot and an ARIA snapshot after each action, the files read back through the engine, the chat, the console and the WebSocket frames the page sent. It changes no product code. - run takes the machine's port lock, starts the engine, rocket-ui and the shell, drives each journey, and stops only what it started. Its exit code is the worst result. launch, drive, doctor, engine and cleanup are the single steps. - --engine source, the default, starts dist/server through #2870's launcher, with a store of the run's own. build-engine fills dist/server from the nearest server release and syncs the branch's engine Python. Without the launcher, launch falls back to the pinned image, says so, and names the checks that cannot pass there. - It takes an engine as its own only by what its launch recorded, so stale state cannot reset or remove another checkout's engine or signal a process that reused a pid. - Each journey resets the workspace, the sessions and the layout, seeds its fixtures through the engine, and signs in as a user does. Evidence goes to .verify/agent/ (git-ignored), with key names only, never values. - Each check failed alone on a narrow mutation of the product code it guards. Checks no such mutation could fail alone were cut. - The cost check matches #2873's meta line, which now says the estimate leaves out thinking. - lint:scripts lets the skill's scripts import playwright from the skill's own package.json, which CI does not install. Every other import is still checked. The eval runner, its data and shared:eval are in their own PR on top of this one.
…ed cost, and undo its file changes
- A turn takes at most 60 steps (waves) by default. At the limit the loop
plans no more, answers with the best-effort summary, and the turn ends as
needs_input with a continue request: the card says "Paused after 60 steps"
and Continue starts a new turn with a fresh budget. Messages sent during
the turn do not reset the count. Agent Step Limit in Settings changes it,
and 0 turns it off.
- A UsageMeter estimates each model request from characters, 4 to a token,
priced from a dated table with one row for the stand-in's model
(claude-sonnet-5, Anthropic's price page, read 2026-10-27). The prompt
start is priced at the cache-write rate on a turn's first request, at the
cache-read rate after, and as a write again after a gap over five minutes,
only for a provider that caches it and a question that asks.
- Before every model request, the retry after a pipeline restart and the
best-effort summary included, the meter reserves the input estimate and
an output allowance. A request that could pass the turn's cost limit is
not sent, and the turn pauses with Continue. A failed request keeps its
reservation. The limit is $2 per turn by default (Agent Cost Limit, 0
turns it off); a model with no price shows tokens and has no limit.
- The bot row's meta line shows the estimate ("~$0.42 est."), and
TurnResult.usage carries the tokens and the cost, marked estimated.
- The latest turn's card has Undo for that turn's file changes. The file
toolkit records, in memory only, each path's content at its first touch
and the version and hash each agent change left. Undo checks every path
first: no editor holds unsaved changes, the version is the agent's, no one
else changed it between agent changes, and the stored content matches what
the agent left (the server's copy afresh, or the staged content for a path
still staged). Then every inverse starts in one block, through the
versioned write. In manual mode a file the agent created and never
committed has its staged entry dropped, so nothing stays pending. A note
row says what Undo did, and the model reads it in the next turn.
…hinking The stand-in model request sends no thinking param, so Sonnet 5 thinks by default and bills it as output the browser never sees. The estimate counts only the reply text, so "~$X est." and the cost limit are lower bounds. The row's meta line now reads "~$X est., not counting thinking", and the cost limit setting says a turn can cost more than the limit.
791c40f to
4df0253
Compare
545b01f to
09abac0
Compare
…th the rr-verify-agent skill rr-verify-agent, in .claude/skills/ beside #2870's verify-rocketride, drives Rocket Builder's Agent tab with Playwright against a real engine and model, one journey per behavior this stack adds, and records proof: a screenshot and an ARIA snapshot after each action, the files read back through the engine, the chat, the console and the WebSocket frames the page sent. It changes no product code. - run takes the machine's port lock, starts the engine, rocket-ui and the shell, drives each journey, and stops only what it started. Its exit code is the worst result. launch, drive, doctor, engine and cleanup are the single steps. - --engine source, the default, starts dist/server through #2870's launcher, with a store of the run's own. build-engine fills dist/server from the nearest server release and syncs the branch's engine Python. Without the launcher, launch falls back to the pinned image, says so, and names the checks that cannot pass there. - It takes an engine as its own only by what its launch recorded, so stale state cannot reset or remove another checkout's engine or signal a process that reused a pid. - Each journey resets the workspace, the sessions and the layout, seeds its fixtures through the engine, and signs in as a user does. Evidence goes to .verify/agent/ (git-ignored), with key names only, never values. - Each check failed alone on a narrow mutation of the product code it guards. Checks no such mutation could fail alone were cut. - The cost check matches #2873's meta line, which now says the estimate leaves out thinking. - lint:scripts lets the skill's scripts import playwright from the skill's own package.json, which CI does not install. Every other import is still checked. The eval runner, its data and shared:eval are in their own PR on top of this one.
c4b698b
into
feat/rocket-agent-turn-contract
…user, and stop only its own runs in Auto (#2869) * refactor(agent): move FileTracker changes() here from the save-race PR The turn's result card lists what the run changed, so FileTracker records each file's change (added, modified, deleted or renamed, and where a renamed file came from) and changes() lists them. A file the run created and then deleted is no change. A deleted file keeps its version, so a file the run only deleted is not read back as seen. The stored copy a delete is checked against stays with each read, write and rename. * feat(agent): end each turn with a result card, let the agent ask the user, and stop only its own runs in Auto - runAgent, agent() and AgentSession.execute return a TurnResult: the outcome (done, needs_input, stopped, failed, incomplete), the files and checks, the usage and any needs-you request. Every turn ends once through execute, including a refused message, an error, Stop and an unmount. - The session file is version 2. It keeps the turns, linked to the rows that close them, and the runs the conversation started. A version 1 file loads with neither. Every saved field passes through redactText. - The shell's ChatMessage gains an optional `card`, and ChatView gains `onCardAction`. The Agent tab puts each turn's card under its row: links to the changed files, Open Variables, a button per choice, and Continue. - ask_user ends the turn as needs_input, as a question or a missing Variable. Only the user's next message answers the request. - terminate and restart ask first in Ask mode. In Auto they run only on a run this conversation started with use, after a fresh getTaskStatus matches the record and the scope. Any other run is refused, and the model is told to ask the user. - A turn running when the Agent tab unmounts is saved with its rows and a closing row (known bug 5). - A use refused because the pipeline is already running says how to run the edited version (known bug 14). * fix(agent): answer a needs-you request only after it showed, document the row card, and check the run on the Ask card A turn the user stopped in the wave that called ask_user kept its request. Its card said only "Stopped", yet the next message answered the question and disabled its buttons. Only a needs_input turn keeps its request now. A message the user sent during the wave that called ask_user stayed queued, then ran as its own turn and answered the question the user had not seen. The loop now takes such a message before it ends the turn. It drops the question, and the model reads the message and may ask again. ChatMessage.card and the onCardAction prop of ChatView and MessageList are documented in ROCKETRIDE_UI_COMPONENTS.md and the ChatView gallery entry. The Ask card for terminate and restart said "This conversation started it." from the run record alone. The engine gives a later run on the same project and source the same token, so the card could claim the user's run. It now reads the run's status first, as the Auto run step does. * feat(agent): pause a turn at its step or cost limit, show its estimated cost, and undo its file changes (#2873) * feat(agent): pause a turn at its step or cost limit, show its estimated cost, and undo its file changes - A turn takes at most 60 steps (waves) by default. At the limit the loop plans no more, answers with the best-effort summary, and the turn ends as needs_input with a continue request: the card says "Paused after 60 steps" and Continue starts a new turn with a fresh budget. Messages sent during the turn do not reset the count. Agent Step Limit in Settings changes it, and 0 turns it off. - A UsageMeter estimates each model request from characters, 4 to a token, priced from a dated table with one row for the stand-in's model (claude-sonnet-5, Anthropic's price page, read 2026-10-27). The prompt start is priced at the cache-write rate on a turn's first request, at the cache-read rate after, and as a write again after a gap over five minutes, only for a provider that caches it and a question that asks. - Before every model request, the retry after a pipeline restart and the best-effort summary included, the meter reserves the input estimate and an output allowance. A request that could pass the turn's cost limit is not sent, and the turn pauses with Continue. A failed request keeps its reservation. The limit is $2 per turn by default (Agent Cost Limit, 0 turns it off); a model with no price shows tokens and has no limit. - The bot row's meta line shows the estimate ("~$0.42 est."), and TurnResult.usage carries the tokens and the cost, marked estimated. - The latest turn's card has Undo for that turn's file changes. The file toolkit records, in memory only, each path's content at its first touch and the version and hash each agent change left. Undo checks every path first: no editor holds unsaved changes, the version is the agent's, no one else changed it between agent changes, and the stored content matches what the agent left (the server's copy afresh, or the staged content for a path still staged). Then every inverse starts in one block, through the versioned write. In manual mode a file the agent created and never committed has its staged entry dropped, so nothing stays pending. A note row says what Undo did, and the model reads it in the next turn. * fix(agent): say that the cost estimate and the cost limit leave out thinking The stand-in model request sends no thinking param, so Sonnet 5 thinks by default and bills it as output the browser never sees. The estimate counts only the reply text, so "~$X est." and the cost limit are lower bounds. The row's meta line now reads "~$X est., not counting thinking", and the cost limit setting says a turn can cost more than the limit. --------- Co-authored-by: Krish Garg <kg@g14-node1.tailb99b1b.ts.net> --------- Co-authored-by: Krish Garg <kg@g14-node1.tailb99b1b.ts.net>
…mmit all meets a teammate's change (#2884) * fix(agent): check a staged file against the copy the page read, so Commit all meets a teammate's change In manual mode a staged file's conflict anchor was a fresh server read taken when it was first staged. A teammate's change from before then became the anchor, so Commit all wrote over it. The anchor is now the copy the page read, so the commit meets a conflict. The view's prime records the path only: the dev preview primed what its batch read returned, unsaved typing included, and the cache then served it as the server's copy. The preview and teammate exploration covers both. Going back to a team drops the page's cached copies, so a delete the agent decided on before a switch away and back anchored on the server's copy after it, and Commit all deleted a teammate's change. The view's delete now takes an anchor, the hash of the stored copy the delete is based on. delete_file passes the copy the agent last read or wrote. Under unsaved typing that is the stored copy below the typing, which Discard mine brings back. The view checks the anchor in the delete's turn, against the same capture its tombstone records, so no copy read later can become the delete's anchor. In write-through mode it checks the server's copy. * feat(agent): end each turn with a result card, let the agent ask the user, and stop only its own runs in Auto (#2869) * refactor(agent): move FileTracker changes() here from the save-race PR The turn's result card lists what the run changed, so FileTracker records each file's change (added, modified, deleted or renamed, and where a renamed file came from) and changes() lists them. A file the run created and then deleted is no change. A deleted file keeps its version, so a file the run only deleted is not read back as seen. The stored copy a delete is checked against stays with each read, write and rename. * feat(agent): end each turn with a result card, let the agent ask the user, and stop only its own runs in Auto - runAgent, agent() and AgentSession.execute return a TurnResult: the outcome (done, needs_input, stopped, failed, incomplete), the files and checks, the usage and any needs-you request. Every turn ends once through execute, including a refused message, an error, Stop and an unmount. - The session file is version 2. It keeps the turns, linked to the rows that close them, and the runs the conversation started. A version 1 file loads with neither. Every saved field passes through redactText. - The shell's ChatMessage gains an optional `card`, and ChatView gains `onCardAction`. The Agent tab puts each turn's card under its row: links to the changed files, Open Variables, a button per choice, and Continue. - ask_user ends the turn as needs_input, as a question or a missing Variable. Only the user's next message answers the request. - terminate and restart ask first in Ask mode. In Auto they run only on a run this conversation started with use, after a fresh getTaskStatus matches the record and the scope. Any other run is refused, and the model is told to ask the user. - A turn running when the Agent tab unmounts is saved with its rows and a closing row (known bug 5). - A use refused because the pipeline is already running says how to run the edited version (known bug 14). * fix(agent): answer a needs-you request only after it showed, document the row card, and check the run on the Ask card A turn the user stopped in the wave that called ask_user kept its request. Its card said only "Stopped", yet the next message answered the question and disabled its buttons. Only a needs_input turn keeps its request now. A message the user sent during the wave that called ask_user stayed queued, then ran as its own turn and answered the question the user had not seen. The loop now takes such a message before it ends the turn. It drops the question, and the model reads the message and may ask again. ChatMessage.card and the onCardAction prop of ChatView and MessageList are documented in ROCKETRIDE_UI_COMPONENTS.md and the ChatView gallery entry. The Ask card for terminate and restart said "This conversation started it." from the run record alone. The engine gives a later run on the same project and source the same token, so the card could claim the user's run. It now reads the run's status first, as the Auto run step does. * feat(agent): pause a turn at its step or cost limit, show its estimated cost, and undo its file changes (#2873) * feat(agent): pause a turn at its step or cost limit, show its estimated cost, and undo its file changes - A turn takes at most 60 steps (waves) by default. At the limit the loop plans no more, answers with the best-effort summary, and the turn ends as needs_input with a continue request: the card says "Paused after 60 steps" and Continue starts a new turn with a fresh budget. Messages sent during the turn do not reset the count. Agent Step Limit in Settings changes it, and 0 turns it off. - A UsageMeter estimates each model request from characters, 4 to a token, priced from a dated table with one row for the stand-in's model (claude-sonnet-5, Anthropic's price page, read 2026-10-27). The prompt start is priced at the cache-write rate on a turn's first request, at the cache-read rate after, and as a write again after a gap over five minutes, only for a provider that caches it and a question that asks. - Before every model request, the retry after a pipeline restart and the best-effort summary included, the meter reserves the input estimate and an output allowance. A request that could pass the turn's cost limit is not sent, and the turn pauses with Continue. A failed request keeps its reservation. The limit is $2 per turn by default (Agent Cost Limit, 0 turns it off); a model with no price shows tokens and has no limit. - The bot row's meta line shows the estimate ("~$0.42 est."), and TurnResult.usage carries the tokens and the cost, marked estimated. - The latest turn's card has Undo for that turn's file changes. The file toolkit records, in memory only, each path's content at its first touch and the version and hash each agent change left. Undo checks every path first: no editor holds unsaved changes, the version is the agent's, no one else changed it between agent changes, and the stored content matches what the agent left (the server's copy afresh, or the staged content for a path still staged). Then every inverse starts in one block, through the versioned write. In manual mode a file the agent created and never committed has its staged entry dropped, so nothing stays pending. A note row says what Undo did, and the model reads it in the next turn. * fix(agent): say that the cost estimate and the cost limit leave out thinking The stand-in model request sends no thinking param, so Sonnet 5 thinks by default and bills it as output the browser never sees. The estimate counts only the reply text, so "~$X est." and the cost limit are lower bounds. The row's meta line now reads "~$X est., not counting thinking", and the cost limit setting says a turn can cost more than the limit. --------- Co-authored-by: Krish Garg <kg@g14-node1.tailb99b1b.ts.net> --------- Co-authored-by: Krish Garg <kg@g14-node1.tailb99b1b.ts.net> --------- Co-authored-by: Krish Garg <kg@g14-node1.tailb99b1b.ts.net>
…, and stop the tool clock while the user answers (#2883) * feat(agent): ask before the agent changes a file with unsaved changes, and stop the tool clock while the user answers When the agent wanted to change a file the user had unsaved changes in, the call was refused and the model told the user to save first. Now the user is asked: Save mine first, Discard mine or Keep mine, one dialog at a time. The open editors do what the user chose. A call's time limit stands still while the question is open, in its prepare and in its run. A call its tool refused before changing anything is unstarted, so the finish rule asks for no check. One in which the user's save landed or sent a write still counts as a change, even when that save or another editor's save then failed, since the file may have changed. A save that joins a Ctrl+S already on its way counts once it lands. Open questions close as Keep mine when the run ends or the user stops it. * test(agent): count a tab behind only by a failed write as the known limit, also after a later read With versioned writes, a read that takes a teammate's copy in counts it for the editor too. When that read waited for a write that then failed, the failure's count lands before the read and the editor misses only that one, while the file moves past the failure's version. The text tab exploration broke its not-stored rule in 3 such orders, where the tab shows the stored file. They now count under the known limit for a failed write, as the same order without the teammate's change already does. * fix(agent): show the unsaved-changes question below 1024 px, where the closed drawer is a hidden dialog Below 1024 px the shell's sidebar is a drawer. It stays in the page as a modal dialog, with `visibility: hidden` while it is closed. otherDialogOpen() counted it, so every question resolved unasked, and the agent was told to try again after the user closed a dialog they could not see. otherDialogOpen() now counts only dialogs that checkVisibility({ visibilityProperty: true }) reports visible. The call also passes the option's old name, checkVisibilityCSS, which Chrome 105 to 120 know instead. An open drawer still blocks the question. A browser without checkVisibility counts every dialog, as before, so a question never opens over another dialog. * fix(agent): check a staged file against the copy the page read, so Commit all meets a teammate's change (#2884) * fix(agent): check a staged file against the copy the page read, so Commit all meets a teammate's change In manual mode a staged file's conflict anchor was a fresh server read taken when it was first staged. A teammate's change from before then became the anchor, so Commit all wrote over it. The anchor is now the copy the page read, so the commit meets a conflict. The view's prime records the path only: the dev preview primed what its batch read returned, unsaved typing included, and the cache then served it as the server's copy. The preview and teammate exploration covers both. Going back to a team drops the page's cached copies, so a delete the agent decided on before a switch away and back anchored on the server's copy after it, and Commit all deleted a teammate's change. The view's delete now takes an anchor, the hash of the stored copy the delete is based on. delete_file passes the copy the agent last read or wrote. Under unsaved typing that is the stored copy below the typing, which Discard mine brings back. The view checks the anchor in the delete's turn, against the same capture its tombstone records, so no copy read later can become the delete's anchor. In write-through mode it checks the server's copy. * feat(agent): end each turn with a result card, let the agent ask the user, and stop only its own runs in Auto (#2869) * refactor(agent): move FileTracker changes() here from the save-race PR The turn's result card lists what the run changed, so FileTracker records each file's change (added, modified, deleted or renamed, and where a renamed file came from) and changes() lists them. A file the run created and then deleted is no change. A deleted file keeps its version, so a file the run only deleted is not read back as seen. The stored copy a delete is checked against stays with each read, write and rename. * feat(agent): end each turn with a result card, let the agent ask the user, and stop only its own runs in Auto - runAgent, agent() and AgentSession.execute return a TurnResult: the outcome (done, needs_input, stopped, failed, incomplete), the files and checks, the usage and any needs-you request. Every turn ends once through execute, including a refused message, an error, Stop and an unmount. - The session file is version 2. It keeps the turns, linked to the rows that close them, and the runs the conversation started. A version 1 file loads with neither. Every saved field passes through redactText. - The shell's ChatMessage gains an optional `card`, and ChatView gains `onCardAction`. The Agent tab puts each turn's card under its row: links to the changed files, Open Variables, a button per choice, and Continue. - ask_user ends the turn as needs_input, as a question or a missing Variable. Only the user's next message answers the request. - terminate and restart ask first in Ask mode. In Auto they run only on a run this conversation started with use, after a fresh getTaskStatus matches the record and the scope. Any other run is refused, and the model is told to ask the user. - A turn running when the Agent tab unmounts is saved with its rows and a closing row (known bug 5). - A use refused because the pipeline is already running says how to run the edited version (known bug 14). * fix(agent): answer a needs-you request only after it showed, document the row card, and check the run on the Ask card A turn the user stopped in the wave that called ask_user kept its request. Its card said only "Stopped", yet the next message answered the question and disabled its buttons. Only a needs_input turn keeps its request now. A message the user sent during the wave that called ask_user stayed queued, then ran as its own turn and answered the question the user had not seen. The loop now takes such a message before it ends the turn. It drops the question, and the model reads the message and may ask again. ChatMessage.card and the onCardAction prop of ChatView and MessageList are documented in ROCKETRIDE_UI_COMPONENTS.md and the ChatView gallery entry. The Ask card for terminate and restart said "This conversation started it." from the run record alone. The engine gives a later run on the same project and source the same token, so the card could claim the user's run. It now reads the run's status first, as the Auto run step does. * feat(agent): pause a turn at its step or cost limit, show its estimated cost, and undo its file changes (#2873) * feat(agent): pause a turn at its step or cost limit, show its estimated cost, and undo its file changes - A turn takes at most 60 steps (waves) by default. At the limit the loop plans no more, answers with the best-effort summary, and the turn ends as needs_input with a continue request: the card says "Paused after 60 steps" and Continue starts a new turn with a fresh budget. Messages sent during the turn do not reset the count. Agent Step Limit in Settings changes it, and 0 turns it off. - A UsageMeter estimates each model request from characters, 4 to a token, priced from a dated table with one row for the stand-in's model (claude-sonnet-5, Anthropic's price page, read 2026-10-27). The prompt start is priced at the cache-write rate on a turn's first request, at the cache-read rate after, and as a write again after a gap over five minutes, only for a provider that caches it and a question that asks. - Before every model request, the retry after a pipeline restart and the best-effort summary included, the meter reserves the input estimate and an output allowance. A request that could pass the turn's cost limit is not sent, and the turn pauses with Continue. A failed request keeps its reservation. The limit is $2 per turn by default (Agent Cost Limit, 0 turns it off); a model with no price shows tokens and has no limit. - The bot row's meta line shows the estimate ("~$0.42 est."), and TurnResult.usage carries the tokens and the cost, marked estimated. - The latest turn's card has Undo for that turn's file changes. The file toolkit records, in memory only, each path's content at its first touch and the version and hash each agent change left. Undo checks every path first: no editor holds unsaved changes, the version is the agent's, no one else changed it between agent changes, and the stored content matches what the agent left (the server's copy afresh, or the staged content for a path still staged). Then every inverse starts in one block, through the versioned write. In manual mode a file the agent created and never committed has its staged entry dropped, so nothing stays pending. A note row says what Undo did, and the model reads it in the next turn. * fix(agent): say that the cost estimate and the cost limit leave out thinking The stand-in model request sends no thinking param, so Sonnet 5 thinks by default and bills it as output the browser never sees. The estimate counts only the reply text, so "~$X est." and the cost limit are lower bounds. The row's meta line now reads "~$X est., not counting thinking", and the cost limit setting says a turn can cost more than the limit. --------- Co-authored-by: Krish Garg <kg@g14-node1.tailb99b1b.ts.net> --------- Co-authored-by: Krish Garg <kg@g14-node1.tailb99b1b.ts.net> --------- Co-authored-by: Krish Garg <kg@g14-node1.tailb99b1b.ts.net> --------- Co-authored-by: Krish Garg <kg@g14-node1.tailb99b1b.ts.net>
…th the rr-verify-agent skill rr-verify-agent, in .claude/skills/ beside #2870's verify-rocketride, drives Rocket Builder's Agent tab with Playwright against a real engine and model, one journey per behavior this stack adds, and records proof: a screenshot and an ARIA snapshot after each action, the files read back through the engine, the chat, the console and the WebSocket frames the page sent. It changes no product code. - run takes the machine's port lock, starts the engine, rocket-ui and the shell, drives each journey, and stops only what it started. Its exit code is the worst result. launch, drive, doctor, engine and cleanup are the single steps. - --engine source, the default, starts dist/server through #2870's launcher, with a store of the run's own. build-engine fills dist/server from the nearest server release and syncs the branch's engine Python. Without the launcher, launch falls back to the pinned image, says so, and names the checks that cannot pass there. - It takes an engine as its own only by what its launch recorded, so stale state cannot reset or remove another checkout's engine or signal a process that reused a pid. - Each journey resets the workspace, the sessions and the layout, seeds its fixtures through the engine, and signs in as a user does. Evidence goes to .verify/agent/ (git-ignored), with key names only, never values. - Each check failed alone on a narrow mutation of the product code it guards. Checks no such mutation could fail alone were cut. - The cost check matches #2873's meta line, which now says the estimate leaves out thinking. - lint:scripts lets the skill's scripts import playwright from the skill's own package.json, which CI does not install. Every other import is still checked. The eval runner, its data and shared:eval are in their own PR on top of this one.
Before this, a turn could run without end and spend without a cap, the user saw no cost, and nothing could take back what the agent changed in files.
Now a turn pauses after 60 steps or at an estimated $2, with a card and a Continue button that starts a new turn with a fresh budget. Both limits are settings, and 0 turns either off. Each reply shows its estimated cost ("~$0.42 est., not counting thinking"), from a dated price table for the stand-in's model, and the turn result records it. The latest turn's card has Undo, which puts that turn's file changes back. It refuses the whole Undo, naming the files, when any file has unsaved edits, changed since, or can't be checked, so it never writes over someone else's work.
Part of #2718: layer 9 of 11, on #2869's branch
feat/rocket-agent-turn-contract. Merge the stack top-down: each PR into the branch below it, down to #2835 intofeat/rocket-agent-browser. A bottom-up squash conflicts unless each next branch is rebased onto the squashed base first. The testing layer builds on this one.The step limit
A step is a wave, the thinking block the user sees. A turn takes at most 60 by default. Agent Step Limit, a new setting next to
agentApproval, changes it, and 0 turns it off.limited, and the turn ends asneeds_inputwith acontinuerequest.stopped, with no Continue, however the run was ending.TurnResult.usageas diagnostics. Nothing counts them against a limit.The cost estimate and the cost limit
The browser sees no token counts: the model pipeline answers with text only (known bug 15). A
UsageMeter(usage.ts) estimates each turn from characters, 4 to a token, input and output apart.createChatModelfeeds it once per model request, the retry after a pipeline restart included.runAgentfeeds it for a model a test passes. It takes characters, soModelCallis unchanged. Reported token counts can feed it later without changingModelCalleither.claude-sonnet-5, from Anthropic's price page (https://platform.claude.com/docs/en/about-claude/pricing, read 2026-10-27): $2.00 input, $2.50 cache write (5-minute), $0.20 cache read and $10.00 output per million tokens. The row says the provider caches the prompt start: Anthropic's model node marks it (SUPPORTS_PROMPT_CACHE_PREFIXinnodes/src/nodes/llm_anthropic/anthropic.py).cachePrefix. The start (role, instructions and history, what the engine caches) is priced at the cache-write rate on the turn's first request, at the cache-read rate after, and at the write rate again when more than five minutes passed since the last request that wrote or read it. The rest of the question is at the full input rate.needs_input, kindcontinue, reasoncost_limit. A reply replaces the allowance with its own estimate. A failed request, and one whose provider failed, keeps its reservation.metaline, andTurnResult.usage.costwithestimated: true.usage.tokensholds the token estimate.thinkingparam, so Sonnet 5 thinks by default and bills that thinking as output, which the browser never sees. So the estimate and the $2 limit are lower bounds. The meta line says "not counting thinking", and the Agent Cost Limit setting says a turn can cost more than the limit.needs_input, reasoncost_limit, with Continue.Undo
The latest turn's card has Undo. It puts back the files that turn changed, one level, and nothing else: never a run, a deploy, a publish, a schedule or a Variable. The model has no undo tool.
What is recorded. The file toolkit keeps an
UndoRecordper run (toolkits/undo.ts). At a path's first touch in the turn it keeps the content before (null for a new file). After each change it keeps the version that change left and the hash of the content or the absence it left. A rename records both ends, and a delete or a rename reads the file first for it. A delete of a path with nothing staged reads the server's copy afresh, since the cached copy may be older than a teammate's change that the delete removes too: a file the agent never read has no copy for the commit check to refuse the delete with. Each change after a path's first touch is compared with what the agent's last change of it left: the version it starts from, and the hash of the content it replaces. When either differs, someone else changed the path during the turn, and Undo refuses it: its content before the turn would overwrite the other change. A freshread_fileof a teammate's edit does not change that, since the record keeps what the agent itself left, not what it last read. The record stays in memory: no content goes into the session file, a prompt or the chat, so a reload or a team switch leaves no Undo. Above 4 MB of kept content the card says "Undo is unavailable: this turn changed more than 4 MB of files."When it is offered. Only on the latest turn, and only while the chat is idle and every change of the turn has ended. A call that timed out may still be writing after the turn ends; the button stays disabled until it does, then the card is drawn again. A turn that starts takes Undo off the card before it, since it will be the latest.
The checks. Before anything changes, every path must pass:
peekBase), so a teammate's edit on the server is seen. For a path whose change is still staged in Manual mode, it is the staged content (readVersionedwithstored: true), since the server still holds the old copy. Only a missing file reads as an absence. Any other failed read, such as a dropped connection, refuses the whole Undo and names the path ("pipelines/a.pipe could not be checked (socket closed)").If any path fails, Undo changes nothing and the row names each path and why ("Undo refused, nothing was changed: pipelines/a.pipe has unsaved changes in an editor.").
The inverses. The editors and versions are checked again, and then every inverse starts in one synchronous block. The content before is written back through #2840's versioned write with
expectset to the version the agent left. A file the agent created is removed, and a deleted one is written back. In Manual mode, writing the content back drops the staged entry, and a file the agent created and never committed has its staged entry dropped (discardPath) instead of a delete staged, sohasPendingChanges()is false after. Auto mode writes through. The store has no conditional write, so a change that lands on the server between the check and the write is lost; that is known bug 3, the same as any save. A failure part way is reported as it is: "Reversed: … Not reversed: … (the error)."The row. Undo adds a note row: "Undo: the user reversed the file changes of the turn before, so these files are back as they were before it: pipelines/a.pipe."
toHistorydrops note rows, with one exception for a row that starts "Undo: ", which the model reads as the user's. A refusal row does not reach the model, since nothing changed. A message the user sends while Undo runs (its checks, inverses and row) waits until the row is added, so the turn it opens starts after Undo and reads the row (AgentSession.afterUndo, through the tab's send). A tab closed while Undo runs drops the message it held. The session then adds the row to its own rows and saves them, so the reopened conversation shows the row and the next turn reads it. A conversation reopened while Undo runs loads once that row is saved.Settings and the view contract
apps/rocket-ui/package.jsonaddsrocketride.pipeBuilder.agentStepLimit(60) androcketride.pipeBuilder.agentCostLimit(2), both numbers with a minimum of 0.agentHost.tsreads them with the approval settings, and a missing or bad value reads as the default.AgentHostgainslimits; a host without it gets 60 steps and $2.IWorkspaceFileSystemgains two optional members the staged view already has:changedPaths()anddiscardPath(). Undo uses them to tell a staged path from a committed one and to drop a staged entry.AppToolkittakes anonScaffoldhook: a turn that creates an app cannot be undone, since the scaffold's files are not in the record.Tests
Each new behavior has a test that fails without it. On the first round's commit, 49 mutations, one per behavior, each failed at least one test. Each fix from review has a test that fails with that fix reverted alone, and no other new test fails, with two exceptions. Reverting the fresh-read fix also fails the first round's delete test, whose check that fix moved into the record. Reverting the fix that saves Undo's row after its tab closed also fails the reopen test, since the reopened tab reads that row.
limits.test.ts: the pause at the step limit with its card and Continue; a steer does not reset the count; 0 means off; the estimate without the cache discount and with it (the 1.25x first write, a read, the five-minute gap, a row that does not cache); a failed request keeps its reservation; the reservation refuses the retry after a restart, which is never sent; the cost limit ends the turn asneeds_inputwith Continue in Auto, with the plan and the summary both refused; a summary the cost limit refuses after two unusable replies still pauses at the cost limit with Continue; a Stop during the step-limit summary ends the turn stopped with no Continue; the meta text; an unknown model.undo.test.ts, on the real staged view: each kind (modified, created, deleted, renamed) in Auto and in Manual; Manual with the server at A and the agent's B staged, with and without an earlier staged A1, never writing the server; a created file committed since; refusal on a dirty editor, a moved version, another writer between two agent changes, a teammate's server edit to a created file, and an editor that turns dirty while Undo reads the server; a delete after a teammate's server edit puts back the teammate's copy, and refuses when the agent had changed the file before; a teammate's edit the agent read again still refuses, whether the agent then deletes or edits the file; a check read that fails with a dropped connection refuses the whole Undo; a failure part way; a change still on its way; the 4 MB cap; a folder delete; a file that is not text, as the store decodes it, through each of write, edit, delete and rename; create_app and no undo tool; the history row.agent.test.ts, throughAgentSessionand the shell'sTurnController: Continue starts a new turn with its own steps; a message sent during the last step and one sent during the summary are rows of the paused turn, the summary reads the first, and no turn starts for them; the card offers Undo and the row shows the cost; no Undo after a reload, in the saved card, or after a team switch; Undo puts the file back, disables itself while it runs, adds the row the next turn reads, and is gone after; a turn that starts takes it away; a message sent while Undo waits on its server check waits too, and the turn it opens starts after the row and reads it; a message held by a tab that closes during Undo is dropped; an Undo whose tab closed saves its row, which the reopened conversation shows and the next turn reads; a conversation reopened during Undo loads after it, and sends nothing before.usagewithout the newtokensfield.Fixes from review
TurnControlleropened a turn for each, with a fresh budget and no Continue. The loop now takes them before the best-effort summary and again after it (answerFromGatheredinwave/loop.ts). Test: "a message sent in the last step or during the summary is heard by the paused turn, so no turn starts before Continue" (agent.test.ts, with a one-step limit).incomplete, reasonempty_plan, with no Continue.bestEffortnow says when the cost limit refused it, and the run ends ascost_limit. Test: "a best-effort summary the cost limit refuses ends the turn at the cost limit with Continue, though unusable replies ended the run" (limits.test.ts).edit_filegoes through the same check as write, delete and rename. Test: "a file that is not text, read as the store decodes it, makes the turn one that cannot be undone, whichever tool changed it" (undo.test.ts).undo.test.ts).undo.test.ts).undo.test.ts, delete and edit).limited, with a paused card and Continue. A run the user stopped while the summary was written now endsstopped, with no Continue. Test: "a Stop while the step-limit summary is written ends the turn stopped, with no Continue" (limits.test.ts).agent.test.ts, with Undo's server read held).agent.test.ts).agent.test.ts).agent.test.ts).After the restack
deletedContentfails it ("alpha"). Its second half now shows the delete refused, Undo refused ("b.txt changed since the turn"), and the teammate's copy kept.deleteNowkeeps both layers. It asks about unsaved changes first, reads the content before for Undo, sends the delete with the commit check's anchor, and turns the check's refusal into "delete_file refused: … changed since you read it", without telling Undo a delete may have landed.requestgoes only on a needs-you turn, as feat(agent): end each turn with a result card, let the agent ask the user, and stop only its own runs in Auto #2869's review made it: theask_userquestion, or the limit's Continue request. A Stopped card has none.Proof
On this commit:
./builder shared:test: 520 tests, 518 pass, 0 fail, 2 todo (fix(agent): keep secrets out of the model in the browser agent #2836's known limits). Every exploration breaks 0 rules, with the same order counts as C1../builder rocket-ui:buildpasses.Browser journeys, on an engine built from this branch (server-v3.5.0 binary, the branch's engine Python), with Claude through the stand-in, driven by the testing layer's driver. All eleven pass every check:
needs_input,limited, 2 waves, with acontinuerequest and an estimated cost. Continue started a second turn of its own 2 steps that answered the request.Departures from the direction
hasPendingChanges()would stay true. In Auto mode the end state is the same as a rename back.addSystemMessage), not a status row. Outside a turn the chat adds rows only that way; status rows come from a running turn.toHistory's one exception is for it.processreturns usage, the meter takes it and marks it reported" needs token counts the browser never gets (known bug 15). The meter takes characters only.usage.modelRequestsnow counts every request sent, the retry after a pipeline restart included, and not a request the cost limit refused. C1 counted model calls.Known limits
create_directorystays after Undo, and in Manual mode its staged folder stays pending.expecton the write-back never fires in practice. The second check in the same synchronous block sees any change in this page first. It stays, as the direction asks.Follow-up after approval
The estimate left out thinking, and nothing said so. The stand-in's request has no
thinkingparam: its model node sets one only whenextendedThinkingis on, and LangChain drops the unset field. Sonnet 5 then runs adaptive thinking, billed as output, with its text omitted from the reply, and the meter counts only the reply's text. The meta line now reads "~$0.42 est., not counting thinking", and the Agent Cost Limit setting says the estimate leaves out the model's thinking, so a turn can cost more than the limit. The two tests that check the meta text expect the new text, and each fails on the old line. The journeys above ran before this change; it changes only that text and the setting's description.Proved with: 518 of 520 shared tests (2 todo for #2836's limits), all twelve ordering explorations with 0 broken rules, dirtyClose and the shell chat tests, the type-checks and test-file check, the rocket-ui build, and eleven browser journeys passing every check on an engine built from this branch, two of them new (step-cap, undo).
The stack
Eleven PRs, each on the one below it. Merge top-down: each PR squashes into the branch below it, starting with #2885 and ending with #2835 into
feat/rocket-agent-browser. A top-down rehearsal in a scratch clone was clean at every step and ended on #2885's tree. Bottom-up also works if each next branch is first rebased onto the squashed base.