Skip to content

feat(agent): pause a turn at its step or cost limit, show its estimated cost, and undo its file changes - #2873

Merged
dylan-savage merged 2 commits into
feat/rocket-agent-turn-contractfrom
feat/rocket-agent-limits-undo
Oct 28, 2026
Merged

dylan-savage merged 2 commits into
feat/rocket-agent-turn-contractfrom
feat/rocket-agent-limits-undo

Conversation

@krishhgg

@krishhgg krishhgg commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

Before this, a turn could run without end and spend without a cap, the user saw no cost, and nothing could take back what the agent changed in files.

Now a turn pauses after 60 steps or at an estimated $2, with a card and a Continue button that starts a new turn with a fresh budget. Both limits are settings, and 0 turns either off. Each reply shows its estimated cost ("~$0.42 est., not counting thinking"), from a dated price table for the stand-in's model, and the turn result records it. The latest turn's card has Undo, which puts that turn's file changes back. It refuses the whole Undo, naming the files, when any file has unsaved edits, changed since, or can't be checked, so it never writes over someone else's work.

Part of #2718: layer 9 of 11, on #2869's branch feat/rocket-agent-turn-contract. Merge the stack top-down: each PR into the branch below it, down to #2835 into feat/rocket-agent-browser. A bottom-up squash conflicts unless each next branch is rebased onto the squashed base first. The testing layer builds on this one.

The step limit

A step is a wave, the thinking block the user sees. A turn takes at most 60 by default. Agent Step Limit, a new setting next to agentApproval, changes it, and 0 turns it off.

  • At the limit the loop asks for no next plan. It runs the existing best-effort summary, stops with limited, and the turn ends as needs_input with a continue request.
  • The card reads "Paused after 60 steps", with "Continue starts a new turn with a fresh budget." and a Continue button. Continue sends "Continue." as the user's reply, which answers the request, and the new turn counts its own steps.
  • A message the user sends during the turn does not reset the count.
  • A message sent during the last step or during the best-effort summary belongs to the paused turn. The loop takes the queued messages before the summary, so the summary reads them, and again after it. None is left queued to start a turn of its own, so only Continue, or the user's next message after the pause, starts a new turn.
  • A Stop while the best-effort summary is written ends the turn as stopped, with no Continue, however the run was ending.
  • Model requests (retries included) and tool calls stay in TurnResult.usage as diagnostics. Nothing counts them against a limit.

The cost estimate and the cost limit

The browser sees no token counts: the model pipeline answers with text only (known bug 15). A UsageMeter (usage.ts) estimates each turn from characters, 4 to a token, input and output apart.

  • createChatModel feeds it once per model request, the retry after a pipeline restart included. runAgent feeds it for a model a test passes. It takes characters, so ModelCall is unchanged. Reported token counts can feed it later without changing ModelCall either.
  • Prices come from one dated table with one row, the stand-in's model, claude-sonnet-5, from Anthropic's price page (https://platform.claude.com/docs/en/about-claude/pricing, read 2026-10-27): $2.00 input, $2.50 cache write (5-minute), $0.20 cache read and $10.00 output per million tokens. The row says the provider caches the prompt start: Anthropic's model node marks it (SUPPORTS_PROMPT_CACHE_PREFIX in nodes/src/nodes/llm_anthropic/anthropic.py).
  • The cache discount applies only when the row says the provider caches and the question sets cachePrefix. The start (role, instructions and history, what the engine caches) is priced at the cache-write rate on the turn's first request, at the cache-read rate after, and at the write rate again when more than five minutes passed since the last request that wrote or read it. The rest of the question is at the full input rate.
  • The reservation. Before each request, the retry and the best-effort summary included, the meter reserves the request's input estimate plus an output allowance: the model's configured output maximum, or 8,192 tokens when none is set, as for the stand-in. If the estimate so far plus the reservation would pass the limit, the request is not sent, and the turn ends as needs_input, kind continue, reason cost_limit. A reply replaces the allowance with its own estimate. A failed request, and one whose provider failed, keeps its reservation.
  • One estimate drives the limit and the display: "~$0.42 est., not counting thinking" in the bot row's existing meta line, and TurnResult.usage.cost with estimated: true. usage.tokens holds the token estimate.
  • Thinking is not in the estimate. The stand-in sends no thinking param, so Sonnet 5 thinks by default and bills that thinking as output, which the browser never sees. So the estimate and the $2 limit are lower bounds. The meta line says "not counting thinking", and the Agent Cost Limit setting says a turn can cost more than the limit.
  • A model the table does not have shows tokens and no price, and the cost limit does not apply. The card says "No price for this model, so the cost limit does not apply (~12,345 tokens est.)." Today that happens only when a caller passes its own model pipeline token; the stand-in's model has a price.
  • The limit is $2 per turn by default. Agent Cost Limit, a second new setting, changes it, and 0 turns it off. The approval mode never raises it.
  • At the limit the card reads "Paused at the cost limit of $2.00", with the estimate so far and Continue.
  • A best-effort summary the cost limit refuses ends the turn at the cost limit, whatever ended the run (unusable replies, a model error, the step limit). The turn is needs_input, reason cost_limit, with Continue.

Undo

The latest turn's card has Undo. It puts back the files that turn changed, one level, and nothing else: never a run, a deploy, a publish, a schedule or a Variable. The model has no undo tool.

What is recorded. The file toolkit keeps an UndoRecord per run (toolkits/undo.ts). At a path's first touch in the turn it keeps the content before (null for a new file). After each change it keeps the version that change left and the hash of the content or the absence it left. A rename records both ends, and a delete or a rename reads the file first for it. A delete of a path with nothing staged reads the server's copy afresh, since the cached copy may be older than a teammate's change that the delete removes too: a file the agent never read has no copy for the commit check to refuse the delete with. Each change after a path's first touch is compared with what the agent's last change of it left: the version it starts from, and the hash of the content it replaces. When either differs, someone else changed the path during the turn, and Undo refuses it: its content before the turn would overwrite the other change. A fresh read_file of a teammate's edit does not change that, since the record keeps what the agent itself left, not what it last read. The record stays in memory: no content goes into the session file, a prompt or the chat, so a reload or a team switch leaves no Undo. Above 4 MB of kept content the card says "Undo is unavailable: this turn changed more than 4 MB of files."

When it is offered. Only on the latest turn, and only while the chat is idle and every change of the turn has ended. A call that timed out may still be writing after the turn ends; the button stays disabled until it does, then the card is drawn again. A turn that starts takes Undo off the card before it, since it will be the latest.

The checks. Before anything changes, every path must pass:

  • no editor holds unsaved changes there;
  • it is at the version the agent's last change left;
  • nothing else changed it between two of the agent's changes: each later change started from the version, and replaced the content, that the agent's last change left;
  • what stores it still matches the hash of what the agent left. In Auto mode, and for a path committed since the turn, that is a fresh read of the server's copy (peekBase), so a teammate's edit on the server is seen. For a path whose change is still staged in Manual mode, it is the staged content (readVersioned with stored: true), since the server still holds the old copy. Only a missing file reads as an absence. Any other failed read, such as a dropped connection, refuses the whole Undo and names the path ("pipelines/a.pipe could not be checked (socket closed)").

If any path fails, Undo changes nothing and the row names each path and why ("Undo refused, nothing was changed: pipelines/a.pipe has unsaved changes in an editor.").

The inverses. The editors and versions are checked again, and then every inverse starts in one synchronous block. The content before is written back through #2840's versioned write with expect set to the version the agent left. A file the agent created is removed, and a deleted one is written back. In Manual mode, writing the content back drops the staged entry, and a file the agent created and never committed has its staged entry dropped (discardPath) instead of a delete staged, so hasPendingChanges() is false after. Auto mode writes through. The store has no conditional write, so a change that lands on the server between the check and the write is lost; that is known bug 3, the same as any save. A failure part way is reported as it is: "Reversed: … Not reversed: … (the error)."

The row. Undo adds a note row: "Undo: the user reversed the file changes of the turn before, so these files are back as they were before it: pipelines/a.pipe." toHistory drops note rows, with one exception for a row that starts "Undo: ", which the model reads as the user's. A refusal row does not reach the model, since nothing changed. A message the user sends while Undo runs (its checks, inverses and row) waits until the row is added, so the turn it opens starts after Undo and reads the row (AgentSession.afterUndo, through the tab's send). A tab closed while Undo runs drops the message it held. The session then adds the row to its own rows and saves them, so the reopened conversation shows the row and the next turn reads it. A conversation reopened while Undo runs loads once that row is saved.

Settings and the view contract

  • apps/rocket-ui/package.json adds rocketride.pipeBuilder.agentStepLimit (60) and rocketride.pipeBuilder.agentCostLimit (2), both numbers with a minimum of 0. agentHost.ts reads them with the approval settings, and a missing or bad value reads as the default. AgentHost gains limits; a host without it gets 60 steps and $2.
  • IWorkspaceFileSystem gains two optional members the staged view already has: changedPaths() and discardPath(). Undo uses them to tell a staged path from a committed one and to drop a staged entry.
  • AppToolkit takes an onScaffold hook: a turn that creates an app cannot be undone, since the scaffold's files are not in the record.
  • No shell change. The meta line, the card and the note row are the chat's existing ones.

Tests

Each new behavior has a test that fails without it. On the first round's commit, 49 mutations, one per behavior, each failed at least one test. Each fix from review has a test that fails with that fix reverted alone, and no other new test fails, with two exceptions. Reverting the fresh-read fix also fails the first round's delete test, whose check that fix moved into the record. Reverting the fix that saves Undo's row after its tab closed also fails the reopen test, since the reopened tab reads that row.

  • limits.test.ts: the pause at the step limit with its card and Continue; a steer does not reset the count; 0 means off; the estimate without the cache discount and with it (the 1.25x first write, a read, the five-minute gap, a row that does not cache); a failed request keeps its reservation; the reservation refuses the retry after a restart, which is never sent; the cost limit ends the turn as needs_input with Continue in Auto, with the plan and the summary both refused; a summary the cost limit refuses after two unusable replies still pauses at the cost limit with Continue; a Stop during the step-limit summary ends the turn stopped with no Continue; the meta text; an unknown model.
  • undo.test.ts, on the real staged view: each kind (modified, created, deleted, renamed) in Auto and in Manual; Manual with the server at A and the agent's B staged, with and without an earlier staged A1, never writing the server; a created file committed since; refusal on a dirty editor, a moved version, another writer between two agent changes, a teammate's server edit to a created file, and an editor that turns dirty while Undo reads the server; a delete after a teammate's server edit puts back the teammate's copy, and refuses when the agent had changed the file before; a teammate's edit the agent read again still refuses, whether the agent then deletes or edits the file; a check read that fails with a dropped connection refuses the whole Undo; a failure part way; a change still on its way; the 4 MB cap; a folder delete; a file that is not text, as the store decodes it, through each of write, edit, delete and rename; create_app and no undo tool; the history row.
  • agent.test.ts, through AgentSession and the shell's TurnController: Continue starts a new turn with its own steps; a message sent during the last step and one sent during the summary are rows of the paused turn, the summary reads the first, and no turn starts for them; the card offers Undo and the row shows the cost; no Undo after a reload, in the saved card, or after a team switch; Undo puts the file back, disables itself while it runs, adds the row the next turn reads, and is gone after; a turn that starts takes it away; a message sent while Undo waits on its server check waits too, and the turn it opens starts after the row and reads it; a message held by a tab that closes during Undo is dropped; an Undo whose tab closed saves its row, which the reopened conversation shows and the next turn reads; a conversation reopened during Undo loads after it, and sends nothing before.
  • Two C1 assertions compare usage without the new tokens field.

Fixes from review

  • A message sent at the step limit started its own turn. The loop returned at the limit without taking the queued messages, so the shell's TurnController opened a turn for each, with a fresh budget and no Continue. The loop now takes them before the best-effort summary and again after it (answerFromGathered in wave/loop.ts). Test: "a message sent in the last step or during the summary is heard by the paused turn, so no turn starts before Continue" (agent.test.ts, with a one-step limit).
  • A summary the cost limit refused kept the reason the run was ending with. After two unusable replies the turn ended incomplete, reason empty_plan, with no Continue. bestEffort now says when the cost limit refused it, and the run ends as cost_limit. Test: "a best-effort summary the cost limit refuses ends the turn at the cost limit with Continue, though unusable replies ended the run" (limits.test.ts).
  • A file that is not text could be undone into the wrong bytes. The check looked at the JavaScript type, but the store decodes every file to a string. Content with a NUL or a U+FFFD now counts as not text, and edit_file goes through the same check as write, delete and rename. Test: "a file that is not text, read as the store decodes it, makes the turn one that cannot be undone, whichever tool changed it" (undo.test.ts).
  • Undo of a delete could put back an older copy. The delete took its content before from the page's cache. After a read, a teammate's edit on the server and a delete, Undo wrote the old copy back and reported success. A delete of a path with nothing staged now drops the cached copy and reads the server's, and a copy that differs from what the agent's last change left makes a path the agent already changed one Undo refuses (see the entry on a fresh read below). Test: "auto: Undo of a delete puts back what the server held when the agent deleted it; a teammate's edit after the agent's own change refuses the delete" (undo.test.ts).
  • Any failed check read counted as a missing file. For a deleted path, a dropped connection passed the check and let the other inverses run. Only "File not found" from the store, or "No such file" from the view, now reads as an absence; any other failure refuses the whole Undo and names the path. Test: "a check read that fails for any reason but a missing file refuses the whole Undo, and names the path" (undo.test.ts).
  • A fresh read hid a teammate's edit from Undo. The delete check compared the server's copy with the agent's last read. After the agent changed a file, a teammate edited it, and the agent read it again and deleted it, the two matched, and Undo wrote the content before the turn over the teammate's edit. An edit after the fresh read did the same. The record now compares the content each change replaces with what the agent's last change of that path left, for every tool, and a mismatch refuses that path and so the whole Undo. Test: "a teammate's edit the agent read again before its next change still refuses the Undo, whichever tool changed it next" (undo.test.ts, delete and edit).
  • A Stop during the step-limit summary left a paused turn. The summary took the stop as a failed model call, so the turn ended limited, with a paused card and Continue. A run the user stopped while the summary was written now ends stopped, with no Continue. Test: "a Stop while the step-limit summary is written ends the turn stopped, with no Continue" (limits.test.ts).
  • A turn could start while Undo checked the server. It got the history without Undo's row, and then the Undo changed files and added a row saying "the turn before" was reversed. While an Undo runs, the tab now holds a new message until the row is added, so the next turn starts after Undo and reads the row. Test: "a message sent while Undo runs waits for it: the turn it opens starts after the note row and reads it" (agent.test.ts, with Undo's server read held).
  • A message held for Undo could start a turn in a closed tab. When the tab closed while Undo ran, the held message still went to that tab's chat and started a turn no one could see. The session now counts the tab's mounts and sends a held message only to the tab that held it. The test waits for Undo's row before it checks, not a fixed number of ticks. Test: "a message held while Undo runs is dropped when the tab closes before Undo ends, so no turn starts unseen" (agent.test.ts).
  • Undo's row was lost when its tab closed. Undo still put the files back, but its row went to the closed tab's chat. The reopened conversation had no row, so the next turn did not know the files went back. When the tab that pressed Undo has closed by the time Undo ends, the session adds the row to its own rows and saves them. Test: "an Undo that ends after its tab closed saves its note row, so the reopened conversation shows it and the next turn reads it" (agent.test.ts).
  • A conversation reopened during Undo skipped the wait. The reopened tab loaded a new session, which knew of no running Undo, so its first turn ran while Undo still checked the server. The running Undo is now kept by session id. A conversation reopened while it runs loads once Undo has saved its row, so the tab shows the row and its first turn reads it. A send from any session of the conversation waits for it too. Test: "a conversation reopened while Undo runs loads after it: nothing is sent before Undo ends, and the first turn reads the note" (agent.test.ts).

After the restack

  • The commit check refuses a delete over a teammate's change. The PR below this one passes the copy the agent last read or wrote with each delete, and refuses the delete when the file holds another. So after the agent reads a file and a teammate edits it, the agent's delete no longer goes through, and nothing is left for Undo to put back. Undo's delete test now deletes a file an editor tab read and the agent did not, with no copy to check, and Undo still puts the teammate's copy back. Reverting the fresh read in deletedContent fails it ("alpha"). Its second half now shows the delete refused, Undo refused ("b.txt changed since the turn"), and the teammate's copy kept.
  • deleteNow keeps both layers. It asks about unsaved changes first, reads the content before for Undo, sends the delete with the commit check's anchor, and turns the check's refusal into "delete_file refused: … changed since you read it", without telling Undo a delete may have landed.
  • request goes only on a needs-you turn, as feat(agent): end each turn with a result card, let the agent ask the user, and stop only its own runs in Auto #2869's review made it: the ask_user question, or the limit's Continue request. A Stopped card has none.

Proof

On this commit:

  • ./builder shared:test: 520 tests, 518 pass, 0 fail, 2 todo (fix(agent): keep secrets out of the model in the browser agent #2836's known limits). Every exploration breaks 0 rules, with the same order counts as C1.
  • The shell's dirtyClose 14 of 14, and with its chat tests 22 of 22.
  • Type-checks of shared, shell and rocket-ui, and the test-file check: 0 errors. ./builder rocket-ui:build passes.
  • ESLint: 0 new problems in the 19 changed files. Prettier: no file is worse than on C1. gitleaks: no leaks.

Browser journeys, on an engine built from this branch (server-v3.5.0 binary, the branch's engine Python), with Claude through the stand-in, driven by the testing layer's driver. All eleven pass every check:

Journey Checks
unsaved-edits, Keep mine, Save mine first, Discard mine 5, 5 and 5
steps closed-dirty-tab, split-save 3 and 7
build-pipeline 4
secrets 16
needs-you 7
foreign-terminate 8
step-cap (new) 5
undo (new) 8
  • step-cap. With Agent Step Limit at 2, a task that follows a chain of notes ended with "Paused after 2 steps" and Continue, after the best-effort answer, and the row showed "~$0.02 est.". The saved turn is needs_input, limited, 2 waves, with a continue request and an estimated cost. Continue started a second turn of its own 2 steps that answered the request.
  • undo. The agent changed a note's heading. Undo put the note back, left its row, and the card lost its button. The next question to the model carried the row, and the next turn changed the heading again. With the note typed into in an editor, Undo on that turn was refused and named the note, which kept the agent's change.

Departures from the direction

  • A rename is reversed at both ends, not renamed back. The old path gets its content back and the new path is removed. In Manual mode a staged rename back would leave a delete staged at the new path, so hasPendingChanges() would stay true. In Auto mode the end state is the same as a rename back.
  • The Undo row is a note row (addSystemMessage), not a status row. Outside a turn the chat adds rows only that way; status rows come from a running turn. toHistory's one exception is for it.
  • More turns cannot be undone than the direction names. A turn that deleted or moved a folder, created an app, or changed a file that is not text gets "Undo is unavailable" with the reason, since the record cannot put those back. The store decodes every file as UTF-8, so a file that is not text still reads as a string. Content with a NUL or a U+FFFD counts as not text, through every change: write, edit, delete and rename.
  • Not built: reported usage. "When process returns usage, the meter takes it and marks it reported" needs token counts the browser never gets (known bug 15). The meter takes characters only.
  • usage.modelRequests now counts every request sent, the retry after a pipeline restart included, and not a request the cost limit refused. C1 counted model calls.

Known limits

  • The output allowance is 8,192 tokens. The stand-in sets no output maximum, so the direction's fallback applies. The model node's own maximum for this profile is 128,000 tokens, so one long reply can cost more than its reservation; the estimate takes the reply's real length after it arrives.
  • The cache price is a guess. The browser cannot see a cache hit. A prefix under the provider's minimum, or a cache the provider dropped early, makes the estimate low.
  • An empty folder stays. A folder the agent made with create_directory stays after Undo, and in Manual mode its staged folder stays pending.
  • expect on the write-back never fires in practice. The second check in the same synchronous block sees any change in this page first. It stays, as the direction asks.
  • Check, then write in Auto (known bug 3).
  • A delete of a file with nothing staged reads the server first. It drops the page's cached copy and reads the server's, which the cache then holds. That is one server read per delete, and in Manual mode the view shows the server's copy from then on.
  • A conversation reopened while Undo runs waits to load. The tab shows "Loading session…" until Undo has checked, reversed and saved its row.
  • Thinking is not counted, so a turn can cost more than its estimate and more than the cost limit (see the estimate section).
  • No journey for the cost limit. Reaching $2 in a browser turn takes a long run; the unit tests cover it.

Follow-up after approval

The estimate left out thinking, and nothing said so. The stand-in's request has no thinking param: its model node sets one only when extendedThinking is on, and LangChain drops the unset field. Sonnet 5 then runs adaptive thinking, billed as output, with its text omitted from the reply, and the meter counts only the reply's text. The meta line now reads "~$0.42 est., not counting thinking", and the Agent Cost Limit setting says the estimate leaves out the model's thinking, so a turn can cost more than the limit. The two tests that check the meta text expect the new text, and each fails on the old line. The journeys above ran before this change; it changes only that text and the setting's description.

Proved with: 518 of 520 shared tests (2 todo for #2836's limits), all twelve ordering explorations with 0 broken rules, dirtyClose and the shell chat tests, the type-checks and test-file check, the rocket-ui build, and eleven browser journeys passing every check on an engine built from this branch, two of them new (step-cap, undo).

The stack

Eleven PRs, each on the one below it. Merge top-down: each PR squashes into the branch below it, starting with #2885 and ending with #2835 into feat/rocket-agent-browser. A top-down rehearsal in a scratch clone was clean at every step and ended on #2885's tree. Bottom-up also works if each next branch is first rebased onto the squashed base.

  1. fix(agent): let Wave read plan replies itself #2835 plan replies
  2. fix(agent): keep secrets out of the model in the browser agent #2836 secrets
  3. fix(agent): keep a run in its team, start nothing over a timed-out call, and ignore markers of deleted files #2839 team, timeouts and deleted files
  4. fix(agent): give every file a version, so the agent and an editor cannot write over each other #2840 versioned writes
  5. fix(agent): keep one document per open file and pipeline, so splits, closed tabs and saves stay in step #2882 editor documents and the rename fix
  6. feat(agent): ask before the agent changes a file with unsaved changes, and stop the tool clock while the user answers #2883 unsaved-changes dialog and tool clock
  7. fix(agent): check a staged file against the copy the page read, so Commit all meets a teammate's change #2884 manual-mode commit check
  8. feat(agent): end each turn with a result card, let the agent ask the user, and stop only its own runs in Auto #2869 turn result, needs you and run ownership
  9. feat(agent): pause a turn at its step or cost limit, show its estimated cost, and undo its file changes #2873 step and cost limits, cost estimate, Undo (this PR)
  10. feat(agent): check the browser agent end to end in a real browser, with a verify skill for each layer of the stack #2874 verify skill
  11. feat(agent): replay the browser agent's smoke scenarios in CI, from fresh recordings #2885 eval runner

@coderabbitai

coderabbitai Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review: 2
  • develop
  • release/.*

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added module:ui Chat UI and Dropper UI tests labels Oct 27, 2026
@github-actions

Copy link
Copy Markdown
Contributor
🤖 Internal: Discord sync marker

Auto-managed by the Discord notification workflow. Stores the linked Discord message ID and forum thread ID. Do not edit or delete.

@dylan-savage dylan-savage left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. I found no bugs in the limits, the meter or Undo, and the new suites pass locally.

Follow-up: the estimate leaves out thinking. The stand-in sends no thinking param, so Sonnet 5 thinks by default and bills it as output the browser never sees. That makes "~$X est." and the $2 cap lower bounds; worth saying so in the meta line and the setting text.

@krishhgg
krishhgg force-pushed the feat/rocket-agent-turn-contract branch from 4a7d660 to 8074fb3 Compare October 28, 2026 15:01
@krishhgg
krishhgg force-pushed the feat/rocket-agent-limits-undo branch from a31f363 to 9b087fd Compare October 28, 2026 15:01
krishhgg pushed a commit that referenced this pull request Oct 28, 2026
…th the rr-verify-agent skill

rr-verify-agent, in .claude/skills/ beside #2870's verify-rocketride, drives
Rocket Builder's Agent tab with Playwright against a real engine and model,
one journey per behavior this stack adds, and records proof: a screenshot
and an ARIA snapshot after each action, the files read back through the
engine, the chat, the console and the WebSocket frames the page sent. It
changes no product code.

- run takes the machine's port lock, starts the engine, rocket-ui and the
  shell, drives each journey, and stops only what it started. Its exit code
  is the worst result. launch, drive, doctor, engine and cleanup are the
  single steps.
- --engine source, the default, starts dist/server through #2870's
  launcher, with a store of the run's own. build-engine fills dist/server
  from the nearest server release and syncs the branch's engine Python.
  Without the launcher, launch falls back to the pinned image, says so, and
  names the checks that cannot pass there.
- It takes an engine as its own only by what its launch recorded, so stale
  state cannot reset or remove another checkout's engine or signal a
  process that reused a pid.
- Each journey resets the workspace, the sessions and the layout, seeds its
  fixtures through the engine, and signs in as a user does. Evidence goes to
  .verify/agent/ (git-ignored), with key names only, never values.
- Each check failed alone on a narrow mutation of the product code it
  guards. Checks no such mutation could fail alone were cut.
- The cost check matches #2873's meta line, which now says the estimate
  leaves out thinking.
- lint:scripts lets the skill's scripts import playwright from the skill's
  own package.json, which CI does not install. Every other import is still
  checked.

The eval runner, its data and shared:eval are in their own PR on top of
this one.
@krishhgg

Copy link
Copy Markdown
Collaborator Author

Done. The meta line reads "~$X est., not counting thinking", and the Cost Limit setting says the limit leaves out the model's thinking, so a turn can cost more. I checked first: the stand-in's config has no extendedThinking, LangChain drops the unset thinking field, and the meter counts only the reply text.

@krishhgg
krishhgg force-pushed the feat/rocket-agent-turn-contract branch from 8074fb3 to 791c40f Compare October 28, 2026 19:22
@krishhgg
krishhgg force-pushed the feat/rocket-agent-limits-undo branch from 9b087fd to 545b01f Compare October 28, 2026 19:22
krishhgg pushed a commit that referenced this pull request Oct 28, 2026
…th the rr-verify-agent skill

rr-verify-agent, in .claude/skills/ beside #2870's verify-rocketride, drives
Rocket Builder's Agent tab with Playwright against a real engine and model,
one journey per behavior this stack adds, and records proof: a screenshot
and an ARIA snapshot after each action, the files read back through the
engine, the chat, the console and the WebSocket frames the page sent. It
changes no product code.

- run takes the machine's port lock, starts the engine, rocket-ui and the
  shell, drives each journey, and stops only what it started. Its exit code
  is the worst result. launch, drive, doctor, engine and cleanup are the
  single steps.
- --engine source, the default, starts dist/server through #2870's
  launcher, with a store of the run's own. build-engine fills dist/server
  from the nearest server release and syncs the branch's engine Python.
  Without the launcher, launch falls back to the pinned image, says so, and
  names the checks that cannot pass there.
- It takes an engine as its own only by what its launch recorded, so stale
  state cannot reset or remove another checkout's engine or signal a
  process that reused a pid.
- Each journey resets the workspace, the sessions and the layout, seeds its
  fixtures through the engine, and signs in as a user does. Evidence goes to
  .verify/agent/ (git-ignored), with key names only, never values.
- Each check failed alone on a narrow mutation of the product code it
  guards. Checks no such mutation could fail alone were cut.
- The cost check matches #2873's meta line, which now says the estimate
  leaves out thinking.
- lint:scripts lets the skill's scripts import playwright from the skill's
  own package.json, which CI does not install. Every other import is still
  checked.

The eval runner, its data and shared:eval are in their own PR on top of
this one.
Krish Garg added 2 commits October 28, 2026 20:50
…ed cost, and undo its file changes

- A turn takes at most 60 steps (waves) by default. At the limit the loop
  plans no more, answers with the best-effort summary, and the turn ends as
  needs_input with a continue request: the card says "Paused after 60 steps"
  and Continue starts a new turn with a fresh budget. Messages sent during
  the turn do not reset the count. Agent Step Limit in Settings changes it,
  and 0 turns it off.
- A UsageMeter estimates each model request from characters, 4 to a token,
  priced from a dated table with one row for the stand-in's model
  (claude-sonnet-5, Anthropic's price page, read 2026-10-27). The prompt
  start is priced at the cache-write rate on a turn's first request, at the
  cache-read rate after, and as a write again after a gap over five minutes,
  only for a provider that caches it and a question that asks.
- Before every model request, the retry after a pipeline restart and the
  best-effort summary included, the meter reserves the input estimate and
  an output allowance. A request that could pass the turn's cost limit is
  not sent, and the turn pauses with Continue. A failed request keeps its
  reservation. The limit is $2 per turn by default (Agent Cost Limit, 0
  turns it off); a model with no price shows tokens and has no limit.
- The bot row's meta line shows the estimate ("~$0.42 est."), and
  TurnResult.usage carries the tokens and the cost, marked estimated.
- The latest turn's card has Undo for that turn's file changes. The file
  toolkit records, in memory only, each path's content at its first touch
  and the version and hash each agent change left. Undo checks every path
  first: no editor holds unsaved changes, the version is the agent's, no one
  else changed it between agent changes, and the stored content matches what
  the agent left (the server's copy afresh, or the staged content for a path
  still staged). Then every inverse starts in one block, through the
  versioned write. In manual mode a file the agent created and never
  committed has its staged entry dropped, so nothing stays pending. A note
  row says what Undo did, and the model reads it in the next turn.
…hinking

The stand-in model request sends no thinking param, so Sonnet 5 thinks by
default and bills it as output the browser never sees. The estimate counts
only the reply text, so "~$X est." and the cost limit are lower bounds. The
row's meta line now reads "~$X est., not counting thinking", and the cost
limit setting says a turn can cost more than the limit.
@krishhgg
krishhgg force-pushed the feat/rocket-agent-turn-contract branch from 791c40f to 4df0253 Compare October 28, 2026 20:52
@krishhgg
krishhgg force-pushed the feat/rocket-agent-limits-undo branch from 545b01f to 09abac0 Compare October 28, 2026 20:52
krishhgg pushed a commit that referenced this pull request Oct 28, 2026
…th the rr-verify-agent skill

rr-verify-agent, in .claude/skills/ beside #2870's verify-rocketride, drives
Rocket Builder's Agent tab with Playwright against a real engine and model,
one journey per behavior this stack adds, and records proof: a screenshot
and an ARIA snapshot after each action, the files read back through the
engine, the chat, the console and the WebSocket frames the page sent. It
changes no product code.

- run takes the machine's port lock, starts the engine, rocket-ui and the
  shell, drives each journey, and stops only what it started. Its exit code
  is the worst result. launch, drive, doctor, engine and cleanup are the
  single steps.
- --engine source, the default, starts dist/server through #2870's
  launcher, with a store of the run's own. build-engine fills dist/server
  from the nearest server release and syncs the branch's engine Python.
  Without the launcher, launch falls back to the pinned image, says so, and
  names the checks that cannot pass there.
- It takes an engine as its own only by what its launch recorded, so stale
  state cannot reset or remove another checkout's engine or signal a
  process that reused a pid.
- Each journey resets the workspace, the sessions and the layout, seeds its
  fixtures through the engine, and signs in as a user does. Evidence goes to
  .verify/agent/ (git-ignored), with key names only, never values.
- Each check failed alone on a narrow mutation of the product code it
  guards. Checks no such mutation could fail alone were cut.
- The cost check matches #2873's meta line, which now says the estimate
  leaves out thinking.
- lint:scripts lets the skill's scripts import playwright from the skill's
  own package.json, which CI does not install. Every other import is still
  checked.

The eval runner, its data and shared:eval are in their own PR on top of
this one.
@dylan-savage
dylan-savage merged commit c4b698b into feat/rocket-agent-turn-contract Oct 28, 2026
4 checks passed
@dylan-savage
dylan-savage deleted the feat/rocket-agent-limits-undo branch October 28, 2026 23:53
dylan-savage pushed a commit that referenced this pull request Oct 29, 2026
…user, and stop only its own runs in Auto (#2869)

* refactor(agent): move FileTracker changes() here from the save-race PR

The turn's result card lists what the run changed, so FileTracker records
each file's change (added, modified, deleted or renamed, and where a
renamed file came from) and changes() lists them. A file the run created
and then deleted is no change. A deleted file keeps its version, so a
file the run only deleted is not read back as seen. The stored copy a
delete is checked against stays with each read, write and rename.

* feat(agent): end each turn with a result card, let the agent ask the user, and stop only its own runs in Auto

- runAgent, agent() and AgentSession.execute return a TurnResult: the
  outcome (done, needs_input, stopped, failed, incomplete), the files and
  checks, the usage and any needs-you request. Every turn ends once through
  execute, including a refused message, an error, Stop and an unmount.
- The session file is version 2. It keeps the turns, linked to the rows that
  close them, and the runs the conversation started. A version 1 file loads
  with neither. Every saved field passes through redactText.
- The shell's ChatMessage gains an optional `card`, and ChatView gains
  `onCardAction`. The Agent tab puts each turn's card under its row: links to
  the changed files, Open Variables, a button per choice, and Continue.
- ask_user ends the turn as needs_input, as a question or a missing
  Variable. Only the user's next message answers the request.
- terminate and restart ask first in Ask mode. In Auto they run only on a run
  this conversation started with use, after a fresh getTaskStatus matches the
  record and the scope. Any other run is refused, and the model is told to
  ask the user.
- A turn running when the Agent tab unmounts is saved with its rows and a
  closing row (known bug 5).
- A use refused because the pipeline is already running says how to run the
  edited version (known bug 14).

* fix(agent): answer a needs-you request only after it showed, document the row card, and check the run on the Ask card

A turn the user stopped in the wave that called ask_user kept its request.
Its card said only "Stopped", yet the next message answered the question and
disabled its buttons. Only a needs_input turn keeps its request now.

A message the user sent during the wave that called ask_user stayed queued,
then ran as its own turn and answered the question the user had not seen.
The loop now takes such a message before it ends the turn. It drops the
question, and the model reads the message and may ask again.

ChatMessage.card and the onCardAction prop of ChatView and MessageList are
documented in ROCKETRIDE_UI_COMPONENTS.md and the ChatView gallery entry.

The Ask card for terminate and restart said "This conversation started it."
from the run record alone. The engine gives a later run on the same project
and source the same token, so the card could claim the user's run. It now
reads the run's status first, as the Auto run step does.

* feat(agent): pause a turn at its step or cost limit, show its estimated cost, and undo its file changes (#2873)

* feat(agent): pause a turn at its step or cost limit, show its estimated cost, and undo its file changes

- A turn takes at most 60 steps (waves) by default. At the limit the loop
  plans no more, answers with the best-effort summary, and the turn ends as
  needs_input with a continue request: the card says "Paused after 60 steps"
  and Continue starts a new turn with a fresh budget. Messages sent during
  the turn do not reset the count. Agent Step Limit in Settings changes it,
  and 0 turns it off.
- A UsageMeter estimates each model request from characters, 4 to a token,
  priced from a dated table with one row for the stand-in's model
  (claude-sonnet-5, Anthropic's price page, read 2026-10-27). The prompt
  start is priced at the cache-write rate on a turn's first request, at the
  cache-read rate after, and as a write again after a gap over five minutes,
  only for a provider that caches it and a question that asks.
- Before every model request, the retry after a pipeline restart and the
  best-effort summary included, the meter reserves the input estimate and
  an output allowance. A request that could pass the turn's cost limit is
  not sent, and the turn pauses with Continue. A failed request keeps its
  reservation. The limit is $2 per turn by default (Agent Cost Limit, 0
  turns it off); a model with no price shows tokens and has no limit.
- The bot row's meta line shows the estimate ("~$0.42 est."), and
  TurnResult.usage carries the tokens and the cost, marked estimated.
- The latest turn's card has Undo for that turn's file changes. The file
  toolkit records, in memory only, each path's content at its first touch
  and the version and hash each agent change left. Undo checks every path
  first: no editor holds unsaved changes, the version is the agent's, no one
  else changed it between agent changes, and the stored content matches what
  the agent left (the server's copy afresh, or the staged content for a path
  still staged). Then every inverse starts in one block, through the
  versioned write. In manual mode a file the agent created and never
  committed has its staged entry dropped, so nothing stays pending. A note
  row says what Undo did, and the model reads it in the next turn.

* fix(agent): say that the cost estimate and the cost limit leave out thinking

The stand-in model request sends no thinking param, so Sonnet 5 thinks by
default and bills it as output the browser never sees. The estimate counts
only the reply text, so "~$X est." and the cost limit are lower bounds. The
row's meta line now reads "~$X est., not counting thinking", and the cost
limit setting says a turn can cost more than the limit.

---------

Co-authored-by: Krish Garg <kg@g14-node1.tailb99b1b.ts.net>

---------

Co-authored-by: Krish Garg <kg@g14-node1.tailb99b1b.ts.net>
dylan-savage pushed a commit that referenced this pull request Oct 29, 2026
…mmit all meets a teammate's change (#2884)

* fix(agent): check a staged file against the copy the page read, so Commit all meets a teammate's change

In manual mode a staged file's conflict anchor was a fresh server read
taken when it was first staged. A teammate's change from before then became
the anchor, so Commit all wrote over it.

The anchor is now the copy the page read, so the commit meets a conflict.
The view's prime records the path only: the dev preview primed what its
batch read returned, unsaved typing included, and the cache then served it
as the server's copy. The preview and teammate exploration covers both.

Going back to a team drops the page's cached copies, so a delete the agent
decided on before a switch away and back anchored on the server's copy
after it, and Commit all deleted a teammate's change. The view's delete
now takes an anchor, the hash of the stored copy the delete is based on.
delete_file passes the copy the agent last read or wrote. Under unsaved
typing that is the stored copy below the typing, which Discard mine brings
back. The view checks the anchor in the delete's turn, against the same
capture its tombstone records, so no copy read later can become the
delete's anchor. In write-through mode it checks the server's copy.

* feat(agent): end each turn with a result card, let the agent ask the user, and stop only its own runs in Auto (#2869)

* refactor(agent): move FileTracker changes() here from the save-race PR

The turn's result card lists what the run changed, so FileTracker records
each file's change (added, modified, deleted or renamed, and where a
renamed file came from) and changes() lists them. A file the run created
and then deleted is no change. A deleted file keeps its version, so a
file the run only deleted is not read back as seen. The stored copy a
delete is checked against stays with each read, write and rename.

* feat(agent): end each turn with a result card, let the agent ask the user, and stop only its own runs in Auto

- runAgent, agent() and AgentSession.execute return a TurnResult: the
  outcome (done, needs_input, stopped, failed, incomplete), the files and
  checks, the usage and any needs-you request. Every turn ends once through
  execute, including a refused message, an error, Stop and an unmount.
- The session file is version 2. It keeps the turns, linked to the rows that
  close them, and the runs the conversation started. A version 1 file loads
  with neither. Every saved field passes through redactText.
- The shell's ChatMessage gains an optional `card`, and ChatView gains
  `onCardAction`. The Agent tab puts each turn's card under its row: links to
  the changed files, Open Variables, a button per choice, and Continue.
- ask_user ends the turn as needs_input, as a question or a missing
  Variable. Only the user's next message answers the request.
- terminate and restart ask first in Ask mode. In Auto they run only on a run
  this conversation started with use, after a fresh getTaskStatus matches the
  record and the scope. Any other run is refused, and the model is told to
  ask the user.
- A turn running when the Agent tab unmounts is saved with its rows and a
  closing row (known bug 5).
- A use refused because the pipeline is already running says how to run the
  edited version (known bug 14).

* fix(agent): answer a needs-you request only after it showed, document the row card, and check the run on the Ask card

A turn the user stopped in the wave that called ask_user kept its request.
Its card said only "Stopped", yet the next message answered the question and
disabled its buttons. Only a needs_input turn keeps its request now.

A message the user sent during the wave that called ask_user stayed queued,
then ran as its own turn and answered the question the user had not seen.
The loop now takes such a message before it ends the turn. It drops the
question, and the model reads the message and may ask again.

ChatMessage.card and the onCardAction prop of ChatView and MessageList are
documented in ROCKETRIDE_UI_COMPONENTS.md and the ChatView gallery entry.

The Ask card for terminate and restart said "This conversation started it."
from the run record alone. The engine gives a later run on the same project
and source the same token, so the card could claim the user's run. It now
reads the run's status first, as the Auto run step does.

* feat(agent): pause a turn at its step or cost limit, show its estimated cost, and undo its file changes (#2873)

* feat(agent): pause a turn at its step or cost limit, show its estimated cost, and undo its file changes

- A turn takes at most 60 steps (waves) by default. At the limit the loop
  plans no more, answers with the best-effort summary, and the turn ends as
  needs_input with a continue request: the card says "Paused after 60 steps"
  and Continue starts a new turn with a fresh budget. Messages sent during
  the turn do not reset the count. Agent Step Limit in Settings changes it,
  and 0 turns it off.
- A UsageMeter estimates each model request from characters, 4 to a token,
  priced from a dated table with one row for the stand-in's model
  (claude-sonnet-5, Anthropic's price page, read 2026-10-27). The prompt
  start is priced at the cache-write rate on a turn's first request, at the
  cache-read rate after, and as a write again after a gap over five minutes,
  only for a provider that caches it and a question that asks.
- Before every model request, the retry after a pipeline restart and the
  best-effort summary included, the meter reserves the input estimate and
  an output allowance. A request that could pass the turn's cost limit is
  not sent, and the turn pauses with Continue. A failed request keeps its
  reservation. The limit is $2 per turn by default (Agent Cost Limit, 0
  turns it off); a model with no price shows tokens and has no limit.
- The bot row's meta line shows the estimate ("~$0.42 est."), and
  TurnResult.usage carries the tokens and the cost, marked estimated.
- The latest turn's card has Undo for that turn's file changes. The file
  toolkit records, in memory only, each path's content at its first touch
  and the version and hash each agent change left. Undo checks every path
  first: no editor holds unsaved changes, the version is the agent's, no one
  else changed it between agent changes, and the stored content matches what
  the agent left (the server's copy afresh, or the staged content for a path
  still staged). Then every inverse starts in one block, through the
  versioned write. In manual mode a file the agent created and never
  committed has its staged entry dropped, so nothing stays pending. A note
  row says what Undo did, and the model reads it in the next turn.

* fix(agent): say that the cost estimate and the cost limit leave out thinking

The stand-in model request sends no thinking param, so Sonnet 5 thinks by
default and bills it as output the browser never sees. The estimate counts
only the reply text, so "~$X est." and the cost limit are lower bounds. The
row's meta line now reads "~$X est., not counting thinking", and the cost
limit setting says a turn can cost more than the limit.

---------

Co-authored-by: Krish Garg <kg@g14-node1.tailb99b1b.ts.net>

---------

Co-authored-by: Krish Garg <kg@g14-node1.tailb99b1b.ts.net>

---------

Co-authored-by: Krish Garg <kg@g14-node1.tailb99b1b.ts.net>
dylan-savage pushed a commit that referenced this pull request Oct 29, 2026
…, and stop the tool clock while the user answers (#2883)

* feat(agent): ask before the agent changes a file with unsaved changes, and stop the tool clock while the user answers

When the agent wanted to change a file the user had unsaved changes in,
the call was refused and the model told the user to save first.

Now the user is asked: Save mine first, Discard mine or Keep mine, one
dialog at a time. The open editors do what the user chose. A call's time
limit stands still while the question is open, in its prepare and in its
run. A call its tool refused before changing anything is unstarted, so the
finish rule asks for no check. One in which the user's save landed or
sent a write still counts as a change, even when that save or another
editor's save then failed, since the file may have changed. A save that
joins a Ctrl+S already on its way counts once it lands. Open questions
close as Keep mine when the run ends or the user stops it.

* test(agent): count a tab behind only by a failed write as the known limit, also after a later read

With versioned writes, a read that takes a teammate's copy in counts it for
the editor too. When that read waited for a write that then failed, the
failure's count lands before the read and the editor misses only that one,
while the file moves past the failure's version. The text tab exploration
broke its not-stored rule in 3 such orders, where the tab shows the stored
file. They now count under the known limit for a failed write, as the same
order without the teammate's change already does.

* fix(agent): show the unsaved-changes question below 1024 px, where the closed drawer is a hidden dialog

Below 1024 px the shell's sidebar is a drawer. It stays in the page as a
modal dialog, with `visibility: hidden` while it is closed. otherDialogOpen()
counted it, so every question resolved unasked, and the agent was told to try
again after the user closed a dialog they could not see.

otherDialogOpen() now counts only dialogs that
checkVisibility({ visibilityProperty: true }) reports visible. The call also
passes the option's old name, checkVisibilityCSS, which Chrome 105 to 120
know instead. An open drawer still blocks the question. A browser without
checkVisibility counts every dialog, as before, so a question never opens
over another dialog.

* fix(agent): check a staged file against the copy the page read, so Commit all meets a teammate's change (#2884)

* fix(agent): check a staged file against the copy the page read, so Commit all meets a teammate's change

In manual mode a staged file's conflict anchor was a fresh server read
taken when it was first staged. A teammate's change from before then became
the anchor, so Commit all wrote over it.

The anchor is now the copy the page read, so the commit meets a conflict.
The view's prime records the path only: the dev preview primed what its
batch read returned, unsaved typing included, and the cache then served it
as the server's copy. The preview and teammate exploration covers both.

Going back to a team drops the page's cached copies, so a delete the agent
decided on before a switch away and back anchored on the server's copy
after it, and Commit all deleted a teammate's change. The view's delete
now takes an anchor, the hash of the stored copy the delete is based on.
delete_file passes the copy the agent last read or wrote. Under unsaved
typing that is the stored copy below the typing, which Discard mine brings
back. The view checks the anchor in the delete's turn, against the same
capture its tombstone records, so no copy read later can become the
delete's anchor. In write-through mode it checks the server's copy.

* feat(agent): end each turn with a result card, let the agent ask the user, and stop only its own runs in Auto (#2869)

* refactor(agent): move FileTracker changes() here from the save-race PR

The turn's result card lists what the run changed, so FileTracker records
each file's change (added, modified, deleted or renamed, and where a
renamed file came from) and changes() lists them. A file the run created
and then deleted is no change. A deleted file keeps its version, so a
file the run only deleted is not read back as seen. The stored copy a
delete is checked against stays with each read, write and rename.

* feat(agent): end each turn with a result card, let the agent ask the user, and stop only its own runs in Auto

- runAgent, agent() and AgentSession.execute return a TurnResult: the
  outcome (done, needs_input, stopped, failed, incomplete), the files and
  checks, the usage and any needs-you request. Every turn ends once through
  execute, including a refused message, an error, Stop and an unmount.
- The session file is version 2. It keeps the turns, linked to the rows that
  close them, and the runs the conversation started. A version 1 file loads
  with neither. Every saved field passes through redactText.
- The shell's ChatMessage gains an optional `card`, and ChatView gains
  `onCardAction`. The Agent tab puts each turn's card under its row: links to
  the changed files, Open Variables, a button per choice, and Continue.
- ask_user ends the turn as needs_input, as a question or a missing
  Variable. Only the user's next message answers the request.
- terminate and restart ask first in Ask mode. In Auto they run only on a run
  this conversation started with use, after a fresh getTaskStatus matches the
  record and the scope. Any other run is refused, and the model is told to
  ask the user.
- A turn running when the Agent tab unmounts is saved with its rows and a
  closing row (known bug 5).
- A use refused because the pipeline is already running says how to run the
  edited version (known bug 14).

* fix(agent): answer a needs-you request only after it showed, document the row card, and check the run on the Ask card

A turn the user stopped in the wave that called ask_user kept its request.
Its card said only "Stopped", yet the next message answered the question and
disabled its buttons. Only a needs_input turn keeps its request now.

A message the user sent during the wave that called ask_user stayed queued,
then ran as its own turn and answered the question the user had not seen.
The loop now takes such a message before it ends the turn. It drops the
question, and the model reads the message and may ask again.

ChatMessage.card and the onCardAction prop of ChatView and MessageList are
documented in ROCKETRIDE_UI_COMPONENTS.md and the ChatView gallery entry.

The Ask card for terminate and restart said "This conversation started it."
from the run record alone. The engine gives a later run on the same project
and source the same token, so the card could claim the user's run. It now
reads the run's status first, as the Auto run step does.

* feat(agent): pause a turn at its step or cost limit, show its estimated cost, and undo its file changes (#2873)

* feat(agent): pause a turn at its step or cost limit, show its estimated cost, and undo its file changes

- A turn takes at most 60 steps (waves) by default. At the limit the loop
  plans no more, answers with the best-effort summary, and the turn ends as
  needs_input with a continue request: the card says "Paused after 60 steps"
  and Continue starts a new turn with a fresh budget. Messages sent during
  the turn do not reset the count. Agent Step Limit in Settings changes it,
  and 0 turns it off.
- A UsageMeter estimates each model request from characters, 4 to a token,
  priced from a dated table with one row for the stand-in's model
  (claude-sonnet-5, Anthropic's price page, read 2026-10-27). The prompt
  start is priced at the cache-write rate on a turn's first request, at the
  cache-read rate after, and as a write again after a gap over five minutes,
  only for a provider that caches it and a question that asks.
- Before every model request, the retry after a pipeline restart and the
  best-effort summary included, the meter reserves the input estimate and
  an output allowance. A request that could pass the turn's cost limit is
  not sent, and the turn pauses with Continue. A failed request keeps its
  reservation. The limit is $2 per turn by default (Agent Cost Limit, 0
  turns it off); a model with no price shows tokens and has no limit.
- The bot row's meta line shows the estimate ("~$0.42 est."), and
  TurnResult.usage carries the tokens and the cost, marked estimated.
- The latest turn's card has Undo for that turn's file changes. The file
  toolkit records, in memory only, each path's content at its first touch
  and the version and hash each agent change left. Undo checks every path
  first: no editor holds unsaved changes, the version is the agent's, no one
  else changed it between agent changes, and the stored content matches what
  the agent left (the server's copy afresh, or the staged content for a path
  still staged). Then every inverse starts in one block, through the
  versioned write. In manual mode a file the agent created and never
  committed has its staged entry dropped, so nothing stays pending. A note
  row says what Undo did, and the model reads it in the next turn.

* fix(agent): say that the cost estimate and the cost limit leave out thinking

The stand-in model request sends no thinking param, so Sonnet 5 thinks by
default and bills it as output the browser never sees. The estimate counts
only the reply text, so "~$X est." and the cost limit are lower bounds. The
row's meta line now reads "~$X est., not counting thinking", and the cost
limit setting says a turn can cost more than the limit.

---------

Co-authored-by: Krish Garg <kg@g14-node1.tailb99b1b.ts.net>

---------

Co-authored-by: Krish Garg <kg@g14-node1.tailb99b1b.ts.net>

---------

Co-authored-by: Krish Garg <kg@g14-node1.tailb99b1b.ts.net>

---------

Co-authored-by: Krish Garg <kg@g14-node1.tailb99b1b.ts.net>
krishhgg pushed a commit that referenced this pull request Oct 29, 2026
…th the rr-verify-agent skill

rr-verify-agent, in .claude/skills/ beside #2870's verify-rocketride, drives
Rocket Builder's Agent tab with Playwright against a real engine and model,
one journey per behavior this stack adds, and records proof: a screenshot
and an ARIA snapshot after each action, the files read back through the
engine, the chat, the console and the WebSocket frames the page sent. It
changes no product code.

- run takes the machine's port lock, starts the engine, rocket-ui and the
  shell, drives each journey, and stops only what it started. Its exit code
  is the worst result. launch, drive, doctor, engine and cleanup are the
  single steps.
- --engine source, the default, starts dist/server through #2870's
  launcher, with a store of the run's own. build-engine fills dist/server
  from the nearest server release and syncs the branch's engine Python.
  Without the launcher, launch falls back to the pinned image, says so, and
  names the checks that cannot pass there.
- It takes an engine as its own only by what its launch recorded, so stale
  state cannot reset or remove another checkout's engine or signal a
  process that reused a pid.
- Each journey resets the workspace, the sessions and the layout, seeds its
  fixtures through the engine, and signs in as a user does. Evidence goes to
  .verify/agent/ (git-ignored), with key names only, never values.
- Each check failed alone on a narrow mutation of the product code it
  guards. Checks no such mutation could fail alone were cut.
- The cost check matches #2873's meta line, which now says the estimate
  leaves out thinking.
- lint:scripts lets the skill's scripts import playwright from the skill's
  own package.json, which CI does not install. Every other import is still
  checked.

The eval runner, its data and shared:eval are in their own PR on top of
this one.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

module:ui Chat UI and Dropper UI tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants