Repository navigation
fix(agent): keep a run in its team, start nothing over a timed-out call, and ignore markers of deleted files - #2839
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. 🗂️ Base branches to auto review: 2
Please check the settings in the CodeRabbit UI or the
|
🤖 Internal: Discord sync markerAuto-managed by the Discord notification workflow. Stores the linked Discord message ID and forum thread ID. Do not edit or delete. |
afa656a to
72c1e51
Compare
dylan-savage
left a comment
There was a problem hiding this comment.
Tests are 76% of this PR (1,875 lines for 337 lines of code), and several interleavings are tested twice: by the exhaustive runs in ordering.test.ts, then by hand in workspace.test.ts and loop.test.ts (the write in flight at a switch, the timed-out edit in the real workspace, the loop cases that pair an approval with a timeout). Keep one test per behaviour: the exploration for orders of switches, timeouts and approvals, and hand-written tests only for what it can't reach. Share one fake store between the two vfs test files and aim for about 1,200 test lines; doing this once at the top of the stack is fine if restacking costs too much.
Not blocking: land this only together with #2840. Until its versioned writes are in, the CachedVfs.fill change lets a stale read pass the write-through freshness gate, so one save can erase another. Separately, nothing holds a timed-out call that outlives its run against the next turn's edits.
4602f60 to
3c9e88f
Compare
72c1e51 to
6dfcb2c
Compare
|
Tests are down from +1,863 lines to +1,208, and from 42 tests to 23. I reverted each of the PR's fix lines on its own and kept a hand-written test only where it catches a revert the explorations miss. Every duplicate you named is gone. The revert pass also found a real bug. A call that ended while On your notes: this lands together with #2840, since the stack merges in one go. The timed-out call that outlives its run is on our follow-up list. |
dylan-savage
left a comment
There was a problem hiding this comment.
Approving. The trim is done, and both new fixes check out: unfinished() counting every unreported late call, and late results hidden like other tool results. Reverting either one fails its test.
6dfcb2c to
651bef5
Compare
…ll, and ignore markers of deleted files A tool call still running when the team changes lands in the team it started in. A turn in another team's conversation is refused before anything runs. A call on the same files as a timed-out call that still runs is not started, and the model hears when that call ends. A late result is stored under its memory key redacted, as the model reads it. check_app ignores type errors of TypeScript files that no longer exist, matching files by Monaco's own model uri. Going back to a team reuses its workspace only if no other tab changed its staged state, and an old read cannot refill its cache. A listing save reads and writes the app's package in the workspace it started in. The dev session disposes a deleted file's model once the view confirms the deletion, and keeps it when a read only failed.
…t per behaviour, and cut comments to the why The ordering explorations cover the orders of team switches, timeouts and approvals. Each hand-written test left guards a fix line no other kept test or exploration fails on when that line is reverted. Cut: the in-flight write at a switch, the timed-out edit in the real workspace, the loop tests that pair an approval with a timeout, and tests that pass on the base. ordering.test.ts and workspace.test.ts now share FakeStore (fakeStore.harness.ts). One cache test covers every change fill() checks. unfinished() counts every call in late again, as on the base. A timed-out call that ended while endWave ran, after lateReturns() took the calls that had ended, let a done stand before the model read how it ended. Where a revert passed every kept test, a test now fails on it: a deleted file's markers let check_app pass, a done with calls waits for a late result, going back to a team reads its files and file list again, and a dev session reads its package, lock and icons and writes its lock in the team it opened in after a switch. A reopened workspace is no longer written back to builtByScope. The next reopen gets the same workspace from the first entry.
A call that timed out reaches the model as a late entry once it ends. Its result and error went through redactText only, so a value a read had hidden showed in full there, while every other tool result shows its marker. Late returns now go through the same hiding as an outcome, values first, then key shapes.
651bef5 to
91b5fb4
Compare
…not write over each other (#2840) * fix(agent): give every file a version, so the agent and an editor cannot write over each other The agent read a file, the user saved it, and the agent's edit then landed on the text it had read, so the save was gone. In manual mode Commit all, the discards, a rollback and Undo also raced the writes called just before them. Every path in the staged view now has a version. A mutation counts when it is called and then waits behind the path's earlier ones; a read waits for them too, up to 60 s. A delete or rename also counts every path under it, at both ends of a rename. A write that names an older version is refused with StaleVersionError. Commit all, Undo, the discards and a rollback run in call order, and a commit carries only what was staged before it was called. Undo counts and holds its paths from its call, also those a commit called before it retains only once it runs. A checkpoint runs after the writes called before it and holds the ones called after it. A server read that one of the view's own changes overtook is read again. A switch to auto mode is refused while a staged save is on its way, as it is while changes are staged. A staged write to a folder, or a rename onto one, is refused before it stages anything. A staged rename stages its delete and its write together and saves them in one store write, so the view never keeps the delete without the write. An invalidate keeps the cached copy it drops. A read that then takes in a different server copy, or finds one a batch read cached, counts a version, so an editor that loaded the old copy cannot save over a teammate's change, and until a read does, the dropped copy guards the next save. The agent's tools read and write at a version: write_file refuses a file that changed since the read, edit_file applies to a changed file only when its anchor still matches once, and each wave tells the model which files someone else changed. The ordering harness gets the auto run with an editor save, a teammate's change and a Reload, and three manual runs without the dialog. * fix(agent): count a save or Commit all once when the cached copy is dropped while its write is on its way Going back to a team, or the agent's refused write, drops the cached copy. When an editor's save or Commit all's write was still on its way, the read after it took the write's own content in as a changed copy and counted a second version. The editor's next save was then refused as changed on the server, and a canvas save after a Reload was too. The dropped copy now becomes the written content once the write lands, for a write-through save and for each file Commit all writes. * fix(agent): keep one document per open file and pipeline, so splits, closed tabs and saves stay in step (#2882) Two splits of one file kept their own text and version, so a save from one left the other behind and its next save was refused or wrote over the first. A closed tab's unsaved text came back stale, and Ctrl+S and the close dialog's Save skipped the version check. A text file's text, version and save state now belong to a FileDocument that every tab of the path shares and that outlives its last tab while the text is unsaved. A pipeline's belong to a CanvasDocument that lives while Documents holds it. Each document queues its saves; Reload, Overwrite and closing the last tab drop the saves still waiting, which reject with SaveDroppedError, so Save & Run and a publish stop. A refused save offers Reload or Overwrite. The editors register their unsaved typing with the view, so the agent's dirty guard and a rename see it: a rename over unsaved changes is refused, also when it waited its turn behind a save or Commit all, and it reads what the view stores. A rename from the sidebar or the explorer closes the tabs of the file, or of the files under a folder, while it runs and opens them on the new paths, so no typing lands on an old path. Documents keeps typing that lands during a read or a write. Co-authored-by: Krish Garg <krish.garg@rocketride.ai> --------- Co-authored-by: Krish Garg <kg@g14-node1.tailb99b1b.ts.net> Co-authored-by: Krish Garg <krish.garg@rocketride.ai>
Base:
feat/rocket-agent-secrets(#2836). This is the third of eleven stacked PRs: #2835, #2836, this one, the four PRs split from #2840 (versioned writes, editor documents, the unsaved-changes dialog, the commit check), then the turn contract, limits and Undo, testing, and the eval runner. Each piece was redone on Rod's workspace fs layer, Dylan's stop work, #2853's tool approval, the read-only.rocketride/guides and #2836's redaction, not cherry-picked.Team switch mid-run
The bug. rocket-ui's workspace backend translated every path with the module-level
activeScope, which Rod'sonMutateannouncement also reads. A write in flight during a team switch landed in the new team and overwrote its file.The fix.
createWorkspaceBackend(scope)and the staged view'sonMutatekeep their own scope.activeScopestays only for raw binary reads and bus adoption.createRocketAgentHost(client, ws)binds the host to one workspace and passes it toscaffoldAppandpackAppSource, which default to the current one. After a switch, the host'sdevSessionreturns null.scopeKey. A turn in another team's conversation is refused: "This conversation belongs to team Acme; switch back to continue it."package.jsonin the workspace it started in (bug 6, below).Late writes of the old run stage in the old team. The turn ends as
stopped, since the bootstrap cleanup unmounts the agent tabs.Timeout reorder
The bug. When a tool call timed out, the loop started the next call on the same file while the first still ran. The first could land last and erase the second.
The fix.
runWavedoes not start a call on the same files as a call that timed out and still runs, from this wave or an earlier one, or as an earlier call in the wave that was held back that way. It gets "Not run: it waits for edit_file on apps/a.ts, which timed out and is still running." The held-back call is never prepared, so no approval card asks about it. Its outcome isunstarted, #2853's flag for a call whose work never began, so the finish record never hears of it. A call whose prepare failed or timed out, or whose act the user refused, holds nothing back, since nothing of it can land. When the timed-out call ends, the model gets a late entry with its result, and the result is stored under the entry's memory key.Delete stuck
The bug. Monaco keeps a deleted file's model and markers, so check_app kept failing on a file the agent had deleted.
The fix.
appTypeEvidencedrops markers on TypeScript files missing from the live file list and says "Ignored 1 marker for a file that no longer exists: src/unused.ts."On top of #2836
#2836 and this PR both change
agent.ts,loop.ts,toolkits.test.tsandloop.test.ts. The restack keeps both behaviors.wave-N.lateImemory key and puts a summary of it in the late entry. Both are built from the outcome that fix(agent): keep secrets out of the model in the browser agent #2836'strackLatehas already passed throughredactText, so neither holds a key. Test:a call that times out and ends before the rest of its wave still gets its late entry, redacted and stored under its key. The late entry's summary andmemory.peekof its key both show[redacted].execute. The scope rule runs first. Then come fix(agent): keep secrets out of the model in the browser agent #2836's redacted title, this PR's scope stamp for a conversation saved before scopes, and fix(agent): keep secrets out of the model in the browser agent #2836's pasted-key refusal. A key sent in another team's conversation gets the team refusal. A key sent in a conversation saved before scopes still records its team, then is refused. Neither writes the key anywhere. Test:a conversation from another team is refused, and one saved before scopes records its team before the model or a pasted key is reached.save. The session file has fix(agent): keep secrets out of the model in the browser agent #2836's redacted rows and title, and this PR'sscopeKey.executereads the host's scope first.redactOutcomeon every outcome,redactCard, the redaction inhearandAgentSession.screenare unchanged. A held-back call's "Not run" error passes throughredactOutcomelike any other outcome.Folded in
Bug 6: a listing save could write one team's package into another.
saveListinginwebHost.tsread the app'spackage.json, then wrote it through whatever workspace was current. With a slow read and a team switch in between, team A's package landed in team B, staged there in manual mode. The save now takes the current workspace when it starts and reads and writes through that view.writeAppPackagetakes a view, asreadAppPackagealready did. Test:a listing save whose read ends after the team changes writes the package in the team it started in, in manual mode. It fails on the oldsaveListing.Bug 4: a deleted file's model stayed in the dev session. When the session could not read a changed file, it kept the file's model, so the TypeScript worker kept its errors and the preview build kept the module. Now the session asks the view whether the path still exists. Only when the view says no, after a staged delete or a delete written through, does it let go of the model, the same way
dispose()does: one reference per session, and the model is disposed with the last one. Any other read failure, such as a store error, keeps the model, as before. Test:a deleted file's model leaves the dev session, and a file the store failed to read keeps its model. It fails on the old catch and on a version that lets go on any failed read.Review fixes from earlier rounds
Two reviews found eight more holes. Each fix has a test that fails on the version before it.
The first review found these four:
rr-lock.jsoninto the new team. It now keeps the workspace it opened in.%20in a file name could hide a real error. Monaco decodes it in a model's uri; the file list does not. The second review's last fix replaced this one.The second review found these four:
rev. On return, the kept workspace is reused only when the manifest'srevis still the one its store last loaded or wrote. Otherwise a fresh build loads the manifest, as on a first visit. A manifest without arev, from before this change, also gets a fresh build.CachedVfsin packages/shell now counts changes per path. A read fills the cache only when nothing wrote, primed, invalidated, deleted or moved its path while it was on its way, and the whole cache was not dropped. It is the only shell change, and it leaves the staged view alone.wave-1.late0, but the result was never stored there.memory.peekand{{memory.ref:...}}on that key failed. The loop now stores each late result under its key.foo%2520bar.tskept failing check_app. Its model is namedfoo%20bar.ts, and the first fix kept every marker whose name had a%. check_app now asks the marker source for the uri each live file's model would get. The Monaco source usesUri.parse, as the dev session does, so the names match exactly and no%exception is needed.Known limit: two tabs in one team, with no switch, still erase each other's staged changes, as on the base. Each tab's store writes the whole manifest from its own memory, and the
revis checked only on return. A check on every save would need a merge or a refusal, so it is left for later. When a return needs a fresh build, a late write of the old run is not kept, as on the base.Fixes from review
Review found five more holes. Each fix has a test that fails on the commit before it.
unfinished()again counts every timed-out call until the loop hands over how it ended, as fix(agent): keep secrets out of the model in the browser agent #2836 did. Test:a call on the same file as one that timed out is not run, and done waits until the model reads how the first one ended, even when it ends as the finish rule hears a wave. Its hung edit ends inside the second wave'sendWave.latebefore the loop looked. The model got no late entry and nowave-N.lateIkey, and the call held back behind it was never planned again. Now a call stays inlatefrom the moment it is cut loose until the loop hands over how it ended.lateReturns()hides the result as an outcome is hidden, so the entry and memory both hold the hidden text. The late note now says "This call timed out and has now ended", since that can happen in the same wave. Test:a call that times out and ends before the rest of its wave still gets its late entry, redacted and stored under its key. The edit has a 10 ms limit and ends at 25 ms, and a read in the same wave takes 50 ms.src,CachedVfsmarked onlysrc, so a read ofsrc/a.tsstill on its way cached the old file and later reads returned it. A read now fills the cache only when neither its path nor any folder above it changed. Test:a read still on its way does not cache its old copy once its file, a folder above it or the whole cache changed, one case for each change the cache checks.a deleted file's compiled output goes with its model, so a new file at its path is compiled again.an emit still on its way when its file is deleted and made again does not cache the deleted file's output.After the restack
.envread hid, and fails without this.Built on #2853 and the guides
unstartedmarks a call whose work never began, and feat(agent): ask before the browser agent's outward or irreversible actions #2853 leaves such calls out of the finish record. A held-back call isunstartedtoo. A later call waits when an earlier one was held back, but not when it was refused or its prepare failed. That fact stays insiderunWaveas the call each held-back call waits for, not as a second flag.createRocketAgentHost(client, ws)keeps feat(agent): ask before the browser agent's outward or irreversible actions #2853's approval feed, one per page, and bindsvfs,scopeKey, scaffold, pack and dev sessions tows. The run'sApprovalsreads app names through the bound view. A delete approved after a team switch stages in the old team, and commits there on return.createWorkspaceBackend(scope)andtoStorePath(path, scope)sit on the base's root table. Store roots translate into the backend's own scope;.rocketride/reads come from the server's bundle in every team's workspace. A kept or reopened workspace is the same staged view, so itsassertWritablestill refuses guide changes before they stage.Tests
The ordering explorations cover the orders of team switches, timeouts and approvals. A hand-written test stays only for what they cannot reach. Each fix line of this PR was reverted on its own (69 reverts), and each kept test fails on a revert that no other kept test or exploration fails on. Nine of the reverts fail no test; "Notes for Dylan" lists those lines and what each is for.
ordering.test.tstries every order of up to six steps (start, release a store call, switch, time out). With this change, none of 78 orders erases a write. A second pass makes each edit ask for an OK first and adds an approve step. It also fails an order that asks for an OK while an edit's work is on its way. None of 112 orders fails. With the backend translating paths in the active scope, as on fix(agent): keep secrets out of the model in the browser agent #2836, 19 of 77 and 13 of 112 orders fail. Without the hold-back in a wave, 25 of 113 and 32 of 122 do.fakeStore.harness.ts(new) is the fake store client thatordering.test.tsandworkspace.test.tsshare. A read or write of a watched path waits until the test releases it.workspace.test.ts(new) runs rocket-ui's real workspace over that store for what the explorations cannot reach: the old team's change announcements, a scaffold after a switch, going back to a team (a late change kept, another tab's discard, files and the file list read again, an old read after a save), a read on its way through each change the cache checks, and the listing save. It also runs the dev session: its package, lock and icon reads and its lock write after a switch, its models and its emit cache.loop.test.ts: a call held back by a timed-out call in its wave, in an earlier wave (and a call on another file that runs), or behind a held-back call. A done is sent back until the model has read how a timed-out call ended: with no calls, with calls that all worked, and when the call ends while the finish rule hears the wave. The late entry when the call ends before the rest of its wave, redacted and stored under its key. A timed-out prepare holds nothing back.agent.test.ts(new): another team's conversation is refused by the team's name, and a conversation saved before scopes saves its team before the model or a pasted key is reached.toolkits.test.ts: the errors of a deleted file are ignored and named, so the check passes, and the same errors fail it while the file exists. Each file is matched to its model by the name the editor gives it.Dylan's review found several orders tested twice. The hand-written copies are gone: the write in flight at a switch, the timed-out edit in the real workspace, and the loop tests that pair an approval with a timeout. So are tests whose every revert another kept test fails, and two loop tests that pass on #2836, since they cover #2853's stop and OK clock. The preview's helper-level tests became one test that drives the dev session across a switch, since only that catches the session leaving its workspace out. Tests went from 1,863 to 1,208 added lines.
All 290 tests in
./builder shared:testpass but #2836's 2 known limits, which are marked todo. #2836 alone has 268, so this PR adds 22, down from 42.The apps/shared, apps/rocket-ui and packages/shell type-checks pass, and so does
./builder rocket-ui:build. The changed test files type-check but for 3 errors thatstagedFileSystem.test.tsalready has on #2836. ESLint finds nothing new in the changed files, and Prettier is clean on every changed line.Browser journeys
Both with an engine whose Python comes from this branch (release server-v3.5.0 binary, since no release matches the branch's C++), so chat sessions save.
build-pipeline, on this commit: passes 4 of 4. The turn ends on its own, writespipelines/verify-chat.pipe, which the engine validates with no errors, and the final reply is saved in the session.secrets, on this commit before theunfinished()fix, which changes only when a done is accepted: passes 14 of 14,sessions-savedincluded, so the checks on the saved sessions read real files. The fake key in a pipeline reaches the model only as a«secret:…»marker. A pasted key and a key sent mid-run reach no frame the page sent, no saved session and no catalog row, and both show as[redacted]in the user's own rows.The team switch cannot be driven here: the OSS engine has one team. The ordering tests and the workspace tests cover it.
Notes for Dylan
isCurrent()waiting for the store's own manifest write: going back right after a save still reuses the workspace. Inloop.ts,stopped()naming only calls still running;runWaveholding files only for timed-out calls with no ending yet, so a call is not held back by one that has ended; and a late failure's error in its entry. A new conversation saves nothing before its first turn ends. A conversation from the user's own files is refused as "your personal workspace".readAppPackage,writeAppPackage,readAppText,readAppImageDataUriandprepareDependencies.WebDevSessionkeeps the workspace from its constructor and passes it to those reads and to the lock write.WebDevSessionnow keeps one content listener per path instead of a list, so it can let go of one model.dispose()goes through the samereleaseModel. The last release of a model also drops its entries from the shared emit cache.webHost.tsbinds onlysaveListing. Its other calls only read, anddeployalready takes the workspace when it starts, throughpackAppSource's default.unstartedflag, so the finish record never hears of it, as with a refused call.runLoop, a late entry is read before yourREAD_THE_USERcheck.scopeKey. The session saves a conversation from before scopes once more at its first turn, to record its team.Proved with: 288 of 290 shared tests (2 todo for #2836's limits), the ordering explorations (0 of 78 and 0 of 112 orders erase a write), a revert of each fix line, the three type-checks, the rocket-ui build, and the
build-pipelineandsecretsbrowser journeys passing every check on an engine built from this branch.The stack
Eleven PRs, each on the one below it. Merge top-down: each PR squashes into the branch below it, starting with #2885 and ending with #2835 into
feat/rocket-agent-browser. A top-down rehearsal in a scratch clone was clean at every step and ended on #2885's tree. Bottom-up also works if each next branch is first rebased onto the squashed base.