Repository navigation
[Proposal]: Mutable Agent Server Configuration #2523
Description
Is there an existing feature request for this?
- I have searched existing issues and feature requests, and this is not a duplicate.
Problem or Use Case
The all or nothing approach to server configuration and startup makes it difficult to create things like warm servers with snapshots of user data. In this general case, we want as many of the configuration and startup tasks as possible to be done before matching a warm server with a user, so the server has as little as possible to do and the experience is as fast as possible.
- Many environment variables are not available initially when a warm server resumes without a full restart of the container - e.g.: the
session_api_keyfrom the previous server run - User data is not available when the server starts. (As the user may not even have been matched to the server yet.)
Implementing this will make resuming existing paused conversations much faster in the cloud environment.
Proposed Solution
- We define a new environment variable
OH_DEFERRED_INIT.- If
OH_DEFERRED_INITis falsy, things work exactly as they do now. - Otherwise
- The server starts, but it is in a dormant mode - VSCode and browser use are running and ready to go, but the conversation, event, and bash routers will not have been added to the server yet, nor will their lifespan events have been run.
- Dormant mode servers will not respond to anything except the
/initendpoint and health endpoints - At some point after start up, configuration data and workspace content becomes available.
- Workspace content is either passed in by mounting a network directory using something like rclone or copied in and unzipped.
- Configuration data is posted to the
/initendpoint, kicking off the server initialization which is now done in the lifespan. - The init endpoint responds with a 200 once this is complete. Subsequent calls to the init endpoint will result in a 400 error.
- The workspace is zipped and written to a webhook endpoint periodically / on shutdown.
- If
Alternatives Considered
We've looked a lot at creating snapshots of user data using technologies like firecracker, but the downside is that the snapshots tend to be very large (Many Gb each), meaning that long term storage of one per conversation is quite infeasible. If we refactor the agent server to do this and use firecracker we can:
- Create full server snapshots of the dormant server
- Create file only diff snapshots - this will cover changes made to the user workspace and be much smaller than the multi Gb snapshots we would otherwise be dealing with.
- Combine these in startup to produce conversations which are fast to resume and reasonably sized.
If we continue with K8s, we can:
- Create warm runtimes which do not contain user data so may be used when resuming conversations - not just starting new ones.
- Use a technology like
rcloneor a simple tar of the workspace into the container in the init action. - Combine these on resume to make resuming an existing conversation as fast as starting a new one.
This effectively makes the slow sandbox resume problem orthogonal to the microvm / hypervisor / efficiency problem.
Priority / Severity
- Would make resuming paused conversations in the cloud much faster.
- Would make it more feasible to reduce costs by eliminating PVCs altogether.
Estimated Scope
Medium
Feature Area
- Agent API / Core functionality
- Tools / Tool system
- Skills / Plugins
- Agent Server
- Workspace management
- Configuration / Settings
- Examples / Templates
- Documentation
- Testing / Development tools
- Performance / Optimization
- Integrations (GitHub, APIs, etc.)
- Other
Technical Implementation Ideas (Optional)
No response
Additional Context
The app team are willing to take this one on as part of our work in boosting the performance and reducing the cost of the runtime api
Activity
[-][Feature]: Mutable Agent Server Configuration[/-][+][Proposal]: Mutable Agent Server Configuration[/+]enyst commentedon Apr 7, 2026
This sounds good to me. I'd love to know how are we thinking about the API to do this, I think maybe it can be both generic, and friendly to local-only?
xingyaoww commentedon Apr 10, 2026
This sounds great to me
Agree on this, we will need to think about how do we integrate this into
Workspaceclasses - maybe as an optional part of the context manager initialization, it will (1) spin up docker, (2) then send/init(IF user provides configuration), which seems reasonable to metofarr commentedon May 17, 2026
Resurrecting this — a few things worth folding into the proposal as additional motivation, plus some open questions that fall out once you look at it from the deployment-context angle.
Additional motivation
The three deployment contexts the server needs to serve. The current proposal reads like it only targets the K8s warm-pool case. Worth calling them out explicitly, because the design answer differs for each:
(3) in particular implies
/initmay need to be callable more than once per server lifetime (with a/deinitor equivalent to release the previous workspace) — that's a real design fork the current write-up doesn't address.Concrete numbers as the baseline. Today: PVCs are retained for ~2 weeks (cost), and starting a fresh pod + attaching a PVC to resume an old conversation takes ~10–30s. Stating these as the SLO this proposal is trying to move turns it from a refactor into a measurable win.
Frame this as a resume-path optimisation. Resume is slow today not because the runtime is intrinsically heavy, but because we can't afford to keep PVCs warm forever, so resume always pays a cold-start cost. That reframing matters because it tells reviewers what to benchmark.
Single-writer invariant on cheap object storage. This is the biggest technical idea missing from the current alternatives section. Because the orchestrator guarantees only one pod at a time is bound to a given workspace, the backing store doesn't need distributed-FS semantics — an object store with a lease/lock around the single writer is sufficient. That invariant is what makes rclone-on-S3 (or even a simpler pull-on-init / push-on-pause) cheaper than PVCs in the first place, and it's worth elevating to an explicit design constraint.
Decoupling the "warm pool" from "pre-attached storage." Today's warm pool needs PVCs because the workspace must already be there at attach time. Dormant mode lets the warm pool be stateless — no per-user storage attached until
/init. Arguably the bigger architectural payoff: no PVC reservations at all means the warm pool can be much larger for the same $.Open questions before the API design
These came up while mapping the proposal onto the three contexts above. They feel like prerequisites for the
Workspace-integration question @enyst and @xingyaoww raised:/initone-shot or recyclable? A 400-after-success is fine for sandbox-per-conversation, but kills the "few conversations per pod" case. A/deinit(flush workspace → object store, drop state, return to dormant) would cover both modes from a single primitive./init, and how does that get swapped for the user'ssession_api_key? The proposal mentionssession_api_keyas a symptom of the problem but not as a requirement on the/initpayload./initrequest body) would right-size the work. The proposal calls this out generically; an actual list would make the scope concrete.This comment was drafted by an AI agent (OpenHands) on behalf of @tofarr.
tofarr commentedon May 17, 2026
Resurrecting this — a few things worth folding into the proposal as additional motivation, plus some open questions that fall out once you look at it from the deployment-context angle.
Additional motivation
The three deployment contexts the server needs to serve. The current proposal reads like it only targets the K8s warm-pool case. Worth calling them out explicitly, because the design answer differs for each:
(3) in particular implies
/initmay need to be callable more than once per server lifetime (with a/deinitor equivalent to release the previous workspace) — that's a real design fork the current write-up doesn't address.Concrete numbers as the baseline. Today: PVCs are retained for ~2 weeks (cost), and starting a fresh pod + attaching a PVC to resume an old conversation takes ~10–30s. Stating these as the SLO this proposal is trying to move turns it from a refactor into a measurable win.
Frame this as a resume-path optimisation. Resume is slow today not because the runtime is intrinsically heavy, but because we can't afford to keep PVCs warm forever, so resume always pays a cold-start cost. That reframing matters because it tells reviewers what to benchmark.
Single-writer invariant on cheap object storage. This is the biggest technical idea missing from the current alternatives section. Because the orchestrator guarantees only one pod at a time is bound to a given workspace, the backing store doesn't need distributed-FS semantics — an object store with a lease/lock around the single writer is sufficient. That invariant is what makes rclone-on-S3 (or even a simpler pull-on-init / push-on-pause) cheaper than PVCs in the first place, and it's worth elevating to an explicit design constraint.
Decoupling the "warm pool" from "pre-attached storage." Today's warm pool needs PVCs because the workspace must already be there at attach time. Dormant mode lets the warm pool be stateless — no per-user storage attached until
/init. Arguably the bigger architectural payoff: no PVC reservations at all means the warm pool can be much larger for the same $.Open questions before the API design
These came up while mapping the proposal onto the three contexts above. They feel like prerequisites for the
Workspace-integration question @enyst and @xingyaoww raised:/initone-shot or recyclable? A 400-after-success is fine for sandbox-per-conversation, but kills the "few conversations per pod" case. A/deinit(flush workspace → object store, drop state, return to dormant) would cover both modes from a single primitive./init, and how does that get swapped for the user'ssession_api_key? The proposal mentionssession_api_keyas a symptom of the problem but not as a requirement on the/initpayload./initrequest body) would right-size the work. The proposal calls this out generically; an actual list would make the scope concrete.This comment was drafted by an AI agent (OpenHands) on behalf of @tofarr.