Skip to content

fix(eve): persist framework prompt announcements - #3131

Merged
ruiconti merged 36 commits into
mainfrom
ruiconti/persist-framework-prompt-context
Sep 9, 2026
Merged

ruiconti merged 36 commits into
mainfrom
ruiconti/persist-framework-prompt-context

Conversation

@ruiconti

@ruiconti ruiconti commented Sep 8, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Framework announcements reached the model but were omitted from session.history. When task or skill state changed, eve rebuilt that context on the next request, changing an earlier part of the prompt and reducing cache reuse. This change saves those announcements with the conversation so subsequent requests retain the existing prefix and append updates. It also removes the pending-task silence instruction, which could suppress later user replies when retained in history.

Related to #3129.

Behavior

promptMessages now receives currentMessages.history, including task, skill, and delivery announcements as user-role messages from the first turn. Task snapshots are read from session.state before each model call, so they include newly started work.

For two background tasks, A and B:

Before
Call 1: [system]             [user request]
Call 2: [system + A pending] [user request] [start A → accepted]
Call 3: [system + A pending] [user request] [start A → accepted] [start B → accepted]

After
Call 1: [system] [user request]
Call 2: [system] [user request] [start A → accepted] [A pending]
Call 3: [system] [user request] [start A → accepted] [A pending] [start B → accepted] [A pending, B pending]

Previously, starting A changed the system prompt, and the snapshot remained stale when B started. Each new request now extends the preceding one, with unchanged announcements omitted.

HistoryStateKey tracks the last announcements saved in history. It advances only when a step returns updated history and resets when history is cleared or compacted. Failed model calls do not advance it, and completed compaction survives a later model failure. Client context and temporary approval-resume instructions remain outside durable history.

The pending-task silence instruction is removed. Once retained in history, it could suppress a later user question: a partial completion followed by “what is 7×8?” produced <eve-empty-delivery/> instead of 56. Pending wakes still accept empty responses, but no longer instruct the model to be silent. Existing batching combines completions already queued; it does not wait for unfinished siblings, so partial reports are allowed.

Validation

The new real-model eval launches five background reviewers in parallel, covering both the first and a later user turn. It verifies overlapping work, all five completion notifications, and a final report. Every subsequent parent request must read at least 98% of the preceding input from the provider cache; a single miss below that threshold fails the eval.

Both cases pass on OpenAI and Anthropic at e8581ac88, with all twelve cache checks passing per provider. The same scenario on unpatched main completes the reviews but fails every cache check, demonstrating that it catches the regression.

Unit and integration coverage verifies announcement persistence across serialized steps, clear, compaction, cancellation, and approval resume. The compaction fixtures now request the review and handoff their agent performs and return repository findings and handoff notes instead of repeated filler; their two-compaction and no-repeat assertions remain. Unit, integration, scenario, compaction, task-reporting, lint, and typecheck checks passed at e8581ac88.

An intermittent failure remains unexplained: earlier OpenAI runs reported an isolated zero-cache response despite an unchanged prefix in the captured trace, and correctly failed CI. The latest run had no such misses. Neither provider's latest eval log contains filtering or an empty-response retry.

The comparison results show why the pending-task silence instruction was removed.

The final regression uses the ordinary fixture model without the experiment wrapper. It requires the user answer before sibling tasks finish and a complete final report; the compaction case also retains its complete-report assertion. At 2e6c44266, rebased onto main d630b218a, 390 focused unit tests, 55 harness integration tests, build, all 45 typecheck tasks, frozen-lockfile install, conflict-file lint/format checks, documentation checks, and invariant checks pass locally. Real-model CI for the rebased revision is pending; the pre-rebase run tests 75241a560.

Checklist

  • This change was requested or approved by a maintainer
  • I ran the relevant checks from CONTRIBUTING.md
  • I added tests and documentation where relevant
  • I added a changeset if this touches the published eve package
  • DCO sign-off passes for every commit (git commit --signoff)

@vercel

vercel Bot commented Sep 8, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
eve-docs Ready Ready Preview, v0 Sep 9, 2026 3:45pm UTC
eve-docs-4759 Ready Ready Preview, v0 Sep 9, 2026 3:45pm UTC
eve-pkg Ready Ready Preview, v0 Sep 9, 2026 3:45pm UTC

@github-actions

github-actions Bot commented Sep 8, 2026 •

Copy link
Copy Markdown
Contributor

Bundle + Package Summary: apps/fixtures/weather-agent

Key takeaways

  • No notable deltas vs main (d630b21).

Delta vs main (d630b21)

Area Metric Baseline Current Delta
Package Packed tarball 8.70 MB 8.70 MB -64 B ✅
Package Unpacked publish size 32.77 MB 32.77 MB -866 B ✅
Package Installed footprint 77.91 MB 77.91 MB -866 B ✅
Package Published files 3911 3911 0
Package Installed files 7814 7814 0
Package Installed package instances 33 33 0
Package Distinct installed package names 32 32 0
Package Installed dependency edges 51 51 0
Package Installed optional peer edges 8 8 0
Runtime Unique function payloads 2 2 0
Runtime Total function bytes 20.78 MB 20.78 MB -536 B ✅
Runtime Public routes 18 18 0
Changed function payloads vs main (d630b21) (2)
Function Status Baseline Current Delta Route changes
functions/__server.func changed 10.39 MB 10.39 MB -268 B ✅ none
functions/.well-known/workflow/v1/flow.func changed 10.39 MB 10.39 MB -268 B ✅ none

eve init install

Metric Baseline Current Delta
Installed footprint 116.35 MB 116.34 MB -866 B ✅
Installed packages 113 113 0
dependencies 4 4 0
devDependencies 2 2 0
Dependency package bytes 48.94 MB 48.94 MB -866 B ✅
devDependency package bytes 5.04 MB 5.04 MB 0 B ➖
Build Metadata
  • Preset: vercel
  • Nitro: nitro@3.0.260903-beta
  • Output directory: apps/fixtures/weather-agent/.vercel/output
  • Build metadata timestamp: 2026-09-09T15:46:28.294Z
  • Route aliases: 18 public, 1 internal (19 total aliases)
  • Vercel routes in config: 21
  • Severity legend: 🔴 dominant/large, 🟠 notable, 🟡 watch, ⚪ small
Package Drill-Down

Package Details

  • Package: eve@0.52.4
  • Package directory: packages/eve
  • Tarball: 8.70 MB (eve-0.52.4.tgz)
  • Unpacked payload: 32.77 MB across 3911 published files
  • Installed footprint: 77.91 MB across 7814 installed files
  • Installed root package: 31.34 MB
  • Installed dependencies: 46.56 MB
  • Installed package instances: 33
  • Distinct installed package names: 32
  • Installed dependency edges: 51
  • Installed optional peer edges: 8
  • Runtime dependencies: 2
  • Peer dependencies: 6 (5 optional)

Installed footprint is measured from an isolated temporary npm install of the packed tarball.
Graph metrics read only package.json files in package directories directly beneath a node_modules boundary, including nested boundaries. Each directory is one package instance; distinct names come from those manifests. Dependency edges count each unique name in dependencies or optionalDependencies per instance; optional peer edges count peerDependencies marked optional.

Heavy installed dependencies

  • eve: 31.34 MB (40.2%)
  • @rolldown/binding-linux-x64-gnu: 19.31 MB (24.8%)
  • ai: 7.06 MB (9.1%)
  • zod: 6.41 MB (8.2%)
  • undici: 3.51 MB (4.5%)
Publish payload breakdown
Published file size
🔴 dist/src/compiled/shadcn-registry/index.js       [#############...........] 9.76 MB 29.8%
🟠 dist/src/compiled/@photon-ai/chat-adapter-ime... [###.....................] 2.27 MB 6.9%
🟠 dist/src/compiled/@ai-sdk/code-mode/index.js     [#.......................] 1.03 MB 3.1%
🟡 dist/src/compiled/@vercel/blob/index.js          [#.......................] 901.4 kB 2.8%
🟡 dist/src/compiled/_chunks/workflow/signal-exi... [#.......................] 514.6 kB 1.6%
🔴 Other published files                            [########################] 18.30 MB 55.9%
Installed footprint breakdown
Installed package size
🔴 eve                             [########################] 31.34 MB 40.2%
🔴 @rolldown/binding-linux-x64-gnu [###############.........] 19.31 MB 24.8%
🔴 ai                              [#####...................] 7.06 MB 9.1%
🔴 zod                             [#####...................] 6.41 MB 8.2%
🟠 undici                          [###.....................] 3.51 MB 4.5%
🟠 nitro                           [#.......................] 1.89 MB 2.4%
🔴 Other installed packages        [######..................] 8.37 MB 10.7%
Runtime dependencies (2)
Package Range Notes
nitro 3.0.260903-beta
undici 8.9.0
Peer dependencies (6)
Package Range Notes
@opentelemetry/api ^1.0.0 optional peer
ai catalog:
braintrust ^3.0.0 optional peer
dd-trace ^6.13.0 optional peer
just-bash ^3.1.0 optional peer
microsandbox ^0.5.0 optional peer
eve init install drill-down

eve init install details

  • Command: eve init my-agent
  • Package manager: npm
  • Installed footprint: 116.34 MB across 9710 installed files
  • Installed packages: 113 total (107 transitive-only)
  • dependencies: 4 direct packages totaling 48.94 MB
  • devDependencies: 2 direct packages totaling 5.04 MB
  • Other transitive package files: 62.36 MB

Installed footprint is measured from an isolated temporary eve init my-agent using the current packed eve tarball.

Heavy installed dependencies

  • eve: 31.34 MB (26.9%)
  • @typescript/typescript-linux-x64: 27.95 MB (24.0%)
  • @rolldown/binding-linux-x64-gnu: 19.31 MB (16.6%)
  • zod: 10.37 MB (8.9%)
  • ai: 7.06 MB (6.1%)
Installed footprint breakdown
Installed package size
🔴 eve                              [########################] 31.34 MB 26.9%
🔴 @typescript/typescript-linux-x64 [#####################...] 27.95 MB 24.0%
🔴 @rolldown/binding-linux-x64-gnu  [###############.........] 19.31 MB 16.6%
🔴 zod                              [########................] 10.37 MB 8.9%
🔴 ai                               [#####...................] 7.06 MB 6.1%
🟠 undici                           [###.....................] 3.51 MB 3.0%
🔴 Other installed packages         [#############...........] 16.80 MB 14.4%
dependencies (4)
Package Range Installed size Share
@vercel/connect 1.0.0 167.9 kB 0.1%
ai ^7.0.82 7.06 MB 6.1%
eve file:eve-0.52.4.tgz 31.34 MB 26.9%
zod 4.5.4 10.37 MB 8.9%
devDependencies (2)
Package Range Installed size Share
@types/node 24.x 2.54 MB 2.2%
typescript 7.0.2 2.50 MB 2.1%
Function Drill-Down

Payload Size Graph

Unique function payload size and share of total
🔴 functions/.well-known/workflow/v1/flow.func     [########################] 10.39 MB 50.0%
🔴 functions/__server.func                         [########################] 10.39 MB 50.0%

Top Function Payloads

🟠 functions/.well-known/workflow/v1/flow.func • 1 public route • 10.39 MB
Metric Value
Public routes /.well-known/workflow/v1/flow
Runtime nodejs24.x
Handler index.mjs
Payload 10.39 MB
Function files 10.39 MB across 113 files
Traced dependencies 0 B
Signal 🟠 Bundled file index.mjs is 2.34 MB (22.5%)

🟠 🔎 Dependency Analysis

📦 Bundled files:

Bundled file size
🟠 index.mjs                        [###########.............] 2.34 MB 22.5%
🟡 _libs/undici.mjs                 [#####...................] 980.8 kB 9.4%
🟡 _chunks/esm-Fqlolk7e.mjs         [###.....................] 723.3 kB 7.0%
🟡 _chunks/chatgpt-model.mjs        [###.....................] 698.9 kB 6.7%
🟡 _chunks/signal-exit-Dsy-TT0V.mjs [###.....................] 616.2 kB 5.9%
🔴 Other bundled files              [########################] 5.03 MB 48.4%

🧾 Vercel Config

{
  "handler": "index.mjs",
  "launcherType": "Nodejs",
  "shouldAddHelpers": false,
  "supportsResponseStreaming": true,
  "runtime": "nodejs24.x",
  "maxDuration": "max",
  "experimentalTriggers": [
    {
      "type": "queue/v2beta",
      "topic": "__eve776561746865722d6167656e74_wkf_workflow_*",
      "consumer": "default",
      "retryAfterSeconds": 5,
      "initialDelaySeconds": 0
    }
  ],
  "environment": {
    "WORKFLOW_PRECONDITION_GUARD": "1"
  }
}

🟠 functions/__server.func • 17 public routes, 1 internal alias • 10.39 MB
Metric Value
Public routes /
/.well-known/workflow/v1/webhook/[token]
/eve/v1/activity/[token]
/eve/v1/callback/[token]
/eve/v1/connections/[name]/callback/[attemptId]/[token]
/eve/v1/connections/[name]/callback/[token]
/eve/v1/health
/eve/v1/info
/eve/v1/session
/eve/v1/session/[parentSessionId]/subagents/[callId]/[childSessionId]/stream
/eve/v1/session/[sessionId]
/eve/v1/session/[sessionId]/cancel
/eve/v1/session/[sessionId]/clear
/eve/v1/session/[sessionId]/compact
/eve/v1/session/[sessionId]/reset
/eve/v1/session/[sessionId]/stream
/eve/v1/task-input/[token]
Internal aliases /__server
Runtime nodejs24.x
Handler index.mjs
Payload 10.39 MB
Function files 10.39 MB across 113 files
Traced dependencies 0 B
Signal 🟠 Bundled file index.mjs is 2.34 MB (22.5%)

🟠 🔎 Dependency Analysis

📦 Bundled files:

Bundled file size
🟠 index.mjs                        [###########.............] 2.34 MB 22.5%
🟡 _libs/undici.mjs                 [#####...................] 980.8 kB 9.4%
🟡 _chunks/esm-Fqlolk7e.mjs         [###.....................] 723.3 kB 7.0%
🟡 _chunks/chatgpt-model.mjs        [###.....................] 698.9 kB 6.7%
🟡 _chunks/signal-exit-Dsy-TT0V.mjs [###.....................] 616.2 kB 5.9%
🔴 Other bundled files              [########################] 5.03 MB 48.4%

🧾 Vercel Config

{
  "handler": "index.mjs",
  "launcherType": "Nodejs",
  "shouldAddHelpers": false,
  "supportsResponseStreaming": true,
  "runtime": "nodejs24.x"
}

Build Timing: e2e/fixtures/agent-tools-sandbox

This is an informational timing measurement inside eve build, from preflight through publication. Output-size measurement and profile writing are excluded.

Build mode: deployable Vercel build with sandbox template prewarm included.

  • Build pipeline: 4.90 s -> 5.00 s (+102.5 ms) vs main (d630b21).
  • Timing is informational: shared GitHub runners are too variable for a hard timing budget.
Detailed phase timings vs `main (d630b21)`
Phase Baseline Current Delta
extension.check 20.7 ms 0.8 ms -19.9 ms
project.resolve 4.2 ms 0.3 ms -3.9 ms
workspace.create 7.2 ms 0.5 ms -6.7 ms
host.prepare 448.0 ms 529.8 ms +81.8 ms
vercel.service-prefix.resolve 1.5 ms 1.4 ms -0.1 ms
nitro.create 425.9 ms 457.2 ms +31.3 ms
sandbox.prewarm 324.1 ms 343.3 ms +19.2 ms
nitro.cache.prepare 0.2 ms 0.2 ms 0.0 ms
nitro.prepare 1.0 ms 0.6 ms -0.4 ms
nitro.public-assets 0.6 ms 0.6 ms 0.0 ms
nitro.prerender 0.4 ms 0.4 ms 0.0 ms
nitro.bundle 3.60 s 3.61 s +9.6 ms
nitro.cache.write 0.3 ms 0.3 ms 0.0 ms
vercel.workflow-function.materialize 61.9 ms 50.9 ms -11.0 ms
agent-summary.emit 0.6 ms 0.6 ms 0.0 ms
nitro.close 0.1 ms 0.1 ms 0.0 ms
output.publish 5.0 ms 8.3 ms +3.3 ms
workspace.remove 1.9 ms 1.6 ms -0.3 ms

@vercel
vercel Bot temporarily deployed to Preview – eve-pkg September 8, 2026 03:36 Inactive
@vercel
vercel Bot temporarily deployed to Preview – eve-pkg September 8, 2026 03:39 Inactive
@vercel
vercel Bot temporarily deployed to Preview – eve-docs-4759 September 8, 2026 03:39 Inactive
@vercel
vercel Bot temporarily deployed to Preview – eve-pkg September 8, 2026 03:42 Inactive
@vercel
vercel Bot temporarily deployed to Preview – eve-pkg September 8, 2026 03:45 Inactive
@vercel
vercel Bot temporarily deployed to Preview – eve-docs-4759 September 8, 2026 03:45 Inactive
@vercel
vercel Bot temporarily deployed to Preview – eve-docs-4759 September 8, 2026 03:53 Inactive
@vercel
vercel Bot temporarily deployed to Preview – eve-pkg September 8, 2026 03:53 Inactive
@vercel
vercel Bot temporarily deployed to Preview – eve-pkg September 8, 2026 13:59 Inactive
@vercel
vercel Bot temporarily deployed to Preview – eve-docs-4759 September 8, 2026 13:59 Inactive
@vercel
vercel Bot temporarily deployed to Preview – eve-docs-4759 September 8, 2026 14:07 Inactive
@vercel
vercel Bot temporarily deployed to Preview – eve-pkg September 8, 2026 14:07 Inactive
@vercel
vercel Bot temporarily deployed to Preview – eve-pkg September 8, 2026 14:17 Inactive
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
…tting

Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
@ruiconti

ruiconti commented Sep 9, 2026 •

Copy link
Copy Markdown
Contributor Author

The parent should answer a new user question while background work continues.

The test starts three warehouse lookups. After a partial result arrives, the user asks:

Alice is packing seven boxes with eight jars in each. How many jars is that?

Expected: a reply containing 56, before the remaining lookups finish.

Both groups receive the same silence instruction on the first completion wake. At the user question, we either keep that instruction in the model's history or remove it. We ran 20 interleaved trials per group, with the same batching code.

Behavior being checked (Opus) Expected Instruction kept at user question Instruction removed at user question
Answer the jar-count question before the remaining lookups finish 20/20 replies containing 56 4/20 — fail 20/20 — pass
Suppress the user's answer with <eve-empty-delivery/> Never (0/20) 16/20 — fail 0/20 — pass
Report all three lookup results after the work finishes 20/20 complete reports 20/20 — pass 20/20 — pass

The first completion wake was silent in 20/20 trials in both groups, before the user asked the question. That is a shared experiment setup check; the behavior under test is answering the subsequent user question and delivering the final results.

The instruction meant for a background notification was also suppressing the user's answer. Opus results and traces.

Sol answered 20/20 in both groups, so the effect depends on the model. These were separate conversations, not identical replays; this comparison measures the instruction's effect, not batching improvements.

This branch was successfully deployed

3 active deployments
Preview – eve-docs — 2e6c4426 Deployed Sep 9, 2026 by vercel[bot]
Preview – eve-docs-4759 — 2e6c4426 Deployed Sep 9, 2026 by vercel[bot]
Preview – eve-pkg — 2e6c4426 Deployed Sep 9, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants