Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/empty-response-fresh-tools.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"eve": patch
---

Allow empty-response recovery to use tools needed by the current request, including fresh reads, instead of treating earlier tool results as sufficient. The recovery instruction still asks the model to reuse suitable results, avoid repeating completed writes, and check uncertain action outcomes before retrying.
2 changes: 2 additions & 0 deletions docs/concepts/sessions-runs-and-streaming.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,8 @@ Note: consider the privacy, confidentiality, and user-experience implications fo

When a task explicitly requires conditional delivery and there is nothing new to report, the agent can finish with exactly `<eve-empty-delivery/>`. eve emits `message.completed` with `message: null` for that intentional silence. The marker must be the entire response, apart from surrounding whitespace; the HTML-escaped form `&lt;eve-empty-delivery/&gt;` is also accepted. A response that quotes the marker in prose or code is delivered normally.

When a model returns no text or tool calls, eve retries the model call once with a temporary recovery instruction. That instruction asks the model to continue the current request, use fresh reads when requested, reuse suitable results, and avoid repeating completed writes. It does not change tool permissions or approval requirements and is not saved in session history.

A delegated subagent publishes progress on its own child-session stream. The parent emits `subagent.called` with a `childSessionId`, which a client uses to attach. Both blocking and background calls emit `subagent.completed` with the actual output after the parent records a successful outcome. A background working receipt is an `action.result` tool output; it does not emit completion. Older streams can contain receipt-bearing completion events marked by `data.backgroundTask`; those are admission only. Completion does not mean that the reusable child session has ended. Background task terminal outcomes also arrive as task-triggered `message.received` notifications with `data.kind: "execution.background_task"`. The default frontend reducer retains these events in `events` but does not project their runtime-authored text as participant messages.

`step.failed` and `turn.failed` carry `{ code, message, details? }` for the failed fragment or turn, and `session.failed` is the terminal session-level variant. `turn.cancelled` is not a failure: the cancelled turn ends without any failure event, `session.waiting` follows, and the session accepts the next message normally. Whatever the turn streamed before cancellation stays on the stream. Durable history keeps the accepted user input and previously settled work, but discards incomplete assistant output and unfinished tool state. When a turn requested an output schema, the finalized payload lands on `result.completed` as `data.result` before the turn boundary. `authorization.required` carries the sign-in challenge (`data.authorization` may include `url`, `userCode`, `expiresAt`, `instructions`), and `authorization.completed` carries `data.outcome` (`"authorized" | "declined" | "failed" | "timed-out"`).
Expand Down
114 changes: 114 additions & 0 deletions packages/eve/src/harness/tool-loop.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5625,6 +5625,9 @@ describe("createToolLoopHarness", () => {
});

describe("empty model response recovery", () => {
const expectedEmptyResponseNudge =
"Your previous response was empty and was not delivered. Continue the user's request from before this notice, following normal tool and approval rules. Use any tools needed to finish the request, including fresh reads when the request needs current information. Reuse successful results when they answer that request. Do not repeat a completed write or other side-effecting action unless another execution is still required by the user's request. If an action's outcome is uncertain, check its status before retrying. Never describe older results as a fresh check. Do not mention this notice.";

const emptyResult: Record<string, unknown> = {
content: [],
finishReason: "other",
Expand Down Expand Up @@ -5756,6 +5759,117 @@ describe("createToolLoopHarness", () => {
}
});

it("preserves a refresh request and stale tool context while allowing a fresh read", async () => {
const staleReadCall = {
input: { resource: "report" },
toolCallId: "stale-read-1",
toolName: "read_report",
type: "tool-call" as const,
};
const staleReadResult = {
...staleReadCall,
output: { type: "json" as const, value: { status: "ready", value: "old" } },
type: "tool-result" as const,
};
const tools = new Map([
[
"read_report",
{
description: "Read the report.",
execute: vi.fn().mockResolvedValue({ status: "ready", value: "new" }),
inputSchema: jsonSchema({ type: "object" }),
name: "read_report",
},
],
]);
const session = createTestSession({
agent: {
modelReference: { id: "test-model" },
system: "You are a test assistant.",
tools: [
{
description: "Read the report.",
name: "read_report",
inputSchema: { type: "object" },
},
],
},
history: [
{ content: "Check the report.", kind: "user", role: "user" },
{ content: [staleReadCall], role: "assistant" },
{ content: [staleReadResult], role: "tool" },
],
});
const refreshRequest = "How many rows does the report have now?";

setupFirstThenAgent(emptyResult, successResult);
const warnSpy = vi.spyOn(console, "warn").mockImplementation(() => {});
const { emit } = createEventCollector();
const runStep = createToolLoopHarness(createTestConfig("conversation", emit, { tools }));

try {
const result = await runStep(session, { message: refreshRequest });
const firstCall = vi.mocked(ToolLoopAgent).mock.calls[0]?.[0];
const retryCall = vi.mocked(ToolLoopAgent).mock.calls[1]?.[0];
const firstAgent = vi.mocked(ToolLoopAgent).mock.results[0]?.value as {
stream: ReturnType<typeof vi.fn>;
};
const retryAgent = vi.mocked(ToolLoopAgent).mock.results[1]?.value as {
stream: ReturnType<typeof vi.fn>;
};
const firstMessages = firstAgent.stream.mock.calls[0]?.[0]?.messages;
const retryMessages = retryAgent.stream.mock.calls[0]?.[0]?.messages as Array<{
content: unknown;
role: string;
}>;

expect(result.next).toBeNull();
expect(firstCall?.tools).toMatchObject({ read_report: expect.anything() });
expect(retryCall?.tools).toMatchObject({ read_report: expect.anything() });
expect(retryMessages.slice(0, -1)).toEqual(firstMessages);
expect(retryMessages.slice(0, -1)).toEqual(
expect.arrayContaining([
expect.objectContaining({ content: refreshRequest, role: "user" }),
expect.objectContaining({
content: expect.arrayContaining([
expect.objectContaining({
input: staleReadCall.input,
toolCallId: staleReadCall.toolCallId,
toolName: staleReadCall.toolName,
type: "tool-call",
}),
]),
role: "assistant",
}),
expect.objectContaining({
content: expect.arrayContaining([
expect.objectContaining({
output: staleReadResult.output,
toolCallId: staleReadCall.toolCallId,
toolName: staleReadCall.toolName,
type: "tool-result",
}),
]),
role: "tool",
}),
]),
);
expect(retryMessages.at(-1)).toMatchObject({
content: expectedEmptyResponseNudge,
kind: "execution.retry",
role: "user",
});
expect(
result.session.history.some(
(message) =>
typeof message.content === "string" && message.content === expectedEmptyResponseNudge,
),
).toBe(false);
} finally {
warnSpy.mockRestore();
}
});

it("offers conditional delivery on an empty-response retry for a scheduled turn", async () => {
setupFirstThenAgent(emptyResult, successResult);
const warnSpy = vi.spyOn(console, "warn").mockImplementation(() => {});
Expand Down
14 changes: 7 additions & 7 deletions packages/eve/src/harness/tool-loop.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2480,15 +2480,15 @@ function rethrowNoOutputAsEmptyResponse(error: unknown): never {
}

/**
* Wire-only note the empty-response reissue appends to its retry, so the
* model answers from the tool results already in context instead of
* re-exploring. Each recovery stage declares its own follow-up text: the
* tool recovery prepends {@link buildDisabledToolNote} as a system note
* (its toolset change busts the prompt cache anyway), this one trails as
* a user note to keep the cached prefix valid.
* Wire-only note the empty-response reissue appends to its retry. Existing
* tool results can belong to an earlier request, so recovery must still
* allow tools needed by the current request. Each recovery stage declares
* its own follow-up text: tool recovery prepends {@link buildDisabledToolNote}
* as a system note (its toolset change busts the prompt cache anyway), this
* one trails as a user note to keep the cached prefix valid.
*/
const EMPTY_RESPONSE_NUDGE =
"Your previous reply was empty and was not delivered. Answer now from the tool results above; do not re-run tools or mention this notice.";
"Your previous response was empty and was not delivered. Continue the user's request from before this notice, following normal tool and approval rules. Use any tools needed to finish the request, including fresh reads when the request needs current information. Reuse successful results when they answer that request. Do not repeat a completed write or other side-effecting action unless another execution is still required by the user's request. If an action's outcome is uncertain, check its status before retrying. Never describe older results as a fresh check. Do not mention this notice.";

function buildEmptyResponseNudge(emptyDeliveryEnabled: boolean): string {
if (!emptyDeliveryEnabled) {
Expand Down