Repository navigation
Description
What happened
A production eve agent (Slack channel, scheduled app-auth run) posted the literal string <evedev-empty-delivery/> into a team channel. The model was following the conditional-delivery instruction and meant to deliver nothing — it just mistyped the marker, and the mistyped token was delivered as an ordinary message. In the same run, empty replies from the model were retried with the "Answer now from the tool results above" nudge, and the retries produced conversational filler ("I now have all the data I need.", "Here is the report:") that was also delivered as channel messages.
Bug 1 — the sentinel check fails open
hasEmptyDeliverySentinel (packages/eve/src/shared/empty-delivery.ts) is an exact includes() on <eve-empty-delivery/> (plus the HTML-escaped form). A correctly-spelled marker is reliably stripped — but any near miss falls through to the channel's message.completed default and is posted verbatim:
| Model's entire reply | Result today |
|---|---|
<eve-empty-delivery/> |
silenced ✓ |
<evedev-empty-delivery/> (observed in production) |
posted to the channel |
<eve-empty-delivery></eve-empty-delivery> (paired form, uncorrupted name) |
posted |
<eve-empty-delivery> / <EVE-EMPTY-DELIVERY/> / <eve-empty-delivery /> |
posted |
An internal control token that reaches a user-visible surface should fail closed: a message that is nothing but a sentinel-shaped tag carries no deliverable content, so the safe degradation for a mangled marker is silence, not literal control text in a team channel.
Bug 2 — the empty-response retry contradicts conditional delivery
The conditional-delivery instruction is injected as a system message, but the empty-response recovery nudge arrives as a user message leading with "Answer now from the tool results above" (packages/eve/src/harness/tool-loop.ts). In unattended app-auth runs there is no person to answer — the framing reliably elicits filler, which is then delivered. When conditional delivery is enabled for the turn, the recovery should lead with the marker option rather than an instruction to answer.
Environment / verification
- Observed in production on eve 0.39.1; the relevant code paths are unchanged on current
main(and were byte-identical in the published dist through 0.42.0 when we checked). - Searched existing issues; only the originating feature PRs (feat(eve): schedules - support prompt-driven empty delivery #248, feat(eve): schedules - support empty delivery #207) reference the marker.
Proposed fix
PR #2386 implements both: tolerant whole-message-anchored matching for sentinel-shaped tags (keeping the exact sentinel's match-anywhere behavior that existing tests pin), spelled to be linear-time on raw model output, plus a silence-first retry nudge under conditional delivery, with regression tests for the exact production shapes. Happy to adjust direction per maintainer preference — the PR lists its design decisions and known limitations explicitly.
Activity
roboeve commentedon Aug 21, 2026
triaged #2388 as a
p2channelsbug and applied the labels. reproduced all six malformed-marker cases onmain; the same cases are silenced on PR #2386, whose focused 41 matcher tests and scheduled retry regression pass. no duplicate report found, so i didn’t open a competing proposal.ruiconti commentedon Aug 21, 2026
@ace139 which model demonstrated this behavior?
ace139 commentedon Aug 22, 2026
@ruiconti GLM-5.2 via Vercel AI Gateway
benpankow commentedon Oct 9, 2026
Hi @ace139, thanks for filing this! Going through older issues, I believe this was resolved in 0.69.0 by #3840. Feel free to re-open if this is still relevant.