fix(session): cache messages across prompt loop to preserve prompt cache byte-identity#31527
fix(session): cache messages across prompt loop to preserve prompt cache byte-identity#31527BYK wants to merge 1 commit into
Conversation
|
The following comment was made by an LLM, it may be inaccurate: No duplicate PRs found. (The search returned PR #31527 which is the current PR itself, but no other related/duplicate PRs addressing the same issue.) |
810aaff to
f6cf0be
Compare
…che byte-identity filterCompactedEffect(sessionID) reloads all messages from the DB at the start of every prompt loop iteration. Between reloads, the DB returns rows with new object identity, breaking Anthropic's prompt cache which relies on byte-identical prefixes. Cache messages across loop iterations — first iteration does a full load, subsequent iterations only append new messages by ID. Full reload is flagged after subtask, compaction, or overflow operations.
f6cf0be to
c4699ba
Compare
|
Automated PR Cleanup Thank you for contributing to opencode. Due to the high volume of PRs from users and AI agents, we periodically close older PRs using automated criteria so maintainers can focus review time on the most active and community-supported contributions. This PR was closed because it matched the following cleanup criteria:
PRs created within the last month are not affected by this cleanup. If you believe this PR was closed incorrectly, or if you are still actively working on it, please leave a comment explaining why it should be reopened. A maintainer can review and reopen it if appropriate. Thanks again for taking the time to contribute. |
Issue for this PR
Closes #31525
Type of change
What does this PR do?
filterCompactedEffect(sessionID)reloads all messages from the DB at the start of every prompt loop iteration. Between reloads, the DB returns rows with new object identity, breaking Anthropic's prompt cache which relies on byte-identical prefixes.This PR caches messages across loop iterations:
needsFullReloadto get a fresh loadThis preserves object identity for existing messages across iterations, improving prompt cache hit rates.
Re-filing of #25367 which was auto-closed by the cleanup bot.
How did you verify your code works?
Tested locally with multi-turn sessions. Verified via logs that message count stays stable across iterations and only new messages are appended after tool calls.
Screenshots / recordings
N/A — backend performance improvement.
Checklist