📌 fix: Stabilize Agent Prompt Cache Prefix by danny-avila · Pull Request #12907 · LibreChat-AI/LibreChat

danny-avila · 2026-05-01T14:24:59Z

Summary

make agent instructions the stable cacheable prefix and move memory, RAG/file context, request tool context, and runtime additions into additional_instructions
anchor date/time at conversation creation and persist createdAt on first insert for reuse across turns
split web_search into static citation/search instructions plus dynamic anchored conversation date/time
resolve temporal special variables from the conversation anchor and classify temporal instruction text as dynamic
update @librechat/agents to 3.1.75

Fixes #12634
Depends on LibreChat-AI/agents#128

Tests

npx jest specs/parsers.spec.ts --runInBand
npm run build && npx jest src/tools/toolkits/web.spec.ts src/agents/context.spec.ts src/agents/tests/run-summarization.test.ts src/agents/tests/initialize.test.ts --runInBand
npx jest src/methods/conversation.spec.ts --runInBand
npx jest server/controllers/agents/client.test.js --runInBand --testNamePattern="memory context|MCP instructions"

Note: packages/api build passed with the existing Redis type warning in src/cache/cacheFactory.ts.

github-actions · 2026-05-01T14:32:17Z

GitNexus: 🚀 deployed

The LibreChat-pr-12907 index is now live on the MCP server.
Deploy run

Copilot

Pull request overview

This PR stabilizes the cacheable “system prompt prefix” for agent runs by keeping agent/tool instructions static while shifting volatile per-turn context (memory/RAG/file context, runtime timestamps, etc.) into additional_instructions, anchored to the conversation’s creation time.

Changes:

Anchors date/time special-variable resolution to a persisted conversation creation timestamp (captured server-side and reused across turns).
Splits web_search context into a static instruction block and a dynamic runtime block built from the anchored conversation time.
Refactors agent instruction composition to keep stable instructions separate from dynamic “system tail”, and bumps @librechat/agents to 3.1.75.

Reviewed changes

Copilot reviewed 21 out of 22 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
packages/data-schemas/src/methods/conversation.ts	Adds `createdAtOnInsert` support to persist a stable conversation `createdAt` during first upsert.
packages/data-schemas/src/methods/conversation.spec.ts	Tests `createdAtOnInsert` is applied only on insert and not overwritten on updates.
packages/data-provider/src/parsers.ts	Extends `replaceSpecialVars` with optional `now` anchor + whitespace-tolerant placeholders.
packages/data-provider/specs/parsers.spec.ts	Adds tests for anchored time replacement and whitespace in placeholders.
packages/api/src/types/http.ts	Adds `conversationCreatedAt` to request typing for timestamp anchoring.
packages/api/src/tools/toolkits/web.ts	Splits web search context into static + `buildWebSearchDynamicContext(now)` runtime block.
packages/api/src/tools/toolkits/web.spec.ts	Verifies static context has no volatile substitutions; dynamic context is deterministic by anchor.
packages/api/src/agents/run.ts	Separates stable tool/agent instructions vs dynamic tail via `dynamicToolContextMap` and `additional_instructions`.
packages/api/src/agents/initialize.ts	Anchors temporal special vars to `req.conversationCreatedAt` and classifies temporal instructions as dynamic.
packages/api/src/agents/context.ts	Splits stable instructions (`instructions`) from dynamic runtime context (`additional_instructions`).
packages/api/src/agents/context.spec.ts	Updates/extends tests for stable vs dynamic instruction composition.
packages/api/src/agents/tests/run-summarization.test.ts	Adds regression test ensuring stable and dynamic instruction fields remain separated.
packages/api/src/agents/tests/initialize.test.ts	Tests temporal vars move to dynamic tail and anchor is applied.
packages/api/package.json	Bumps `@librechat/agents` to `^3.1.75`.
api/server/services/ToolService.js	Produces both `toolContextMap` (static) and `dynamicToolContextMap` (anchored runtime) for tools.
api/server/controllers/agents/request.js	Resolves and attaches `req.conversationCreatedAt` (new vs existing convo) for prompt anchoring.
api/server/controllers/agents/client.js	Preserves stable/dynamic instruction fields when applying shared run context.
api/server/controllers/agents/client.test.js	Updates expectations: memory context now lives in `additional_instructions` rather than stable `instructions`.
api/app/clients/tools/util/handleTools.js	Adds `dynamicToolContextMap` output and anchors dynamic web search context from request.
api/app/clients/BaseClient.js	Persists anchored `createdAt` on first conversation insert via `createdAtOnInsert`.
api/package.json	Bumps `@librechat/agents` to `^3.1.75`.
package-lock.json	Locks `@librechat/agents` to `3.1.75`.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…-cache-stable-prefix # Conflicts: # api/server/controllers/agents/client.js # api/server/controllers/agents/client.test.js

github-actions · 2026-05-01T22:41:46Z

GitNexus: 🚀 deployed

The LibreChat-pr-12907 index is now live on the MCP server.
Deploy run

github-actions · 2026-05-01T23:53:21Z

GitNexus: 🚀 deployed

The LibreChat-pr-12907 index is now live on the MCP server.
Deploy run

github-actions · 2026-05-01T23:59:37Z

GitNexus: 🚀 deployed

The LibreChat-pr-12907 index is now live on the MCP server.
Deploy run

danny-avila · 2026-05-02T00:09:37Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9e947099d1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

danny-avila · 2026-05-02T00:24:06Z

@codex review

chatgpt-codex-connector · 2026-05-02T00:30:20Z

Codex Review: Didn't find any major issues. You're on a roll.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

github-actions · 2026-05-02T00:31:03Z

GitNexus: 🚀 deployed

The LibreChat-pr-12907 index is now live on the MCP server.
Deploy run

danny-avila · 2026-05-02T00:33:15Z

@codex review

github-actions · 2026-05-02T00:39:58Z

GitNexus: 🚀 deployed

The LibreChat-pr-12907 index is now live on the MCP server.
Deploy run

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6905776dc6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

danny-avila · 2026-05-02T00:42:02Z

@codex review

github-actions · 2026-05-02T00:49:29Z

GitNexus: 🚀 deployed

The LibreChat-pr-12907 index is now live on the MCP server.
Deploy run

chatgpt-codex-connector · 2026-05-02T00:52:06Z

Codex Review: Didn't find any major issues. Hooray!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

* fix: stabilize agent prompt cache prefix * chore: refresh agents sdk lockfile integrity * test: format agent memory assertion * test: type agent context fixtures * fix: preserve MCP instruction precedence * fix: reuse resolved conversation anchor * fix: keep resumable startup immediate

fix: stabilize agent prompt cache prefix

4c57d8e

Copilot AI review requested due to automatic review settings May 1, 2026 14:25

Copilot started reviewing on behalf of danny-avila May 1, 2026 14:25 View session

Copilot AI reviewed May 1, 2026

View reviewed changes

Comment thread api/server/controllers/agents/request.js Outdated

danny-avila mentioned this pull request May 1, 2026

feat: add staticInstructions for multi-block Anthropic prompt cache o… LibreChat-AI/agents#52

Closed

Merge remote-tracking branch 'origin/dev' into danny-avila/fix-prompt…

67f70ca

…-cache-stable-prefix # Conflicts: # api/server/controllers/agents/client.js # api/server/controllers/agents/client.test.js

danny-avila added 3 commits May 2, 2026 08:42

chore: refresh agents sdk lockfile integrity

6d3fe7e

test: format agent memory assertion

d47c7c0

test: type agent context fixtures

9e94709

danny-avila linked an issue May 1, 2026 that may be closed by this pull request

[Bug]: Prompt caching invalidated due to dynamic system prompt #12634

Closed

1 task

danny-avila changed the title ~~fix: stabilize agent prompt cache prefix~~ fix: Stabilize Agent Prompt Cache Prefix May 1, 2026

danny-avila changed the title ~~fix: Stabilize Agent Prompt Cache Prefix~~ 📌 fix: Stabilize Agent Prompt Cache Prefix May 2, 2026

chatgpt-codex-connector Bot reviewed May 2, 2026

View reviewed changes

Comment thread packages/api/src/agents/context.ts Outdated

fix: preserve MCP instruction precedence

cc36a7a

fix: reuse resolved conversation anchor

6905776

chatgpt-codex-connector Bot reviewed May 2, 2026

View reviewed changes

Comment thread api/server/controllers/agents/request.js Outdated

fix: keep resumable startup immediate

8cfd2c3

danny-avila merged commit f3e1201 into dev May 2, 2026
15 checks passed

danny-avila deleted the danny-avila/fix-prompt-cache-stable-prefix branch May 2, 2026 00:55

marlonka mentioned this pull request Aug 17, 2026

[Enhancement]: Optimize GPT-5.6 prompt caching for OpenAI + Azure OpenAI with stable-prefix breakpoints #14949

Open

13 tasks

Uh oh!

Conversation

danny-avila commented May 1, 2026

Summary

Tests

Uh oh!

github-actions Bot commented May 1, 2026

GitNexus: 🚀 deployed

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

github-actions Bot commented May 1, 2026

GitNexus: 🚀 deployed

Uh oh!

github-actions Bot commented May 1, 2026

GitNexus: 🚀 deployed

Uh oh!

github-actions Bot commented May 1, 2026

GitNexus: 🚀 deployed

Uh oh!

danny-avila commented May 2, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

danny-avila commented May 2, 2026

Uh oh!

chatgpt-codex-connector Bot commented May 2, 2026

Uh oh!

github-actions Bot commented May 2, 2026

GitNexus: 🚀 deployed

Uh oh!

danny-avila commented May 2, 2026

Uh oh!

github-actions Bot commented May 2, 2026

GitNexus: 🚀 deployed

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

danny-avila commented May 2, 2026

Uh oh!

github-actions Bot commented May 2, 2026

GitNexus: 🚀 deployed

Uh oh!

chatgpt-codex-connector Bot commented May 2, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants