Skip to content

🕐 feat: Add promptCacheTtl model parameter for 1h/5m cache duration - #13835

Merged
danny-avila merged 6 commits into
devfrom
feat/prompt-cache-ttl
Jun 18, 2026
Merged

danny-avila merged 6 commits into
devfrom
feat/prompt-cache-ttl

Conversation

@danny-avila

@danny-avila danny-avila commented Jun 18, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Closes #12965

Closes #13507

Adds a user-configurable Prompt Cache Duration parameter (promptCacheTtl, dropdown: 5m | 1h) alongside the existing promptCache toggle for Anthropic, Bedrock, and OpenRouter endpoints.

The agents SDK now defaults prompt-cache writes to a 1h TTL (danny-avila/agents#249). This parameter defaults to undefined, so the SDK's default (1h) applies unless a user explicitly opts down to the legacy 5m TTL from the UI.

Why

1h cache writes cost 2× base input (vs 1.25× for 5m) but survive much longer, which is a net win for agentic / multi-turn workloads where the cached prefix is reused well beyond the 5-minute window. Making 1h the default while exposing an opt-out keeps cost-sensitive users in control.

Changes

data-provider (parameter definition)

  • schemas.ts: promptCacheTtl: z.enum(['5m','1h']).optional() on the conversation schema; added to anthropic/openRouter picks
  • parameterSettings.ts: enum/dropdown SettingDefinition for anthropic and bedrock, wired into the endpoint parameter arrays
  • types.ts, bedrock.ts: type union + bedrock picks/field list

data-schemas (persistence)

  • convo.ts / preset.ts: promptCacheTtl?: '5m' | '1h'
  • defaults.ts: mongoose { type: String }

api (backend wiring)

  • anthropic/llm.ts: threads promptCacheTtl into llmConfig inside the supportsCacheControl block
  • openai/llm.ts: forwards it in the OpenRouter branch
  • types/openai.ts: type field

i18n — en/translation.json: label, description, and "Default (1 hour)" placeholder keys

Testing

  • data-provider: build + 115 schema tests pass
  • data-schemas: build clean
  • api: tsc --noEmit 0 errors; anthropic llm.spec 99 tests pass (incl. 2 new: set → '1h' lands in llmConfig; unset → undefined so the SDK default applies)
  • ESLint clean across all changed files

Notes

  • Default undefined means no behavior change for existing users beyond the SDK's new 1h default — the control is purely an opt-out path to 5m.
  • Depends on the agents SDK version already present on dev.

Adds a user-configurable `promptCacheTtl` parameter (dropdown: 5m | 1h)
alongside the existing `promptCache` toggle for Anthropic, Bedrock, and
OpenRouter endpoints. Default is undefined so the agents SDK applies its
own default (1h), letting users opt down to the legacy 5m TTL.

- data-provider: schema, parameterSettings dropdown, types, bedrock picks
- data-schemas: convo/preset types + mongoose defaults
- api: thread promptCacheTtl into anthropic + openai(OpenRouter) llmConfig
- i18n: en translation keys for label/description/default placeholder
- tests: anthropic llm.spec coverage for set + unset cases
@danny-avila

Copy link
Copy Markdown
Collaborator Author

@codex review

@danny-avila danny-avila changed the title 🕐 feat: Add promptCacheTtl model parameter for 1h/5m cache duration 🕐 feat: Add promptCacheTtl model parameter for 1h/5m cache duration Jun 18, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e15bfa8a65

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

'topP',
'stop',
'promptCache',
'promptCacheTtl',

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Gate Bedrock 1h TTL to supported Claude models

When a Bedrock user selects 1h, this parser now preserves promptCacheTtl for every Bedrock model, while the cleanup below only toggles promptCache based on claude/nova and never clears or allowlists the TTL. Bedrock's extended 1h cache TTL is only accepted on select Claude 4.5 models; choosing it on other cache-enabled Bedrock models such as Nova or older Claude versions will still send the unsupported TTL and can make those requests fail. Clear promptCacheTtl when caching is off/unsupported and only pass 1h for the Bedrock models that support it.

Useful? React with 👍 / 👎.

Comment on lines +117 to +118
promptCacheTtl:
options.modelOptions?.promptCacheTtl ?? anthropicSettings.promptCacheTtl.default,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Honor custom Anthropic TTL defaults

This resolves promptCacheTtl only from modelOptions before defaultParams/addParams are processed, and the later loops only apply keys from knownAnthropicParams, which does not include this new field. For a custom Anthropic endpoint that exposes promptCacheTtl through customParams.paramDefinitions or forces it with addParams, the configured 5m/1h TTL is silently ignored and the SDK default is used instead. Thread this key through the same default/add handling as the other Anthropic system parameters.

Useful? React with 👍 / 👎.

Comment on lines +636 to +637
if (promptCacheTtl != null) {
llmConfig.promptCacheTtl = promptCacheTtl;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Apply OpenRouter TTL defaults before assigning

This copies promptCacheTtl only from modelOptions; unlike promptCache, the defaultParams and addParams branches above never update a TTL variable, and dropParams: ['promptCacheTtl'] cannot remove a selected value because this assignment happens after the drop checks. For OpenRouter custom endpoints using customParams.paramDefinitions or admin addParams/dropParams to control the new TTL, the setting is ignored or cannot be disabled. Resolve and drop the TTL alongside enablePromptCache before assigning it to llmConfig.

Useful? React with 👍 / 👎.

…TTL params (Codex review)

- bedrock.ts: clear promptCacheTtl whenever promptCache is off/unsupported,
  so an unsupported 1h is never sent on a non-caching Bedrock request
- openai/llm.ts: resolve promptCacheTtl through the same defaultParams/
  addParams/dropParams machinery as promptCache (via promptCacheTtlValue)
  so OpenRouter custom endpoints can configure/override/drop it
- tests: bedrock TTL-tied-to-promptCache cases; OpenRouter TTL default/add/drop
@danny-avila

Copy link
Copy Markdown
Collaborator Author

Addressed in b64aaf26c:

P2 — Gate Bedrock 1h TTL to supported models (bedrock.ts). Tied promptCacheTtl to promptCache: after the claude/nova promptCache resolution, the parser now clears promptCacheTtl whenever promptCache !== true — i.e. when caching is off or unsupported for the model. So a non-caching Bedrock model (or an explicit promptCache: false) can no longer carry a stray 1h into a request. I deliberately did not add a per-model 1h allowlist: that's an SDK-layer concern, and the agents SDK is intentionally designed to keep the TTL tied to promptCache rather than maintain a model whitelist (Bedrock gracefully downgrades an unsupported ttl rather than rejecting it — verified live on Claude 4.6 during the agents PR). Added regression tests (preserve on caching model / clear on explicit-off / clear on non-caching model).

P2 — Apply OpenRouter TTL defaults before assigning (openai/llm.ts). Fixed — promptCacheTtl now flows through a resolved promptCacheTtlValue mirroring enablePromptCache: seeded from modelOptions, fillable via defaultParams, overridable via addParams, and clearable via dropParams: ['promptCacheTtl'] (the drop now runs before the assignment). Added a default/add/drop regression test.

P2 — Honor custom Anthropic TTL defaults (anthropic/llm.ts). No change — promptCacheTtl already has full parity with promptCache here. In this file promptCache is not in knownAnthropicParams and is not threaded through defaultParams/addParams either; both are resolved purely as systemOptions from modelOptions (so customParams.paramDefinitions values, which arrive via modelOptions, already work), and both land on requestOptions so dropParams can remove them. Threading only promptCacheTtl through add/default handling would make it diverge from promptCache — the opposite of the consistency the finding is after. If we want admin addParams to force cache settings on the native Anthropic endpoint, that's a separate change that should cover promptCache too.

tsc clean; api (182) + data-provider (268) suites + lint green.

@codex review

@danny-avila danny-avila linked an issue Jun 18, 2026 that may be closed by this pull request

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b64aaf26c7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +652 to +653
if (promptCacheTtlValue != null) {
llmConfig.promptCacheTtl = promptCacheTtlValue;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Honor OpenRouter TTL with default cache switch

This assignment only runs when enablePromptCache is exactly true. In the OpenRouter settings UI, promptCache is displayed as enabled by default, but DynamicSwitch does not persist default values until the switch is toggled; selecting only the new TTL dropdown therefore sends modelOptions.promptCacheTtl without modelOptions.promptCache. That common path silently drops the requested TTL, making the new control ineffective unless the user also toggles prompt caching or an admin defaultParams sets promptCache.

Useful? React with 👍 / 👎.

…t (Codex review)

OPENROUTER_DEFAULT_PARAMS injects promptCache:true into defaultParams, so a
TTL-only dropdown selection (promptCacheTtl set, promptCache switch untouched)
still resolves caching on and forwards the TTL. Add regression tests via the
real getOpenAIConfig entry point: TTL-only -> promptCache+TTL both set;
explicit promptCache:false -> both dropped.
@danny-avila

Copy link
Copy Markdown
Collaborator Author

P2 — Honor OpenRouter TTL with default cache switch (openai/llm.ts). This is a false positive — the TTL-only path already works. For OpenRouter, getOpenAIConfig → getDefaultParams always injects OPENROUTER_DEFAULT_PARAMS = { promptCache: true } into defaultParams (openai/config.ts:15,33). That runs through the defaultParams loop and flips enablePromptCache from undefined → true before the enablePromptCache === true gate. So when a user picks only the TTL dropdown (modelOptions.promptCacheTtl set, promptCache switch never persisted), caching still resolves on and the TTL is forwarded.

The existing baseline test already encodes this default-on behavior (getOpenAIConfig(mockApiKey, {}, 'openrouter') → promptCache: true). I added two regression tests through the real getOpenAIConfig entry point in config.spec.ts to lock in the exact scenario from the finding:

  • TTL-only selection (modelOptions: { promptCacheTtl: '1h' }, no promptCache) → llmConfig.promptCache === true and llmConfig.promptCacheTtl === '1h' ✅
  • explicit opt-out (modelOptions: { promptCache: false, promptCacheTtl: '1h' }) → both dropped, so a disabled cache never carries a TTL ✅

Deliberately keeping the TTL assignment inside the enablePromptCache === true block: emitting a TTL while caching is off would be semantically wrong (and could cause the SDK to enable caching the user didn't ask for). b80531e8d; full config.spec.ts (95) + tsc green.

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b80531e8dc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

const openRouter: SettingsConfiguration = [
...openAI,
anthropic.promptCache,
anthropic.promptCacheTtl,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Add the TTL control to auto-detected OpenRouter custom endpoints

This only adds promptCacheTtl to the built-in OpenRouter settings, but the auto-detected custom OpenRouter path is driven by api/server/services/Config/loadCustomConfig.js, where addOpenRouterDefaults still synthesizes customParams.paramDefinitions with only { key: 'promptCache' }. For existing custom/proxy OpenRouter configs whose name or baseURL matches OpenRouter, the UI will therefore expose the cache switch but not the new TTL dropdown even though the schema/backend now support it; include the TTL definition in that injected default set too.

Useful? React with 👍 / 👎.

Comment on lines +229 to +230
if (systemOptions.promptCacheTtl != null) {
(requestOptions as Record<string, unknown>).promptCacheTtl = systemOptions.promptCacheTtl;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Clear Anthropic TTL when prompt caching is dropped

When an Anthropic endpoint config uses dropParams: ['promptCache'] and the conversation/preset still has promptCacheTtl, this assignment runs before the drop-params loop. That loop later deletes only requestOptions.promptCache, leaving promptCacheTtl in llmConfig without caching enabled; in the admin opt-out path, the request can still carry a TTL (or let the SDK infer cache writes) even though prompt caching was dropped. Also clear promptCacheTtl when promptCache is dropped.

Useful? React with 👍 / 👎.

Comment on lines +117 to +118
promptCacheTtl:
options.modelOptions?.promptCacheTtl ?? anthropicSettings.promptCacheTtl.default,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Apply Anthropic custom TTL defaults

With this new setting, admins can configure custom/native Anthropic endpoints with customParams.paramDefinitions: [{ key: 'promptCacheTtl', default: '5m' }]; initializeCustom passes those values as defaultParams, but getLLMConfig snapshots the TTL only from modelOptions here and the later defaultParams/addParams loops do not handle promptCacheTtl. In that supported custom-params path, leaving the dropdown unset still sends no TTL and falls back to the SDK's 1h default instead of the configured 5m default.

Useful? React with 👍 / 👎.

type: 'enum',
default: anthropicSettings.promptCacheTtl.default,
options: ['5m', '1h'],
component: 'dropdown',

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Persist custom-endpoint TTL dropdown selections

For custom endpoints, parseCustomParams overwrites these definitions to optionType: 'custom', and DynamicDropdown's custom branch only updates local state before returning instead of calling setOption. So if an admin explicitly exposes promptCacheTtl on a custom Anthropic/OpenRouter endpoint, the new dropdown renders but user selections are never written into the conversation/model parameters, and the request keeps using the default TTL. Either avoid exposing this as a custom dropdown or make the dropdown persist custom values like the other controls.

Useful? React with 👍 / 👎.

Comment on lines +514 to +515
if (typedData.promptCache !== true) {
typedData.promptCacheTtl = undefined;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Gate Bedrock 1h TTL to models that support it

For Bedrock, this only drops the TTL when caching is off, so any cache-capable Bedrock model keeps a selected promptCacheTtl; that includes Claude 4.6/4/3.x and Nova because promptCache is defaulted on above. The AWS Bedrock docs list 1-hour TTL only for Claude Opus/Sonnet/Haiku 4.5, with other listed Claude models limited to 5 minutes, so selecting 1h for those models can send an unsupported cache TTL and fail the request instead of falling back safely.

Useful? React with 👍 / 👎.

…ex review)

dropParams: ['promptCache'] deleted requestOptions.promptCache but left
promptCacheTtl behind, so the admin opt-out path could still carry a TTL
on a request with caching disabled. Clear the TTL alongside promptCache.
@danny-avila

danny-avila commented Jun 18, 2026 •

Copy link
Copy Markdown
Collaborator Author

Round 3 — one fix, the rest are scoped out or false positives. Details:

P2 — Clear Anthropic TTL when prompt caching is dropped (anthropic/llm.ts:230). ✅ Fixed in 35f5e15b4. dropParams: ['promptCache'] deleted requestOptions.promptCache but left promptCacheTtl behind. Added a clear-TTL-alongside-promptCache step after the dropParams loop, plus a regression test (dropParams: ['promptCache'] → both promptCache and promptCacheTtl undefined). anthropic suite (100) + tsc + lint green.

P2 — Gate Bedrock 1h TTL to models that support it (bedrock.ts:515). Re-raise of the round-1 item — same disposition. This is a deliberate design decision: the TTL is tied to promptCache, not gated by a per-model allowlist. On the SDK side the Bedrock cache point is applied solely on promptCache === true with no model gating, and Bedrock gracefully downgrades an unsupported ttl rather than rejecting it — verified live on 5-minute-only Claude models (Sonnet/Opus 4.6 → HTTP 200) during agents#249. A maintained model whitelist would drift against AWS's model list and duplicate gating in the wrong layer; the doc's "supported models" table lists where 1h is honored, not where the field is rejected.

The remaining three are all about the custom / admin-configured endpoint path, which I'm intentionally keeping out of scope for this PR (the built-in Anthropic/Bedrock/OpenRouter UI works and is tested):

P2 — Persist custom-endpoint TTL dropdown selections (parameterSettings.ts:418). Correct observation, but pre-existing and generic: DynamicDropdown's optionType === Custom branch is a // TODO that updates local state and returns without setOption, so it affects every custom dropdown, not this control. DynamicSwitch (what promptCache uses) does call setOption, which is why the switch persists on custom endpoints and dropdowns don't. Fixing the shared component is a separate change with its own blast radius across other custom dropdowns — better as a dedicated PR than bundled here.

P2 — Add the TTL control to auto-detected OpenRouter custom endpoints (parameterSettings.ts:830). Deliberately not doing this — it's blocked by the above: injecting promptCacheTtl into addOpenRouterDefaults' paramDefinitions would render a dropdown that silently doesn't persist (the custom-DynamicDropdown gap). Exposing a non-working control is worse than not exposing it; this should land together with the DynamicDropdown fix.

P2 — Apply Anthropic custom TTL defaults (anthropic/llm.ts:118). At parity with promptCache: in this file neither promptCache nor promptCacheTtl is threaded through the defaultParams/addParams loops (only knownAnthropicParams are, and neither is in that set) — both are resolved purely from modelOptions as systemOptions. So an admin paramDefinitions default for promptCacheTtl is handled exactly like one for promptCache. Threading only the TTL would make it diverge from promptCache; bringing both to admin-default parity on the native Anthropic endpoint is a separate, intentional change.

@danny-avila

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 35f5e15b4d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +117 to +118
promptCacheTtl:
options.modelOptions?.promptCacheTtl ?? anthropicSettings.promptCacheTtl.default,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Honor Anthropic promptCacheTtl defaults

For native Anthropic custom endpoints, buildAnthropicCustomConfig passes customParams.paramDefinitions as defaultParams, but this value is computed only from modelOptions; the later defaultParams/addParams loops only handle web_search and knownAnthropicParams, so an admin-configured default like { key: 'promptCacheTtl', default: '5m' } is silently ignored and requests continue using the SDK's 1h default unless the user manually selects the dropdown. Thread promptCacheTtl through the same default/add handling used for OpenRouter before applying it to requestOptions.

Useful? React with 👍 / 👎.

description: 'com_endpoint_anthropic_prompt_cache_ttl',
descriptionCode: true,
type: 'enum',
default: anthropicSettings.promptCacheTtl.default,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve unset TTL in custom overrides

When a custom endpoint includes promptCacheTtl in customParams.paramDefinitions just to relabel or expose this setting, loadCustomConfig merges this definition and then validateSettingDefinitions fills any dropdown without a default with options[0], so the intended unset default becomes '5m'. That default is later extracted and sent on OpenRouter/custom requests, unexpectedly opting those endpoints into the legacy 5-minute cache instead of leaving the SDK/provider 1-hour default in effect.

Useful? React with 👍 / 👎.

@danny-avila

Copy link
Copy Markdown
Collaborator Author

Round 4 — both findings are in the custom/admin-configured endpoint path that I'm intentionally keeping out of scope for this PR (the built-in Anthropic/Bedrock/OpenRouter UI works and is tested). No code changes.

P2 — Honor Anthropic promptCacheTtl defaults (anthropic/llm.ts:118). Same disposition as rounds 1 and 3: at parity with promptCache. Neither promptCache nor promptCacheTtl is threaded through the Anthropic defaultParams/addParams loops (only knownAnthropicParams are, and neither is in that set) — both resolve from modelOptions as systemOptions. An admin paramDefinitions default for the TTL is therefore handled exactly like one for promptCache. Bringing both to admin-default parity on the native Anthropic endpoint is a separate, intentional change.

P2 — Preserve unset TTL in custom overrides (parameterSettings.ts:416). Confirmed accurate, but it's a generic property of the custom-param system, not specific to this control. validateSettingDefinitions forces any defaultless Dropdown to options[0] (generate.ts:397-399), and it runs only on customParams.paramDefinitions (loadCustomConfig.js:219) — never on the built-in settings, where default: undefined is preserved and the SDK's 1h default stays in effect. So this only manifests if an admin explicitly adds promptCacheTtl to a custom endpoint's paramDefinitions (the same opt-in custom path as the round-3 items), and it affects every defaultless custom dropdown identically.

The custom-endpoint cluster (this + round-3 A/C/D) is best handled as a dedicated follow-up that fixes the shared DynamicDropdown custom-persistence // TODO and decides admin-default threading for system params across endpoints — rather than bundled here. Considering the in-scope review converged: the standard-endpoint feature is complete with regression coverage across bedrock.spec (268), anthropic/llm.spec (100), and openai/{llm,config}.spec.

@danny-avila
danny-avila merged commit 268fcbb into dev Jun 18, 2026
31 checks passed
@danny-avila
danny-avila deleted the feat/prompt-cache-ttl branch June 18, 2026 20:36
fuuuzzy pushed a commit to fuuuzzy/LibreChat that referenced this pull request Jul 7, 2026
…LibreChat-AI#13835)

* 🕐 feat: Add promptCacheTtl model parameter for 1h/5m cache duration

Adds a user-configurable `promptCacheTtl` parameter (dropdown: 5m | 1h)
alongside the existing `promptCache` toggle for Anthropic, Bedrock, and
OpenRouter endpoints. Default is undefined so the agents SDK applies its
own default (1h), letting users opt down to the legacy 5m TTL.

- data-provider: schema, parameterSettings dropdown, types, bedrock picks
- data-schemas: convo/preset types + mongoose defaults
- api: thread promptCacheTtl into anthropic + openai(OpenRouter) llmConfig
- i18n: en translation keys for label/description/default placeholder
- tests: anthropic llm.spec coverage for set + unset cases

* 🔧 fix: Tie Bedrock promptCacheTtl to promptCache + thread OpenRouter TTL params (Codex review)

- bedrock.ts: clear promptCacheTtl whenever promptCache is off/unsupported,
  so an unsupported 1h is never sent on a non-caching Bedrock request
- openai/llm.ts: resolve promptCacheTtl through the same defaultParams/
  addParams/dropParams machinery as promptCache (via promptCacheTtlValue)
  so OpenRouter custom endpoints can configure/override/drop it
- tests: bedrock TTL-tied-to-promptCache cases; OpenRouter TTL default/add/drop

* 🎨 style: Sort imports in openai/llm.spec.ts (CI sort-imports)

* ✅ test: Prove OpenRouter TTL-only selection honors promptCache default (Codex review)

OPENROUTER_DEFAULT_PARAMS injects promptCache:true into defaultParams, so a
TTL-only dropdown selection (promptCacheTtl set, promptCache switch untouched)
still resolves caching on and forwards the TTL. Add regression tests via the
real getOpenAIConfig entry point: TTL-only -> promptCache+TTL both set;
explicit promptCache:false -> both dropped.

* 🔖 chore: Bump librechat-data-provider to 0.8.506

* 🔧 fix: Drop Anthropic promptCacheTtl when promptCache is dropped (Codex review)

dropParams: ['promptCache'] deleted requestOptions.promptCache but left
promptCacheTtl behind, so the admin opt-out path could still carry a TTL
on a request with caching disabled. Clear the TTL alongside promptCache.
ThomasVuNguyen pushed a commit to ThomasVuNguyen/LibreChat that referenced this pull request Jul 15, 2026
…LibreChat-AI#13835)

* 🕐 feat: Add promptCacheTtl model parameter for 1h/5m cache duration

Adds a user-configurable `promptCacheTtl` parameter (dropdown: 5m | 1h)
alongside the existing `promptCache` toggle for Anthropic, Bedrock, and
OpenRouter endpoints. Default is undefined so the agents SDK applies its
own default (1h), letting users opt down to the legacy 5m TTL.

- data-provider: schema, parameterSettings dropdown, types, bedrock picks
- data-schemas: convo/preset types + mongoose defaults
- api: thread promptCacheTtl into anthropic + openai(OpenRouter) llmConfig
- i18n: en translation keys for label/description/default placeholder
- tests: anthropic llm.spec coverage for set + unset cases

* 🔧 fix: Tie Bedrock promptCacheTtl to promptCache + thread OpenRouter TTL params (Codex review)

- bedrock.ts: clear promptCacheTtl whenever promptCache is off/unsupported,
  so an unsupported 1h is never sent on a non-caching Bedrock request
- openai/llm.ts: resolve promptCacheTtl through the same defaultParams/
  addParams/dropParams machinery as promptCache (via promptCacheTtlValue)
  so OpenRouter custom endpoints can configure/override/drop it
- tests: bedrock TTL-tied-to-promptCache cases; OpenRouter TTL default/add/drop

* 🎨 style: Sort imports in openai/llm.spec.ts (CI sort-imports)

* ✅ test: Prove OpenRouter TTL-only selection honors promptCache default (Codex review)

OPENROUTER_DEFAULT_PARAMS injects promptCache:true into defaultParams, so a
TTL-only dropdown selection (promptCacheTtl set, promptCache switch untouched)
still resolves caching on and forwards the TTL. Add regression tests via the
real getOpenAIConfig entry point: TTL-only -> promptCache+TTL both set;
explicit promptCache:false -> both dropped.

* 🔖 chore: Bump librechat-data-provider to 0.8.506

* 🔧 fix: Drop Anthropic promptCacheTtl when promptCache is dropped (Codex review)

dropParams: ['promptCache'] deleted requestOptions.promptCache but left
promptCacheTtl behind, so the admin opt-out path could still carry a TTL
on a request with caching disabled. Clear the TTL alongside promptCache.
LogicalAbsurd pushed a commit to LogicalAbsurd/LibreChat that referenced this pull request Aug 27, 2026
…LibreChat-AI#13835)

* 🕐 feat: Add promptCacheTtl model parameter for 1h/5m cache duration

Adds a user-configurable `promptCacheTtl` parameter (dropdown: 5m | 1h)
alongside the existing `promptCache` toggle for Anthropic, Bedrock, and
OpenRouter endpoints. Default is undefined so the agents SDK applies its
own default (1h), letting users opt down to the legacy 5m TTL.

- data-provider: schema, parameterSettings dropdown, types, bedrock picks
- data-schemas: convo/preset types + mongoose defaults
- api: thread promptCacheTtl into anthropic + openai(OpenRouter) llmConfig
- i18n: en translation keys for label/description/default placeholder
- tests: anthropic llm.spec coverage for set + unset cases

* 🔧 fix: Tie Bedrock promptCacheTtl to promptCache + thread OpenRouter TTL params (Codex review)

- bedrock.ts: clear promptCacheTtl whenever promptCache is off/unsupported,
  so an unsupported 1h is never sent on a non-caching Bedrock request
- openai/llm.ts: resolve promptCacheTtl through the same defaultParams/
  addParams/dropParams machinery as promptCache (via promptCacheTtlValue)
  so OpenRouter custom endpoints can configure/override/drop it
- tests: bedrock TTL-tied-to-promptCache cases; OpenRouter TTL default/add/drop

* 🎨 style: Sort imports in openai/llm.spec.ts (CI sort-imports)

* ✅ test: Prove OpenRouter TTL-only selection honors promptCache default (Codex review)

OPENROUTER_DEFAULT_PARAMS injects promptCache:true into defaultParams, so a
TTL-only dropdown selection (promptCacheTtl set, promptCache switch untouched)
still resolves caching on and forwards the TTL. Add regression tests via the
real getOpenAIConfig entry point: TTL-only -> promptCache+TTL both set;
explicit promptCache:false -> both dropped.

* 🔖 chore: Bump librechat-data-provider to 0.8.506

* 🔧 fix: Drop Anthropic promptCacheTtl when promptCache is dropped (Codex review)

dropParams: ['promptCache'] deleted requestOptions.promptCache but left
promptCacheTtl behind, so the admin opt-out path could still carry a TTL
on a request with caching disabled. Clear the TTL alongside promptCache.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: support 1-hour Bedrock prompt cache TTL (cost optimization)

1 participant