Skip to content

ai: honour response_cache for local reviews - #493

Open
shardulsdk-mpiric wants to merge 1 commit into
sashiko-dev:mainfrom
shardulsdk-mpiric:mpiric/local-response-cache
Open

shardulsdk-mpiric wants to merge 1 commit into
sashiko-dev:mainfrom
shardulsdk-mpiric:mpiric/local-response-cache

Conversation

@shardulsdk-mpiric

Copy link
Copy Markdown
Contributor

AiSettings carries response_cache, but only the daemon ever consulted
it. sashiko review builds its provider with create_provider_from_ai,
which is uncached, so turning the setting on did nothing on that path and
there was nothing in the output to say so.

create_provider_cached cannot be reused there, since it derives the cache
location from settings.database.url and a local review holds AiSettings
alone with no database beside it. This adds create_provider_cached_from_ai,
which places the cache under the XDG data directory already used for the
prompt bundle and lets SASHIKO_RESPONSE_CACHE override it. The helper that
resolves that directory is now shared rather than copied. response_cache
still defaults to false, so nothing changes unless it is switched on.

It matters most on a retry. A run that fails part way re-issues every
earlier call, and with no cache each of those is paid for again. One such
retry after a late-stage failure cost me $3.33 against a hosted endpoint,
which is what sent me looking at this.

make check-all passes, and there is a test for the path resolution
covering both the environment override and the XDG default. I am relying on
#480 having made the cache key safe across provider configurations rather
than having exercised that myself.

AiSettings carries response_cache, but only the daemon ever consulted
it. sashiko review builds its provider with create_provider_from_ai,
which is uncached, so setting response_cache did nothing on that path
and gave no sign that it had not taken effect.

create_provider_cached cannot be used there. It derives the cache
location from settings.database.url, and a local review holds AiSettings
alone with no database beside it.

Add create_provider_cached_from_ai, which wraps the provider when
response_cache is set and places the cache under the XDG data directory
already used for the prompt bundle, overridable with
SASHIKO_RESPONSE_CACHE. The helper that resolves that directory is
shared rather than copied. response_cache still defaults to false, so
nothing changes unless it is switched on.

This matters most when a review is retried. A run that fails part way
re-issues every earlier call, and without a cache each of those is paid
for again.

Assisted-by: Claude:claude-opus-5
Signed-off-by: Shardul Bankar <shardul.b@mpiricsoftware.com>
@kees

kees commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

I could use this! :)

I think it should use a common method, though. This is kind of duplicating code from the daemon. What about consolidating it like attached?

Also, I think the original PR suffered from creating two caches since the stdio workers under the daemon would end up creating the non-daemon cache location.
0001-ai-honour-response_cache-for-local-reviews.patch

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants