Skip to content

fix(usage): stop charging L3 reuse cache-hit tokens as paid cost (Issue #548) - #555

Merged
radianceded merged 1 commit into
Gnosil:mainfrom
RyoKusnadi:fix/usage-l3-cache-hit-cost
Nov 5, 2026
Merged

radianceded merged 1 commit into
Gnosil:mainfrom
RyoKusnadi:fix/usage-l3-cache-hit-cost

Conversation

@RyoKusnadi

Copy link
Copy Markdown
Contributor

Summary

Problem: semantix usage and the dashboard show a negative paid cost and savings above 100% when the log has gateway L3 reuse turns. The issue's example (1000 in, 800 cache hit, L3 reuse) gives paid=-0.00016 and a 159% savings rate. Mixed logs also under-report what the upstream turns really cost.

How it happens: when the gateway answers from a stored result, it writes a usage event with l3_reuse: true and cache_hit_tokens set to the size of the reused answer (gateway/pipeline.go, the recordUsage call in the L3 branch). That is a local reply, not a provider prefix-cache hit. Any semantix usage run on a log with such a turn hits it.

Cause: in kernel/usage/usage.go Summarize, billed input is TokensIn - CacheHitTokens - l3In, and the hit bucket is charged CacheHitTokens * hit price. For an L3 turn the hit tokens are already inside l3In, so they are subtracted twice from the miss bucket and still charged at the hit price. Each L3 turn adds a negative -(hits)*(miss-hit)/1e6.

Fix: sum the cache-hit tokens of L3 turns (l3Hit) and charge only CacheHitTokens - l3Hit at the hit price and subtract only that from billed input. L3 turns now leave every billed bucket. I did not clamp the total at zero, as the issue asks, so mixed logs stay correct. The CacheHitTokens figure in the summary is unchanged.

Scope

kernel/usage only (Summarize). InjectROI already counts L3 savings from l3In/l3Out and is not affected. Per-provider stats only count Exact events and L3 events are not exact, so they are not affected either.

Validation

  • Before the fix: TestSummarizeL3OnlyCostsZero fails with paid = -0.00015999999999999999, want 0, and TestSummarizeL3MixedKeepsUpstreamCost fails with paid = 4.400000000000001e-05, want 0.000204.

  • After: both pass.

  • Ran: go vet ./kernel/usage/, go test ./kernel/usage/ -race, go test ./cmd/semantix/, gofmt -l kernel/usage, git diff --check.

  • Not run: go vet ./... and go test ./... -race for the whole repo.

  • go vet ./... (run on ./kernel/usage/ only)

  • go test ./... -race (run on ./kernel/usage/ only, plus ./cmd/semantix/ without -race)

  • git diff --check

  • Additional checks are documented below, or are not applicable.

Compatibility and data impact

None for the log format. CostPaidUSD, SavingsUSD and SavingsRate change for logs that contain L3 reuse turns, to the correct values.

Security and privacy

None.

Evidence

Issue example (one L3 turn, 1000 in, 800 hit): paid -0.00016 and rate 1.59 before; paid 0 and rate 1 after.

Related issues

Closes #548

Checklist

  • The change is focused and does not include unrelated work.
  • Tests or documentation cover the changed behavior.
  • User-facing behavior and examples are updated where needed.
  • No credentials, private source code, personal data, or sensitive local paths are included.
  • Wire-contract changes are additive and synchronized with docs/events.md.

#548)

Signed-off-by: Ryo Kusnadi <ryokusnadi@gmail.com>

@radianceded radianceded left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@radianceded
radianceded merged commit f19a313 into Gnosil:main Nov 5, 2026
2 checks passed
@RyoKusnadi
RyoKusnadi deleted the fix/usage-l3-cache-hit-cost branch November 5, 2026 12:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug][P2] L3-only usage produces negative paid cost and savings above 100%

2 participants