Skip to content

Bound the wire-redaction MappingStore retention #57

Description

@amondnet

Follow-up from AI review on #56 (gemini-code-assist, cubic-dev-ai, greptile).

Problem

MappingStore (crates/honmoon-core/src/secret_tokenizer/mod.rs) accumulates every placeholder→secret mapping for the lifetime of the gateway process and never evicts. Two consequences:

  1. Unbounded memory growth — a long-running or high-traffic gateway with --redact-secrets keeps every historical secret mapping in memory, risking OOM (gemini secret_tokenizer:386, cubic secret_tokenizer:381).
  2. Per-response full clone — once any mapping exists, handle_response calls snapshot() which clones the entire map on every response, including clean responses with no placeholders (greptile mitm.rs:695). This is a deliberate tradeoff (the clone avoids holding the mutex across body streaming, see the comment at mitm.rs handle_response), but it scales with the unbounded store.

Why not a naive fix

A plain LRU/TTL eviction would silently drop mappings still needed for detokenization, leaking unresolved placeholders or breaking the cache-stable determinism guarantee (#20). As gemini notes, retention must be tied to a session/request lifecycle, not a blind cache policy.

Proposed direction

  • Design a retention policy scoped to conversation/session lifetime (or an explicit capacity + correctness-preserving eviction) that keeps detokenization correct.
  • Once retention is bounded, the per-response snapshot() clone cost is bounded too; optionally revisit whether a cheaper reference/Arc sharing scheme can replace the clone without holding the lock during streaming.

Deferred from #56 as out of scope for that PR; the current behavior is correct for typical (small, stable secret-set) sessions.

Metadata

Metadata

Assignees

No one assigned

    Labels

    effort:LLarge — about a weektype:refactorCode refactoring without behavior change

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions