Thrift — cost-first memory for AI agent fleets (MCP server + proxy, OSS)

[YohadH]

Every AI agent has a "memory" file with everything it needs to know. On
every single run, the agent reloads the whole file — including the ~99%
that's irrelevant to the current task. Run a fleet of agents all day and
you're paying, again and again, for the same dead context. It's usually the
single biggest line on the token bill.

What Thrift does

Instead of "load everything," Thrift loads only the memory relevant to the
current task, under a fixed token budget. The agent says "I'm fixing a
bug in the proxy" and gets back only what's related — Thrift filters out
the irrelevant rather than stuffing in whatever fits the budget.

And it proves the savings: every load writes a receipt — what you'd have
paid loading everything vs. what you actually paid. Not "trust me, you
saved" — a measured number.

MCP tools

Tool	What it does
recall(agentId, tokenBudget, task)	Returns only the relevant memories under budget + a receipt { injectedTokens, baselineTokens, savedTokens }
remember(scope, text, agentId?, sessionId?, tags?, pinned?)	Cheap write path. No mandatory LLM enrichment.
search_memory(agentId, task?, tags?, limit?)	Search without a budget — for browsing / control panels

Install

{
  "mcpServers": {
    "thrift": { "command": "npx", "args": ["thrift-memory"] }
  }
}

Restart your MCP client (Claude Code / Cursor / Windsurf) — it auto-detects
recall, remember, search_memory. No code rewrite.

No code change? There's a proxy too

For agents that resend the full prompt every turn, run Thrift as a proxy and
change only base_url:

npx thrift-proxy \
  --upstream=https://api.anthropic.com/ \
  --port=8787 \
  --budget=4000 \
  --meter-path=~/.thrift/meter.jsonl

It also handles 429s automatically (retry + backoff, respects
Retry-After). Binds to [127.0.0.1](https://127.0.0.1/) by default since it forwards your real
API key upstream.

See the savings

npx thrift-panel serve
# http://127.0.0.1:8585/

No DB, no build step — reads the same files the MCP server and proxy write.

Results

Dogfooded on my own fleet of 24 production agents running daily — ~87%
cut in memory-related tokens.

• npm: https://npmjs.com/package/thrift-memory
• GitHub: https://github.com/YohadH/thrift-memory
• License: Apache-2.0

Posting here because this problem feels MCP-shaped specifically — curious
whether others building on MCP have hit the same "reload everything every
turn" cost, and how you're handling it today. Feedback and issues very
welcome.

[Brandzp]

Use it on my company working great !
thank you !!!