You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
No way to rebuild the search index without restarting: rebuildIndex only runs when the index is empty #1179
rebuildIndex(kv) is only reachable at boot, and only when bm25Index.size === 0. Once an index exists there is no supported way to rebuild it, which leaves three situations with no clean answer:
A snapshot that turns out to be stale or partial. Restarting reloads it (it is not empty), so the gap survives every restart.
Deleted observations. Their entries stay in BM25 and the vector index until something clears them; search can rank ids that no longer resolve.
Restore from backup. Backups reasonably cover the KV store; the index is derived, so after a restore the natural move is "rebuild it" — and there is no way to ask.
Deleting the persisted index to force the empty-index path also throws away the vectors, so the next boot re-embeds the entire corpus with the LLM/embedding provider — an hour of GPU on a 12k-observation store — rather than only the missing ones.
What I'm running locally
A POST /agentmemory/reindex that:
rebuilds BM25 in full from KV into a fresh SearchIndex, then restoreFrom()s it, so search is never empty mid-rebuild and rows for deleted observations disappear
takes vectors: "missing" | "all" | "none" — missing (default) embeds only ids absent from the vector index and drops vectors whose doc is gone, which turns a 4-minute job into a 20-second one on repeat runs
runs off the request path and returns 202 immediately, with ?status=true for progress, because the engine kills an HTTP invocation at default_timeout (180s) long before a full rebuild finishes
persists once at the end
On our store that is 12 400 docs and 9 402 embeddings in 255s, and it is what a weekly timer calls to keep the index honest.
Would you take a PR for this? I'd follow whatever shape you prefer for the route and status reporting — the two design points I care about are the fresh-index swap (so a failed rebuild cannot empty a working index) and missing as the default for vectors (so it is cheap enough to run on a schedule).
Activity