feat(vector): auto-index observations into vector store | 向量索引自动集成#228
feat(vector): auto-index observations into vector store | 向量索引自动集成#228mechanic-Q wants to merge 1 commit intorohitg00:mainfrom
Conversation
When an EmbeddingProvider is configured, automatically add compressed observations to the VectorIndex during the observe lifecycle (synthetic compression path). This enables hybrid BM25+Vector search without requiring manual index building. The auto-indexing runs after BM25 indexing within a try-catch so embedding failures don't block observation capture. Combined with configurable embedding model (AGENTMEMORY_LOCAL_EMBEDDING_MODEL), users can enable multilingual vector search by setting the model to bge-m3 or multilingual-e5. Passes vectorIndex and embeddingProvider as optional params to registerObserveFunction — fully backwards compatible. 当配置了嵌入模型时，自动将压缩后的观察结果添加到向量索引， 实现 BM25+向量混合搜索。嵌入失败不影响观察捕获。 配合可配置嵌入模型使用，设置 AGENTMEMORY_LOCAL_EMBEDDING_MODEL=bge-m3 即可启用中文等多语言语义搜索。
|
@mechanic-Q is attempting to deploy a commit to the rohitg00's projects Team on Vercel. A member of the Team first needs to authorize it. |
📝 WalkthroughWalkthroughUpdated ChangesVector Indexing Integration
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Review rate limit: 4/8 reviews remaining, refill in 29 minutes and 56 seconds.Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@src/functions/observe.ts`:
- Around line 244-255: The current code awaits embeddingProvider.embed(...)
inside the observe hook which blocks the response and stream triggers; change
this to a fire-and-forget pattern by removing the await and chaining a promise
handler: call embeddingProvider.embed(narrative).then(vec =>
vectorIndex.add(obsId, payload.sessionId, vec)).catch(err => logger.warn("Vector
auto-index failed for observation", { obsId, error: err instanceof Error ?
err.message : String(err) })); keep the surrounding conditional (vectorIndex &&
embeddingProvider) and do not reintroduce await so embedding work runs
asynchronously and does not block stream::set or the hook response.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: aad77881-b4cf-4f03-8c28-df9a87ed4403
📒 Files selected for processing (2)
src/functions/observe.tssrc/index.ts
| if (vectorIndex && embeddingProvider) { | ||
| try { | ||
| const narrative = (synthetic.title || "") + " " + (synthetic.narrative || ""); | ||
| const vec = await embeddingProvider.embed(narrative); | ||
| vectorIndex.add(obsId, payload.sessionId, vec); | ||
| } catch (err) { | ||
| logger.warn("Vector auto-index failed for observation", { | ||
| obsId, | ||
| error: err instanceof Error ? err.message : String(err), | ||
| }); | ||
| } | ||
| } |
There was a problem hiding this comment.
await embeddingProvider.embed() is on the critical path — blocks the hook response and all downstream stream triggers.
The observe hook returns only after the embedding completes (line 247). ML inference (e.g., Xenova/bge-m3) adds 100 ms–2 s to every tool-use observation for anyone who opts into an embedding provider. This also contradicts the PR's stated design ("failures are logged and do not prevent observation capture"), because successful embeddings still block the hook response and the stream::set triggers at lines 256–277.
The existing pattern for async ML work in this same function is fire-and-forget via sdk.triggerVoid (see the vision-embed dispatch at lines 146–152). The simplest fix consistent with that pattern is dropping the await and using a chained .catch() so embedding errors are still swallowed:
⚡ Proposed fix — fire-and-forget embedding (non-blocking)
- if (vectorIndex && embeddingProvider) {
- try {
- const narrative = (synthetic.title || "") + " " + (synthetic.narrative || "");
- const vec = await embeddingProvider.embed(narrative);
- vectorIndex.add(obsId, payload.sessionId, vec);
- } catch (err) {
- logger.warn("Vector auto-index failed for observation", {
- obsId,
- error: err instanceof Error ? err.message : String(err),
- });
- }
- }
+ if (vectorIndex && embeddingProvider) {
+ const narrative = (synthetic.title || "") + " " + (synthetic.narrative || "");
+ embeddingProvider.embed(narrative)
+ .then((vec) => vectorIndex.add(obsId, payload.sessionId, vec))
+ .catch((err) =>
+ logger.warn("Vector auto-index failed for observation", {
+ obsId,
+ error: err instanceof Error ? err.message : String(err),
+ }),
+ );
+ }📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if (vectorIndex && embeddingProvider) { | |
| try { | |
| const narrative = (synthetic.title || "") + " " + (synthetic.narrative || ""); | |
| const vec = await embeddingProvider.embed(narrative); | |
| vectorIndex.add(obsId, payload.sessionId, vec); | |
| } catch (err) { | |
| logger.warn("Vector auto-index failed for observation", { | |
| obsId, | |
| error: err instanceof Error ? err.message : String(err), | |
| }); | |
| } | |
| } | |
| if (vectorIndex && embeddingProvider) { | |
| const narrative = (synthetic.title || "") + " " + (synthetic.narrative || ""); | |
| embeddingProvider.embed(narrative) | |
| .then((vec) => vectorIndex.add(obsId, payload.sessionId, vec)) | |
| .catch((err) => | |
| logger.warn("Vector auto-index failed for observation", { | |
| obsId, | |
| error: err instanceof Error ? err.message : String(err), | |
| }), | |
| ); | |
| } |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/functions/observe.ts` around lines 244 - 255, The current code awaits
embeddingProvider.embed(...) inside the observe hook which blocks the response
and stream triggers; change this to a fire-and-forget pattern by removing the
await and chaining a promise handler: call
embeddingProvider.embed(narrative).then(vec => vectorIndex.add(obsId,
payload.sessionId, vec)).catch(err => logger.warn("Vector auto-index failed for
observation", { obsId, error: err instanceof Error ? err.message : String(err)
})); keep the surrounding conditional (vectorIndex && embeddingProvider) and do
not reintroduce await so embedding work runs asynchronously and does not block
stream::set or the hook response.
|
Hi, When AGENTMEMORY_AUTO_COMPRESS=true, mem::observe triggers mem::compress and does not run the new vectorIndex.add() code path, so the vector index stays empty even if an EmbeddingProvider is configured. Severity: action required | Category: correctness How to fix: Index vectors in mem::compress Agent prompt to fix - you can give this to your LLM of choice:
Spotted by Qodo code review - free for open-source projects. |
2 participants
Summary | 概述
Automatically add compressed observations to the VectorIndex during the observe lifecycle, enabling hybrid BM25+Vector search without manual index building.
在观察生命周期中自动将压缩记忆添加到向量索引，实现 BM25+向量混合搜索。
Motivation | 动机
The VectorIndex and EmbeddingProvider infrastructure exists but observations were only added to the BM25 index. The vector index remained empty unless manually populated. This meant the hybrid search (BM25 + Vector RRF fusion) couldn't leverage semantic similarity — only BM25 keyword matching was active. For non-English content (Chinese, multilingual), BM25 alone is insufficient because the tokenizer can't handle CJK text well. Vector search bridges this gap.
向量索引基础设施已存在，但观察结果仅被添加到 BM25 索引，向量索引保持为空。混合搜索无法利用语义相似度，仅靠 BM25 关键词匹配。对于中文等非英文内容，纯 BM25 效果差，向量搜索可以弥补这一差距。
Changes | 改动
src/functions/observe.ts: After synthetic compression and BM25 indexing, auto-embed the narrative and add to vector indexsrc/index.ts: PassvectorIndexandembeddingProvidertoregisterObserveFunctionCombined with PR #223 (configurable embedding)
# Enable multilingual hybrid search (BM25 + Vector) AGENTMEMORY_LOCAL_EMBEDDING_MODEL=Xenova/bge-m3This combination gives agentmemory full Chinese/multilingual semantic search capability — BM25 handles exact terms, vector handles meaning.
Backwards Compatibility | 向后兼容
New params are optional. When
vectorIndexorembeddingProvideris null/undefined (default), the auto-indexing is skipped. No behavior change for existing deployments.Summary by CodeRabbit
Release Notes