AGENTMEMORY_DROP_STALE_INDEX=true in .env has no effect; vectors persist at 2048-dim even with EMBEDDING_PROVIDER=local #456

Environment

• agentmemory v0.9.17 (global npm install)
• @xenova/transformers v2.17.2 (dependency of agentmemory)
• Node v24.13.1 via nvm
• macOS 26.4.1
• ~/.agentmemory/.env: EMBEDDING_PROVIDER=local, AGENTMEMORY_DROP_STALE_INDEX=true
• npm install -g @xenova/transformers also executed

Bug 1: AGENTMEMORY_DROP_STALE_INDEX=true in .env does NOT work

The dimension guard code reads process.env["AGENTMEMORY_DROP_STALE_INDEX"] directly (line ~18417 in the dist bundle). However, .env file values are loaded by loadEnvFile() and merged into a getMergedEnv() object — they are never written to process.env.

As a result, setting AGENTMEMORY_DROP_STALE_INDEX=true in ~/.agentmemory/.env has zero effect. It must be exported as a real shell environment variable to work.

// src/state/index-persistence.ts or equivalent
if (process.env["AGENTMEMORY_DROP_STALE_INDEX"] === "true")  // ← reads process.env, NOT getMergedEnv()
    console.warn("...discarding...");
else
    throw new Error("Refusing to start...");

Fix: the dimension guard should read from getMergedEnv() instead of process.env directly, consistent with how EMBEDDING_PROVIDER and other config values are read.

Bug 2: Vectors written at 2048-dim despite EMBEDDING_PROVIDER=local declaring 384-dim

What happened

1. Set EMBEDDING_PROVIDER=local in .env before first run
2. Started agentmemory, ran agentmemory demo — it worked
3. Wrote 2 new memories from Claude Code (via the agentmemory plugin + REST API)
4. Restarted agentmemory → fatal error: "persisted vector index has 5/5 vectors at 2048-dim, provider declares 384"

Full error message (first occurrence, 6 vectors)

[agentmemory] Fatal: Error: [agentmemory] Refusing to start: persisted vector index has 6 of 6 vectors with the wrong dimension. Active provider (local) declares 384; dimensions seen on disk: 2048. First mismatched obsIds: obs_mp9k4w42_996de1400f2d (dim=2048), obs_mp9k4w8b_9bc3120f1ac1 (dim=2048), obs_mp9k4w8j_36bbbff5a447 (dim=2048), obs_mp9k4w8u_e2514a0623f9 (dim=2048), obs_mp9k4w93_4cb2c3e5f796 (dim=2048). Loading would silently corrupt search (cross-dimension cosine returns 0). Choose one:
  - Re-embed the existing index against the new provider, then start.
  - Set AGENTMEMORY_DROP_STALE_INDEX=true to discard the persisted vectors and rebuild from live observations.
  - Switch the embedding provider back to the one that wrote the index.

Full error message (second occurrence, 5 vectors, after DROP_STALE_INDEX passed via shell env)

[agentmemory] Fatal: Error: [agentmemory] Refusing to start: persisted vector index has 5 of 5 vectors with the wrong dimension. Active provider (local) declares 384; dimensions seen on disk: 2048. First mismatched obsIds: obs_mp9llgfb_66fa67bd6dc6 (dim=2048), mem_mp9llwis_f0deca7b7f4b (dim=2048), obs_mp9llwk8_879aec2dd846 (dim=2048), mem_mp9lnc9a_3ca6d5dd667a (dim=2048), obs_mp9lncz1_856fb20013d0 (dim=2048). Loading would silently corrupt search (cross-dimension cosine returns 0).

Evidence

• all-MiniLM-L6-v2 model is confirmed working correctly — returns 384-dim vectors when tested directly from the same Node context and working directory
• Startup log correctly shows Embedding provider: local (384 dims) — so EMBEDDING_PROVIDER=local IS being read
• withDimensionGuard wrapper exists on the LocalEmbeddingProvider (checks v.length !== expected and throws)
• vectorIndexAddGuarded has a dimension mismatch check before vi.add()
• Yet all persisted vectors are 2048-dim, with fresh obsIds from the current session

Notable obsId patterns

• First batch (demo): obs_mp9k4w42_*, obs_mp9k4w8b_* — sessionIds reference demo_mp9k4w3v_*
• Second batch (Claude Code memories): obs_mp9llgfb_*, mem_mp9llwis_*, obs_mp9llwk8_*, mem_mp9lnc9a_*, obs_mp9lncz1_*
• Both use obs_ and mem_ prefixes, all at 2048-dim

Code paths traced

• detectEmbeddingProvider() (tools-registry): correctly returns "local" when EMBEDDING_PROVIDER=local
• createEmbeddingProvider() (main bundle): switch-case routes "local" to LocalEmbeddingProvider with 384 dims
• withDimensionGuard(): wraps embed to throw on dimension mismatch
• vectorIndexAddGuarded(): catches dimension mismatch and embed failures, returns false without adding
• VectorIndex.add(): simple Map.set(obsId, { embedding, sessionId })
• VectorIndex.validateDimensions(): iterates stored vectors, checks entry.embedding.length !== expected

Questions

1. Is there a second code path that creates an embedding provider outside of createEmbeddingProvider()?
2. Does the worker process environment differ from what getMergedEnv() sees (e.g., iii-engine spawns worker with modified env)?
3. Could @xenova/transformers dynamic import() fail silently in the worker context, with a fallback to another provider?
4. Are vectors persisted to disk via a path that bypasses the runtime dimension guard?

Workaround

# Stop agentmemory daemon first

# Delete all persisted data
rm -rf ~/dev/agentmemory/data/

# Start fresh with DROP_STALE_INDEX as shell env var
AGENTMEMORY_DROP_STALE_INDEX=true npx @agentmemory/agentmemory

Note: AGENTMEMORY_DROP_STALE_INDEX=true must be passed as a shell environment variable, NOT just written in ~/.agentmemory/.env.