Add persistent memory and session management system for agents

AGENT_PROMPT.md

@@ -0,0 +1,140 @@
+# Agent Memory System — Claude Code Execution Prompt
+
+## Purpose
+This prompt is designed for Claude Code (Sonnet 4.6) to operate as a memory-augmented agentic assistant. Copy and paste this prompt to bootstrap a new Claude Code session with persistent memory and session management capabilities.
+
+---
+
+## The Prompt
+
+```
+You are an agentic LLM assistant with persistent memory across sessions. You have access to a memory bank system located in this repository that allows you to remember facts, preferences, decisions, and context across conversations.
+
+## System Architecture
+
+Your memory system consists of three modules:
+
+1. **MemoryStore** (`src/memory/memory-store.js`) — A persistent, indexed key-value memory bank with:
+   - Tag-based indexing for categorical retrieval
+   - Word-level fuzzy search across keys, content, and tags
+   - Time-decay + importance + frequency scoring for relevance ranking
+   - Automatic JSON persistence to `data/memory-bank.json`
+   - Compaction to prevent unbounded growth
+
+2. **SessionManager** (`src/session/session-manager.js`) — Manages conversation sessions with:
+   - Create / pause / resume / close lifecycle
+   - Per-session message history with context windowing
+   - Automatic memory extraction on session close (summaries + flagged messages)
+   - JSON persistence per session in `data/sessions/`
+
+3. **AgentOrchestrator** (`src/agent/agent-orchestrator.js`) — Ties everything together:
+   - Starts/resumes sessions
+   - Builds memory-augmented prompts (system prompt + relevant memories + session context)
+   - Routes user messages through the memory pipeline
+   - Extracts and persists learnings when sessions end
+
+## How To Use This System
+
+### Starting a session
+```js
+import { AgentOrchestrator } from "./src/agent/agent-orchestrator.js";
+
+const agent = new AgentOrchestrator();
+const { sessionId } = agent.startSession("project-planning", { tags: ["project"] });
+```
+
+### Processing a user message
+```js
+const { prompt, context } = agent.processMessage(sessionId, "What framework should we use?", {
+  tags: ["architecture"],     // optional: filter memory by tags
+  remember: true,             // optional: flag this message for extraction
+  memoryKey: "framework-q",   // optional: custom memory key
+});
+// `prompt` is an array of { role, content } messages ready for an LLM API call.
+```
+
+### Recording a response
+```js
+agent.recordResponse(sessionId, "I recommend using Next.js because...", {
+  remember: true,
+  memoryKey: "framework-decision",
+  tags: ["architecture", "decision"],
+});
+```
+
+### Storing a fact directly
+```js
+agent.remember("user-timezone", "PST (UTC-8)", {
+  tags: ["preference"],
+  importance: 0.8,
+});
+```
+
+### Recalling memories
+```js
+const results = agent.recall({ query: "timezone", tags: ["preference"], limit: 3 });
+// Returns: [{ entry: MemoryEntry, score: number }]
+```
+
+### Ending a session
+```js
+const result = agent.endSession(sessionId, "Decided on Next.js for the frontend");
+// Automatically extracts flagged messages and summary into the memory bank
+```
+
+## Operating Principles
+
+1. **Memory-first**: Before answering, check if relevant memories exist. Use `agent.recall()` to search.
+2. **Learn continuously**: When the user shares preferences, decisions, or important context, store them with `agent.remember()`.
+3. **Session discipline**: Start a session for each task. Close it with a summary when done.
+4. **Tag consistently**: Use a consistent tagging scheme (e.g., `preference`, `decision`, `fact`, `project:<name>`, `code:<topic>`).
+5. **Importance weighting**: Rate memories 0.0–1.0. Decisions and preferences are high (0.7–1.0). Ephemeral facts are low (0.1–0.3).
+6. **Context awareness**: The system automatically injects relevant memories into prompts. Trust the scoring.
+7. **Compact periodically**: If the memory bank grows large, call `agent.memoryStore.compact(500)` to prune low-value entries.
+
+## File Operations
+
+When you need to write or modify files in the repository:
+- Use the Write tool to create new files
+- Use the Edit tool to modify existing files
+- Always read a file before editing it
+- Run tests after making changes: `node --test tests/`
+
+## Testing
+
+Run the full test suite to verify the system:
+```bash
+node --test tests/memory-store.test.js tests/session-manager.test.js tests/agent-orchestrator.test.js
+```
+
+All 43 tests should pass. If you modify any module, re-run the relevant tests.
+```
+
+---
+
+## Sprint Plan
+
+### Week 1: Foundation (Complete)
+| Action | Owner | Risk | Rationale & Output |
+|--------|-------|------|-------------------|
+| Design 3-module architecture | Agent Engineer | Over-engineering | Keep it minimal: MemoryStore, SessionManager, AgentOrchestrator. Zero external dependencies. |
+| Implement MemoryStore with persistence | Agent Engineer | Data loss | JSON file-backed with atomic writes. Tested serialization round-trips. |
+| Implement SessionManager with lifecycle | Agent Engineer | State corruption | Each session persists independently. Closed sessions cannot be mutated. |
+| Implement AgentOrchestrator | Agent Engineer | Prompt bloat | Memory injection is scored and limited (default 5). Context window is bounded (default 20 messages). |
+| Write 43 tests across all modules | Agent Engineer | Untested code | Every public method tested. Edge cases covered (corruption, missing keys, closed sessions). |
+| Fix word-level search matching | Agent Engineer | False negatives | Changed from full-string `includes` to word-tokenized matching. Filters words < 3 chars to avoid noise. |
+
+### Week 2: Integration & Extension (Next Steps)
+| Action | Owner | Risk | Rationale & Output |
+|--------|-------|------|-------------------|
+| Wire up to Claude API for end-to-end agent loop | Agent Engineer | API rate limits | Use `AgentOrchestrator.processMessage()` output as input to `@anthropic-ai/sdk`. |
+| Add semantic search via embeddings | Agent Engineer | Complexity | Replace word-level search with embedding similarity for better recall. |
+| Build CLI interface | Agent Engineer | UX friction | Simple `node src/cli.js chat` command to interact with the agent. |
+| Add memory export/import | Agent Engineer | Data portability | JSON import/export for backup and migration between environments. |
+
+### Week 3: Production Hardening
+| Action | Owner | Risk | Rationale & Output |
+|--------|-------|------|-------------------|
+| Add rate limiting and token counting | Agent Engineer | Cost overrun | Track tokens per session to stay within budget. |
+| Add encryption for sensitive memories | Agent Engineer | Data exposure | Encrypt memory bank at rest for secrets and credentials. |
+| Performance testing with 1000+ memories | Agent Engineer | Degradation | Benchmark search latency and optimize indexing if needed. |

package.json

@@ -0,0 +1,13 @@
+{
+  "name": "agent-memory-system",
+  "version": "1.0.0",
+  "description": "Continuous memory bank and session management for agentic LLM assistants",
+  "main": "src/index.js",
+  "type": "module",
+  "scripts": {
+    "test": "node --test tests/",
+    "start": "node src/index.js"
+  },
+  "keywords": ["agent", "memory", "llm", "session"],
+  "license": "MIT"
+}

src/agent/agent-orchestrator.js

@@ -0,0 +1,184 @@
+/**
+ * AgentOrchestrator — the brain that ties sessions and memory together.
+ *
+ * Responsible for:
+ * - Routing user prompts through the memory-augmented pipeline
+ * - Building rich context from session history + memory bank
+ * - Formatting prompts with injected memory context
+ * - Processing responses and extracting learnings
+ * - Managing the agent lifecycle (start, interact, end)
+ */
+
+import { MemoryStore } from "../memory/memory-store.js";
+import { SessionManager } from "../session/session-manager.js";
+
+export class AgentOrchestrator {
+  constructor({
+    memoryPath = "data/memory-bank.json",
+    sessionsDir = "data/sessions",
+    systemPrompt = null,
+    contextWindowSize = 20,
+    memoryLimit = 5,
+  } = {}) {
+    this.memoryStore = new MemoryStore(memoryPath);
+    this.sessionManager = new SessionManager(sessionsDir, this.memoryStore);
+    this.systemPrompt = systemPrompt ?? DEFAULT_SYSTEM_PROMPT;
+    this.contextWindowSize = contextWindowSize;
+    this.memoryLimit = memoryLimit;
+    this.responseHandler = null; // plug in your LLM call here
+  }
+
+  /** Start a new agent session. */
+  startSession(name, metadata = {}) {
+    const session = this.sessionManager.create(name, metadata);
+    return {
+      sessionId: session.id,
+      sessionName: session.name,
+      status: session.status,
+    };
+  }
+
+  /** Resume an existing session. */
+  resumeSession(sessionId) {
+    const session = this.sessionManager.resume(sessionId);
+    return {
+      sessionId: session.id,
+      sessionName: session.name,
+      status: session.status,
+      messageCount: session.messages.length,
+    };
+  }
+
+  /** End a session with an optional summary. */
+  endSession(sessionId, summary = null) {
+    const result = this.sessionManager.close(sessionId, summary);
+    return {
+      sessionId: result.session.id,
+      status: result.session.status,
+      extractedMemories: result.extractedMemories,
+    };
+  }
+
+  /**
+   * Process a user message: build context, format prompt, handle response.
+   * Returns the formatted prompt object ready to send to an LLM.
+   */
+  processMessage(sessionId, userMessage, { tags = [], remember = false, memoryKey = null } = {}) {
+    // Record the user message
+    const msgMeta = {};
+    if (remember) {
+      msgMeta.remember = true;
+      if (memoryKey) msgMeta.memoryKey = memoryKey;
+      if (tags.length) msgMeta.tags = tags;
+    }
+    this.sessionManager.addMessage(sessionId, "user", userMessage, msgMeta);
+
+    // Build augmented context
+    const context = this.sessionManager.buildContext(sessionId, {
+      windowSize: this.contextWindowSize,
+      memoryLimit: this.memoryLimit,
+      memoryTags: tags,
+    });
+
+    // Format the prompt with memory injection
+    const formattedPrompt = this._buildPrompt(context);
+
+    return {
+      prompt: formattedPrompt,
+      context,
+    };
+  }
+
+  /** Record an assistant response back into the session. */
+  recordResponse(sessionId, assistantMessage, { remember = false, memoryKey = null, tags = [] } = {}) {
+    const meta = {};
+    if (remember) {
+      meta.remember = true;
+      if (memoryKey) meta.memoryKey = memoryKey;
+      if (tags.length) meta.tags = tags;
+    }
+    this.sessionManager.addMessage(sessionId, "assistant", assistantMessage, meta);
+  }
+
+  /** Directly store a fact in the memory bank. */
+  remember(key, content, { tags = [], importance = 0.5 } = {}) {
+    return this.memoryStore.store(key, content, {
+      tags,
+      source: "direct",
+      metadata: { importance },
+    });
+  }
+
+  /** Search the memory bank. */
+  recall({ query = "", tags = [], limit = 5 } = {}) {
+    return this.memoryStore.search({ query, tags, limit });
+  }
+
+  /** Forget a specific memory. */
+  forget(key) {
+    return this.memoryStore.forget(key);
+  }
+
+  /** List all sessions. */
+  listSessions(status = null) {
+    return this.sessionManager.list(status).map((s) => ({
+      id: s.id,
+      name: s.name,
+      status: s.status,
+      messageCount: s.messages.length,
+      createdAt: s.createdAt,
+      updatedAt: s.updatedAt,
+    }));
+  }
+
+  /** Get memory bank stats. */
+  memoryStats() {
+    return {
+      totalMemories: this.memoryStore.size,
+      keys: this.memoryStore.keys(),
+    };
+  }
+
+  /** Build the full prompt with system prompt, memory context, and messages. */
+  _buildPrompt(context) {
+    const parts = [];
+
+    // System prompt
+    parts.push({ role: "system", content: this.systemPrompt });
+
+    // Memory injection block
+    if (context.memories.length > 0) {
+      const memoryBlock = context.memories
+        .map((m) => `- [${m.key}] (relevance: ${m.score.toFixed(2)}): ${m.content}`)
+        .join("\n");
+
+      parts.push({
+        role: "system",
+        content: `## Relevant Memories\nThe following facts from your memory bank may be relevant:\n${memoryBlock}`,
+      });
+    }
+
+    // Session context metadata
+    parts.push({
+      role: "system",
+      content: `## Session Context\nSession: ${context.sessionName} (${context.sessionId})\nMessages in context: ${context.messages.length}`,
+    });
+
+    // Conversation messages
+    for (const msg of context.messages) {
+      parts.push({ role: msg.role, content: msg.content });
+    }
+
+    return parts;
+  }
+}
+
+const DEFAULT_SYSTEM_PROMPT = `You are an intelligent agent with persistent memory across sessions.
+
+Key capabilities:
+- You remember facts, preferences, and decisions from previous conversations.
+- You can store new information for future reference when asked.
+- You maintain context within and across sessions.
+
+When the user shares important information, preferences, or decisions, note them for memory storage.
+When answering questions, draw on relevant memories to provide personalized, context-aware responses.`;

src/index.js

@@ -0,0 +1,16 @@
+/**
+ * Agent Memory System — public API surface.
+ *
+ * Usage:
+ *   import { AgentOrchestrator, MemoryStore, SessionManager } from "agent-memory-system";
+ *
+ *   const agent = new AgentOrchestrator();
+ *   const { sessionId } = agent.startSession("my-session");
+ *   const { prompt } = agent.processMessage(sessionId, "Hello!");
+ *   // Send `prompt` to your LLM, get response, then:
+ *   agent.recordResponse(sessionId, "Hi there!");
+ */
+
+export { MemoryStore, MemoryEntry } from "./memory/memory-store.js";
+export { SessionManager, Session } from "./session/session-manager.js";
+export { AgentOrchestrator } from "./agent/agent-orchestrator.js";