Membrane Memory Engine & Chatbot

This project is an advanced Membrane Ecosystem designed to handle long-term interactions without suffering from context window overflow. It features a decoupled Memory Engine that manages active context and persistent memory, alongside a modern Next.js UI.

The core engine is built with FastAPI to independently manage word-level text chunking, persistent memory (context.json), and active context logic.

Users can click on specific words/phrases returned by the LLM to highlight and save them into the active context. Future prompts will automatically inject these selected chunks into the LLM system prompt to prevent token overflow while maintaining coherent long-term interactions.

Click to watch

Standalone Core Usage

The core module and the FastAPI server (main.py) act as an independent Context Memory Engine. Because of the decoupled architecture:

• You can run the core engine separately as a standalone microservice to augment external applications.
• It provides a framework-agnostic REST API (/api/context/*) for granular memory management.
• Any other chatbot client or agentic workflow can utilize this engine for smart word-level chunking and contextual memory, without relying on the provided Next.js UI.

Setup

1. Install uv if you haven't already.
2. Initialize the environment and install dependencies:

   uv add fastapi uvicorn pydantic jinja2 python-multipart requests tiktoken pytest

3. Run the FastAPI application:

   uv run uvicorn main:app --port 5005 --reload

4. Access the UI at http://localhost:5005

Configuration Options

In the UI, you can toggle between three modes:

• Simulate: Runs a mock LLM instantly (great for testing UI interaction).
• Direct API Key: Provide your Groq or OpenAI API key.
• LLM Proxy: Provide a local LLM proxy URL (e.g., http://localhost:8001/v1/chat/completions) generated by a tool like llm-keypool to utilize proxy load balancing and key rotation.

Advanced Evaluation

The project includes a robust evaluation pipeline using DeepEval, Ragas, and TruLens to monitor hallucination, faithfullness, and relevance in the background. You can toggle this mode from the UI configurations menu.