Self-hosted Perplexity alternative powered by Context-1, a state-of-the-art 20B agentic search model from Chroma. This project leverages the model in an environment similar to its training setup, allowing it to work at its full potential and use the tools it was trained with via just a thin harness and minimal prompting. It provides powerful features like the ability to plan most searches, use tools to get data from multiple sources in parallel, and even manage its own context by pruning what it doesn't need. I've paired it with Gemma 3 4B to help with complex queries by providing a feedback loop to the agent, doing quality checks, and providing more robust responses. This technique is also used by the developers of Chonext-1 in their recently published paper.
Context-1: Agentic Search with a 20B Parameter Model
Key contributions:
- RL-trained retrieval agent — decomposes queries, calls tools, evaluates results
- Context management — model prunes its own context mid-search to stay within token budgets
- Staged training — recall-first curriculum, then precision annealing
- Hybrid search — BM25 + dense vectors via reciprocal rank fusion
Benchmarks: 0.97 on Web tasks, 0.96 on BrowseComp-Plus — matches frontier models at 1/10th the size.
- Agentic search — Context-1 decides what to search, when to stop (3-5 rounds typical)
- Four tools —
search_corpus(semantic),grep_corpus(exact match),read_document(expand),prune_chunks(clean context) - Token budgets — 32K context window, 24K soft warn, 28K hard block
- Cited answers — Gemma 4 e4b writes responses with
【c1†hostname】citations - Conversation history — stored in ChromaDB, persists across sessions
- Brave Search integration — real-time web results via LLM Context API
- Retrieval/answer separation — Context-1 retrieves only, Gemma writes answers. Clean split per paper's "search subagent" design.
- Memory via pruning — Context-1 removes irrelevant chunks mid-search. Evidence persists to ChromaDB before prune; corpus compounds.
- Parallel tool execution — Context-1 issues multiple tool calls per round. Batch execution, SSE streams status per batch.
- Trajectory recall — All retrieved evidence saved to ChromaDB even if later pruned. Rewards discovery, not just final context.
- Hybrid search (RRF) —
search_corpuscombines BM25 + dense vectors via reciprocal rank fusion, reranks top 50. - Disciplined token budgets — 32K max, 24K soft-warn, 28K hard-block. Model must prune or stop. No overflow workaround.
Browser ──► App Backend ──► Context-1 Proxy
│
▼
┌───────────────┐
│ Gemma 4 e4b │
│ (orchestrator/│
│ Planner) │
│ query ──► │
│ sub-questions │
└───────┬───────┘
│
▼
┌───────────────────────┐
│ Context-1 20B │
│ (retrieval agent) │
│ ├─ search - Web │
│ ├─ grep - Cache │
│ ├─ read │
│ └─ prune - Autonomus │
│ Context Managment │
└──────────┬────────────┘
│
┌───────────────┴───────────────┐
▼ ▼
ChromaDB Brave Adapter (LLM Context API)
(corpus + history) (web + cache data)
│ │
└───────────────┬───────────────┘
▼
┌───────────────┐
│ Evidence pool │
│ back to Gemma │
│ │
└───────┬───────┘
▼
┌──────────────────┐
│ Formatted answer │
│ + │
│ Refrence Links │
└──────────────────┘
- macOS 14+ on Apple Silicon
- 24 GB unified memory recommended
- Brave Search API key (free tier: 2K queries/month)
Containerized deployment with GPU acceleration via Krunkit.
git clone https://github.com/FaisalAlnahhas1/free-plexity.git
cd free-plexity
# 1. Install Colima with Krunkit (GPU support)
brew install colima krunkit
# 2. Start Colima AI runtime
colima start --vm-type krunkit --cpus 4 --memory 16 --disk 80
# 3. Configure API key
cp .env.example .env
# Edit .env: set BRAVE_API_KEY
# 4. Download models
mkdir -p models
wget -P models/ https://huggingface.co/mlx-community/context-1-MLX-6bit/resolve/main/context-1-7b.gguf
wget -P models/ https://huggingface.co/google/gemma-3-4b-it-GGUF/resolve/main/gemma-3-4b-it-q4_k_m.gguf
# 5. Start
./scripts/colima-start.sh
# Open http://localhost:8080git clone https://github.com/FaisalAlnahhas1/free-plexity.git
cd free-plexity
make setup # Prompts for Brave key, downloads models, installs deps
make start # Starts all services
# Open http://localhost:8080| Service | Port | Purpose |
|---|---|---|
| App Backend | :8080 | Starlette API, SSE streaming, React UI |
| Context-1 Proxy | :8005 | Orchestrates retrieval loop, conversations |
| Brave Adapter | :8004 | Brave LLM Context API with caching |
| llama-server | :8000 | GGUF model serving (Colima) |
| omlx (MLX server) | :8000 | Serves Context-1 + Gemma 4 (Native) |
./scripts/colima-start.sh # Start all services
docker compose -f docker-compose.colima.yml down # Stop
docker compose -f docker-compose.colima.yml logs -f # Follow logsmake setup # First-time setup
make start # Start all services
make stop # Stop all services
make status # Health check
make build # Rebuild frontendmake eval-retrieval # Context-1 tool decisions (mock Brave)
make eval-planner # Gemma citation/format (fixture evidence)
make eval-full # Full SSE pipeline
# Flags: real-api (live Brave), logs (stream colored logs)| Model | Size | Link |
|---|---|---|
| Context-1 (retrieval) | 7B | mlx-community/context-1-7b.gguf |
| Gemma 3 (answer) | 4B | google/gemma-3-4b-it-GGUF |
| Model | Size | Link |
|---|---|---|
| Context-1-MLX-6bit | 20B | mlx-community/context-1-MLX-6bit |
MIT