macMLX

English · 简体中文

Native macOS LLM inference, powered by Apple MLX.

macMLX runs local LLMs on Apple Silicon with a first-class native macOS experience — no cloud, no telemetry, no Electron. A polished SwiftUI app for newcomers, a proper CLI for developers, and an always-on OpenAI-compatible API for everything else.

---

Why macMLX?

MLX inference and a CLI used to be the whole pitch — but as of 2026 both LM Studio and Ollama ship MLX engines on Apple Silicon, and LM Studio has the lms CLI. So the honest comparison is the combination: a genuinely native macOS GUI, an always-on API, and zero Python in one ~50 MB app.

	macMLX	LM Studio	Ollama	oMLX
Native macOS GUI	✅ SwiftUI	Electron	menu-bar only	✅ SwiftUI (v0.4+)
Swift-native in-process engine	✅	❌	❌	❌ (Python core)
MLX inference	✅	✅	✅ (preview)	✅
CLI	✅	✅ lms	✅	launcher only
Resumable downloads + mirrors	✅	⚠ partial	⚠ partial	❌
OpenAI-compatible API	✅ always-on	✅	✅	✅
Zero Python required	✅	✅	✅	❌

Where macMLX stands alone: the inference engine itself is Swift, running in-process — oMLX's native app (v0.4+) fronts a Python core, ours has no Python anywhere in one ~50 MB DMG. On top of that: a proper CLI/TUI sharing the same Swift core, and owning frontier model architectures in pure Swift (the DeepSeek V3.2 port) instead of waiting for upstream.

Requirements

macOS 14.0 (Sonoma) or later · Apple Silicon (M1–M4) · no Python required.

Installation

Download macMLX-vX.X.X.dmg from Releases, mount it, and drag macMLX.app to /Applications. The DMG isn't notarized yet (#19), so clear Gatekeeper on first launch:

xattr -cr /Applications/macMLX.app    # clear quarantine
open /Applications/macMLX.app

(Or right-click the app → Open → Open.)

Feature highlights (v0.2 → v0.9)

Seventeen releases since the v0.1 MVP, by area. This section tracks the latest shipped state — new features land here first, then get a one-line roadmap entry below.

• Engine & models — in-process MLX Swift engine (text + 16 VLM architectures with dedicated OCR-model recognition, models to ~70B); continuous batching (2.5-3.2× aggregate throughput under concurrent clients, engages only under real concurrency); tiered KV prompt cache (RAM + SSD) with longest-common-prefix reuse across agent turns; speculative decoding (draft models + acceptance-rate telemetry); multi-model pool with LRU eviction, LoRA adapter inference, MCP server (macmlx mcp serve); pure-Swift architecture ports — DeepSeek V3.2, Mellum2, Solar-Open, GLM-5.1 — registered as external overlays and parity-verified against the Python reference (support tiers). Runs on a controlled minimal fork of mlx-swift carrying two upstream-merged fixes, dropped at the next upstream release.
• Downloads — resumable across cancels and app quits, live speed/ETA, HuggingFace mirror support, Hub-commit update detection.
• Chat — conversation sidebar (rename, delete, rewind), streaming Markdown, per-message actions, per-model Parameters Inspector, collapsible <think> reasoning blocks.
• Audio — speech-to-text and text-to-speech in-process on MLX through mlx-audio-swift (POST /v1/audio/transcriptions, POST /v1/audio/speech, OpenAI-shaped, with formats reported honestly); in the app, transcription is a cancellable attachment in the chat composer and playback is a button on each reply; audio checkpoints get their own model-library scan. Unit-tested, not yet validated against real checkpoints on our hardware.
• API — always-on OpenAI-compatible server plus Ollama (NDJSON) and Anthropic (/v1/messages) compatibility; structured output (response_format json_object / JSON-schema subset with nested objects, arrays, $ref, any root type, Unicode literals and numeric bounds, constrained decoding); tools pass-through with tool_calls responses; logit_bias, logprobs, stream_options.include_usage with prompt-cache hits in usage, XTC sampler, per-request LoRA adapters, KV-cache quantization; /v1/embeddings + /v1/rerank (a true cross-encoder when the checkpoint is a reranker, bi-encoder fallback otherwise), optional bearer auth, model aliases + idle TTL, reasoning_content separation, model cold-swap by ID, stall watchdog, CORS + probe endpoints; concurrent clients batch onto one model instead of queuing serially.
• CLI — native ANSI dashboards for pull / serve / run, PID coordination shared with the GUI.
• Activity, Benchmark & Logs tabs — a live Activity panel with sudoless Apple-Silicon readouts (GPU occupancy, memory bandwidth, thermal/memory pressure, per-rail power) and the current inference bottleneck with advice, fusing hardware counters with the engine's own prefill/decode phase (the signal an external monitor can't reach); Benchmark's local tok/s · TTFT · peak memory with a community leaderboard, now attributing each run's decode bottleneck; a Pulse-backed log viewer with MLX stdout/stderr teed in.

Full per-release detail: CHANGELOG.md.

Quickstart

GUI — launch macMLX; the setup wizard picks the engine and model directory; download a model from the built-in HuggingFace browser; load and chat.

CLI

macmlx pull mlx-community/Qwen3-8B-4bit     # download
macmlx run Qwen3-8B-4bit "Hello, world"      # single prompt
macmlx serve                                 # API on :8000
macmlx ps / stop                             # status / shutdown

Connecting external tools

The OpenAI-compatible server runs on http://localhost:8000/v1 whenever a model is loaded (or macmlx serve is running). Point any OpenAI client (Cursor, Continue, Cline, Open WebUI, Zed, Raycast, …) at that base URL with any key.

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"Qwen3-8B-4bit","messages":[{"role":"user","content":"Hi"}]}'

Inference engines

Engine	Status	Notes
MLX Swift (default)	✅ Shipping	Apple's mlx-swift-lm, in-process. Text + 16 VLM architectures, models to ~70B, tiered KV cache + model pool + LoRA.
SwiftLM (100B+ MoE)	🔓 Reopenable	Subprocess path, unblocked since sandbox-off (#12 / #13) — not yet committed.
Python mlx-lm	🔓 Reopenable	Subprocess path for max model coverage, in exchange for uv on PATH.

Every engine hides behind one InferenceEngine protocol — the GUI never knows which one runs.

Architecture

macMLX.app (SwiftUI)   macmlx (CLI)
        └──── MacMLXCore ────┘        (Swift SPM package)
                  │
           InferenceEngine → MLXSwiftEngine (in-process)
                  │
           HummingbirdServer → http://localhost:8000/v1
                  │
           Apple Silicon (Metal / ANE)

Data lives under ~/.mac-mlx/ (models, conversations, params, logs, settings) — a dotfile under real $HOME, so the sandboxed app reads/writes without entitlements while staying visible to power users.

Building from source

git clone https://github.com/magicnight/mac-mlx && cd mac-mlx
brew bundle                              # dev tools
open macMLX/macMLX.xcodeproj             # GUI  (or: xcodebuild -scheme macMLX build)
swift build --package-path macmlx-cli    # CLI
swift test  --package-path MacMLXCore    # tests (~3s)

Roadmap

This section is the project's single roadmap, kept current per release: when a 0.x ships, it moves from a future section up to Shipped, and the feature highlights above get updated to match. Everything else — plan documents under docs/superpowers/, design specs, .claude/features/ — is either historical or a pointer back here.

• Shipped (v0.1 → v0.9.0) — native GUI + menu bar + CLI + OpenAI API (v0.1); download & chat polish (v0.2); Benchmark, Logs, chat history, API cold-swap, Ollama compat, sandbox-off (v0.3); the v0.5 engine leap — VLMs, tiered KV cache, model pool, LoRA, MCP server + client pool, chat tool routing; server hardening, embeddings + rerank, and the stability wave (v0.5.1-0.5.3); the DeepSeek V3.2 pure-Swift port, parity-verified at 1e-4; the v0.6 agent backend — continuous batching (2.5-3.2× under concurrency), longest-common-prefix prompt-cache reuse, structured output, speculative decoding, the API-compat pack; the Track G model wave and per-model chat-template overrides (v0.6.1-0.6.2); v0.7.0 silicon-metrics observability — the sudoless Activity panel, the phase-fused bottleneck classifier, per-run benchmark attribution, and OCR-model recognition; v0.8.0, the tiered SSD KV cache hardened end to end — bounded, weight-fingerprinted, restart-surviving with cross-session longest-prefix reuse, and non-blocking; and v0.9.0 — in-process speech-to-text and text-to-speech (/v1/audio/*, plus transcription and playback in the app), a true cross-encoder /v1/rerank, MTP drafter detection, the app finally linking the controlled MLX fork, and thirteen upstream MLX correctness fixes. Per-tag detail in CHANGELOG.md.
• Next release (on main) — the controlled MLX fork moves from mlx-swift 0.31.6 to 0.32.3 (core v0.32.2): twelve of the thirteen fixes it carried are upstream now, so it carries two; the cross-thread evaluation abort that pinned it to the old core is gone; mlx-swift-lm 3.32.3 comes with it. /v1/rerank runs on its MLXRerankers — encoder cross-encoders, Qwen3 rerankers and Jina v3 through one factory — verified on ms-marco-MiniLM against PyTorch and, for ordering, on Qwen3-Reranker-0.6B-4bit, the first time the endpoint met a real checkpoint. See [Unreleased] in CHANGELOG.md.
• In progress — the server as an interoperability target. macOS 27's Foundation Models API, Xcode 27's local-model provider, and third-party routers all speak OpenAI chat completions to localhost:8000; stream_options.include_usage, nested objects, arrays and $ref in response_format JSON schemas, non-ASCII enum values, non-object schema roots and numeric bounds (all of which Apple's @Generable emits) are in; next is a verified Xcode 27 connection. Design: Foundation Models integration.
• Queued — re-land the swift-jinja 2.4 adoption (its upstream blocker is gone); measure, then decide on .balanced prefill chunking; return to upstream mlx-audio-swift once a release keeps audio downloads in the cache it is given and completes a cached snapshot by the Hub's listing (v0.1.5 already builds against mlx-swift-lm 3.32.3 and brings the STT correctness fixes).
• Later — a community benchmarks service; Apple's built-in model as a macMLX engine on macOS 27; evaluate a model-tier mapping panel for Claude Code / Codex; MTP decoding once a loadable target model emits drafter state; custom Metal kernels only if profiling demands it.
• Reopenable (feasible since sandbox-off) — Python / SwiftLM subprocess engines (#12 / #13); signed + notarized DMG (#19, needs a paid developer account). The Homebrew formula is rendered and attached to every release, but the tap repository itself is not published yet (#20).

Contributing · License

Issues and PRs welcome — see CONTRIBUTING.md. Apache 2.0 (LICENSE).

Acknowledgements

MLX + mlx-swift-lm (Apple), Swama, SwiftLM, oMLX, Hummingbird, Sparkle, Pulse, SwiftTUI. Full citations: CITATIONS.bib.