The Short Answer
Rapid-MLX 1.0 is an Apache 2.0, OpenAI- and Anthropic-compatible LLM server for Apple Silicon, released October 29, 2026. Its speed claims are vendor-measured, and "zero cloud" needs care: telemetry is on by default and its chat harness adds web tools. A Mac is a strong developer and edge tier, not a fleet server. On ibl.ai you own all the code and the data, so a local endpoint like this becomes one more model.
The project is raullenchai/Rapid-MLX on GitHub, built on Apple's MLX framework and focused on reliable tool calling for coding agents. Version 1.0.0 shipped October 29, and 1.0.2 followed on November 6, 2026.
What is Rapid-MLX and what shipped in version 1.0?
Rapid-MLX is a local inference server for Macs with Apple Silicon. It exposes an OpenAI-style /v1/chat/completions API and the Anthropic Messages API, so agents built for a cloud model can point at a Mac instead. The repository had about 4,100 GitHub stars on November 7, 2026.
Version 1.0.0 added persistent agent chats with tools and approvals in the terminal, new LFM3 text and LFM3-VL vision models, and video generation with synchronized audio for Macs with 160 GB or more of unified memory.
Version 1.0.2 lets the desktop app unload an idle model while the local API stays up. The project measured Qwen3.5 4B memory falling from 3.25 GB to 0.74 GB when idle. Installation is one command: brew install rapid-mlx.
Is Rapid-MLX really 4x faster, and faster than what?
Rapid-MLX's "up to 4x" is measured against Apple's own mlx-lm server on the same weights, not against Ollama or a cloud API. The same README line says 1.5x on a typical task, measured on Qwen3.5-9B 4-bit on a Mac mini M4 Pro with greedy decoding.
A separate run on a Mac Studio M5 Max with 36 GB of memory compares it with oMLX and LM Studio. The Qwen3.6-35B results are on the project's comparison page with a disclosure that the authors build Rapid-MLX:
| Qwen3.6-35B 4-bit, M5 Max 36 GB | Rapid-MLX 0.16.1 | oMLX 0.7.0 | LM Studio (MLX) |
|---|---|---|---|
| Decode, 1 stream (tok/s) | 173.4 | 145.8 | 124.5 |
| 4 streams, aggregate (tok/s) | 293.9 | 281.9 | 265.6 |
| First token, ~2K prompt (s, lower is better) | 0.55 | 1.14 | 1.49 |
| Tool calls passed (30 prompts) | 30/30 | 29/30 | 29/30 |
Read it with the project's own notes attached. It is a single run per engine, measured October 12, 2026 on version 0.16.1, and Rapid-MLX used speculative decoding on a text-only lane. Under about 5%, the project says, differences are run-to-run noise.
The single-stream lead is 19% over oMLX. With four streams it shrinks to about 4%, inside that noise band.
The clearer win on this model is time to first token, roughly half the competitors' on a 2,000-token prompt. On the page's Qwen3.5-9B run, oMLX was slightly faster to first token.
How fast does Rapid-MLX run on an ordinary Mac?
On Macs most teams already own, Rapid-MLX runs a mid-size model at reading speed or better. The project's speed-by-chip table collects community submissions fetched October 28, 2026 under one protocol, reported as median decode speed:
| Chip (memory) | Qwen3.5 4B 4-bit | Qwen3.6 35B-A3B 4-bit | Qwen3.8 27B 4-bit MTP |
|---|---|---|---|
| M4 (16 GB) | 38.4 | — | — |
| M3 Pro (36 GB) | 48.3 | 58.9 | 8.8 |
| M3 Max (36 GB) | 90.7 | 101.1 | 17.6 |
| M4 Max (128 GB) | — | 129.2 | 24.8 |
| M5 Max (128 GB) | — | — | 32.5 |
The pattern matters more than any cell. Qwen3.6 35B-A3B is a mixture of experts with about 3B parameters active per token, so it decodes faster than the dense 4B model on the same chip. The dense 27B model is several times slower everywhere.
So a 36 GB laptop runs a capable mixture-of-experts model at 59 to 101 tokens per second. A dense 27B model on the same machine runs at 9 to 18, which is usable for one person and not for a team.
Does Rapid-MLX really run with zero cloud?
Not by default. Rapid-MLX runs inference on the Mac, and the README says prompts, completions, paths, IP addresses and API keys are never collected. But two defaults reach the network, and a security review should know about both before calling it offline.
| Default | What leaves the Mac | How to turn it off |
|---|---|---|
| Telemetry (on since 0.15.0) | Anonymous, metadata-only events; the engine sends batches to PostHog Cloud (US) | rapid-mlx telemetry off, RAPID_MLX_TELEMETRY=0 or DO_NOT_TRACK=1 |
| Chat harness web tools | Weather, web search and web_fetch queries, available to tool-capable models by default |
RAPID_MLX_CHAT_WEB_TOOLS=0 before starting a profile |
| Model downloads | Weights fetched from Hugging Face on first use | Pre-stage models, then run offline |
None of this is hidden; the project documents it in its telemetry page and README. The release notes are also candid that the terminal harness's network policy is heuristic and terminal commands are not sandboxed.
The lesson is general. "Local" describes where the model runs. "Air-gapped" describes what the whole system can reach. Our air-gapped AI service exists because those are different claims, and only the second survives an audit.
How many models does Rapid-MLX support?
Rapid-MLX's bundled registry, aliases.json, listed 251 aliases on November 7, 2026, covering text, vision, image, video and embedding models, plus 53 audio aliases in a separate registry. An alias is a model at a given quantization, so distinct models are fewer.
Aliases are pinned. All 251 bundled aliases carry an exact Hugging Face revision hash, and a missing pin fails rather than silently switching weights. For a regulated team that is the more useful feature than the count: it means the model you approved is the model that runs.
Version 1.0.2 also accepts a pasted Hugging Face link and checks whether the model fits the Mac's memory before downloading, so the registry is a starting point rather than a ceiling.
Where does a Mac fit in an enterprise AI stack?
A Mac running Rapid-MLX fits three roles: a developer workstation that keeps proprietary code off third-party APIs, an edge node with no network, and a cheap test bench for trying an open model on your own tasks before buying GPUs.
It is not a fleet server. The project's own LFM3 measurements on an M5 Max reached 951.5 tokens per second aggregate at 8 concurrent requests on a 1.7B model. Serving hundreds of employees on mid-size models still belongs on datacenter GPUs running vLLM or TGI.
Our guide to the self-hosted LLM serving stack covers that split runtime by runtime. And on-device agents are where a fast Mac server earns its place.
One more caution from the security side. Rapid-MLX keeps a prefix cache and saves it to disk on shutdown, which is good for speed. Shared prompt caches are also the surface in the cache side-channel research we covered in September.
How does ibl.ai use a local endpoint like Rapid-MLX?
On ibl.ai you own all the code and the data, the platform is model-agnostic, there is no per-seat pricing, and you can deploy anywhere. An OpenAI-compatible endpoint on a Mac is one more model the platform can call.
Each agent's LLM is chosen per agent in Agentic OS, so a coding agent can run against a local Rapid-MLX server while others stay on a cloud model. Switching back is a setting, not a migration.
That is the real effect of releases like this one. Open-source inference on hardware you already own keeps getting faster, which keeps pushing open-weight prices down. A stack you own can take each gain as it lands.
ibl.ai is family-owned and operated from New York, NY, and our engineers work alongside your team to wire local and private models into the systems you already run.
Sources: release dates, features and the idle-memory figures from the Rapid-MLX v1.0.0 and v1.0.2 release notes; the 4x and 1.5x claims, telemetry and web-tool defaults from the README; the M5 Max comparison from rapidmlx.com/compare; per-chip medians from the project's speed-by-chip table; alias counts from our count of the bundled registry. The brief cited 149 tok/s for Qwen3.6-35B on an M5 Max; no Rapid-MLX source we found shows that figure, and the project's published M5 Max number is 173.4 tok/s. All benchmarks are the project's own or community-submitted.