---
title: "Rapid-MLX 1.0: Local LLMs on a Mac, Read Honestly"
slug: "rapid-mlx-1-0-local-llm-server-apple-silicon-benchmarks"
author: "Miguel Amigot"
date: "2026-11-07 10:00:00"
category: "Premium"
topics: "Rapid-MLX, MLX, Apple Silicon, local LLM, local AI on Mac, self-hosted LLM, inference server, open-source AI, Qwen3.6, on-device AI, model-agnostic AI, AI telemetry, data sovereignty"
summary: "Rapid-MLX 1.0 is an Apache 2.0 LLM server for Apple Silicon. What its speed numbers actually measure, what leaves the Mac by default, and where a Mac fits in an enterprise AI stack."
banner: ""
thumbnail: ""
linkedin: |
  Rapid-MLX 1.0 shipped on October 29: an Apache 2.0, OpenAI- and Anthropic-compatible LLM server for Apple Silicon, installable with one Homebrew command.

  The numbers are good, and worth reading carefully. "Up to 4x faster" is against Apple's own mlx-lm, and the project puts a typical task at 1.5x. Its best Mac Studio M5 Max figure, 173.4 tokens per second on Qwen3.6-35B, is a single run the project measured itself.

  The part enterprises should read twice: anonymous, metadata-only telemetry is on by default, and the chat harness ships web search on. Both switch off. Local inference is not the same thing as air-gapped until you check.

  The bigger point is the shape of the market. Open-source inference keeps getting faster on hardware people already own. When you own all the code and the data, every one of those gains is yours to plug in.

  #iblai #OpenSource #LLM #AppleSilicon #AgenticAI #EnterpriseAI
---

## The Short Answer

**Rapid-MLX 1.0 is an Apache 2.0, OpenAI- and Anthropic-compatible LLM server for Apple Silicon, released October 29, 2026. Its speed claims are vendor-measured, and "zero cloud" needs care: telemetry is on by default and its chat harness adds web tools. A Mac is a strong developer and edge tier, not a fleet server. On ibl.ai you own all the code and the data, so a local endpoint like this becomes one more model.**

The project is [raullenchai/Rapid-MLX](https://github.com/raullenchai/Rapid-MLX) on GitHub, built on Apple's MLX framework and focused on reliable tool calling for coding agents. Version [1.0.0](https://github.com/raullenchai/Rapid-MLX/releases/tag/v1.0.0) shipped October 29, and [1.0.2](https://github.com/raullenchai/Rapid-MLX/releases/tag/v1.0.2) followed on November 6, 2026.

## What is Rapid-MLX and what shipped in version 1.0?

Rapid-MLX is a local inference server for Macs with Apple Silicon. It exposes an OpenAI-style `/v1/chat/completions` API and the Anthropic Messages API, so agents built for a cloud model can point at a Mac instead. The repository had about 4,100 GitHub stars on November 7, 2026.

Version 1.0.0 added persistent agent chats with tools and approvals in the terminal, new LFM3 text and LFM3-VL vision models, and video generation with synchronized audio for Macs with 160 GB or more of unified memory.

Version 1.0.2 lets the desktop app unload an idle model while the local API stays up. The project measured Qwen3.5 4B memory falling from 3.25 GB to 0.74 GB when idle. Installation is one command: `brew install rapid-mlx`.

## Is Rapid-MLX really 4x faster, and faster than what?

Rapid-MLX's "up to 4x" is measured against Apple's own `mlx-lm` server on the same weights, not against Ollama or a cloud API. The same README line says **1.5x on a typical task**, measured on Qwen3.5-9B 4-bit on a Mac mini M4 Pro with greedy decoding.

A separate run on a Mac Studio M5 Max with 36 GB of memory compares it with oMLX and LM Studio. The Qwen3.6-35B results are on the project's [comparison page](https://rapidmlx.com/compare) with a disclosure that the authors build Rapid-MLX:

<table style="width:100%; border-collapse:collapse; margin:1.5rem 0; font-size:0.95rem;">
  <thead>
    <tr style="background:#f5f5f0; border-bottom:2px solid #2175C5;">
      <th style="text-align:left; padding:0.75rem; color:#5f6368;">Qwen3.6-35B 4-bit, M5 Max 36 GB</th>
      <th style="text-align:right; padding:0.75rem; color:#5f6368;">Rapid-MLX 0.16.1</th>
      <th style="text-align:right; padding:0.75rem; color:#5f6368;">oMLX 0.7.0</th>
      <th style="text-align:right; padding:0.75rem; color:#5f6368;">LM Studio (MLX)</th>
    </tr>
  </thead>
  <tbody>
    <tr style="background:#f0f9ff; border-bottom:1px solid #e5e7eb;">
      <td style="padding:0.75rem;"><strong>Decode, 1 stream (tok/s)</strong></td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;"><strong>173.4</strong></td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">145.8</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">124.5</td>
    </tr>
    <tr style="border-bottom:1px solid #e5e7eb;">
      <td style="padding:0.75rem;">4 streams, aggregate (tok/s)</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">293.9</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">281.9</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">265.6</td>
    </tr>
    <tr style="border-bottom:1px solid #e5e7eb;">
      <td style="padding:0.75rem;">First token, ~2K prompt (s, lower is better)</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">0.55</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">1.14</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">1.49</td>
    </tr>
    <tr style="border-bottom:1px solid #e5e7eb;">
      <td style="padding:0.75rem;">Tool calls passed (30 prompts)</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">30/30</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">29/30</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">29/30</td>
    </tr>
  </tbody>
</table>

Read it with the project's own notes attached. It is a single run per engine, measured October 12, 2026 on version 0.16.1, and Rapid-MLX used speculative decoding on a text-only lane. Under about 5%, the project says, differences are run-to-run noise.

The single-stream lead is 19% over oMLX. With four streams it shrinks to about 4%, inside that noise band.

The clearer win on this model is time to first token, roughly half the competitors' on a 2,000-token prompt. On the page's Qwen3.5-9B run, oMLX was slightly faster to first token.

## How fast does Rapid-MLX run on an ordinary Mac?

On Macs most teams already own, Rapid-MLX runs a mid-size model at reading speed or better. The project's [speed-by-chip table](https://github.com/raullenchai/Rapid-MLX/blob/main/docs/benchmarks/speed-by-chip.md) collects community submissions fetched October 28, 2026 under one protocol, reported as median decode speed:

<table style="width:100%; border-collapse:collapse; margin:1.5rem 0; font-size:0.95rem;">
  <thead>
    <tr style="background:#f5f5f0; border-bottom:2px solid #2175C5;">
      <th style="text-align:left; padding:0.75rem; color:#5f6368;">Chip (memory)</th>
      <th style="text-align:right; padding:0.75rem; color:#5f6368;">Qwen3.5 4B 4-bit</th>
      <th style="text-align:right; padding:0.75rem; color:#5f6368;">Qwen3.6 35B-A3B 4-bit</th>
      <th style="text-align:right; padding:0.75rem; color:#5f6368;">Qwen3.8 27B 4-bit MTP</th>
    </tr>
  </thead>
  <tbody>
    <tr style="border-bottom:1px solid #e5e7eb;">
      <td style="padding:0.75rem;">M4 (16 GB)</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">38.4</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">—</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">—</td>
    </tr>
    <tr style="border-bottom:1px solid #e5e7eb;">
      <td style="padding:0.75rem;">M3 Pro (36 GB)</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">48.3</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">58.9</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">8.8</td>
    </tr>
    <tr style="border-bottom:1px solid #e5e7eb;">
      <td style="padding:0.75rem;">M3 Max (36 GB)</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">90.7</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">101.1</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">17.6</td>
    </tr>
    <tr style="border-bottom:1px solid #e5e7eb;">
      <td style="padding:0.75rem;">M4 Max (128 GB)</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">—</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">129.2</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">24.8</td>
    </tr>
    <tr style="border-bottom:1px solid #e5e7eb;">
      <td style="padding:0.75rem;">M5 Max (128 GB)</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">—</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">—</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">32.5</td>
    </tr>
  </tbody>
</table>

The pattern matters more than any cell. Qwen3.6 35B-A3B is a mixture of experts with about 3B parameters active per token, so it decodes faster than the dense 4B model on the same chip. The dense 27B model is several times slower everywhere.

So a 36 GB laptop runs a capable mixture-of-experts model at 59 to 101 tokens per second. A dense 27B model on the same machine runs at 9 to 18, which is usable for one person and not for a team.

## Does Rapid-MLX really run with zero cloud?

Not by default. Rapid-MLX runs inference on the Mac, and the README says prompts, completions, paths, IP addresses and API keys are never collected. But two defaults reach the network, and a security review should know about both before calling it offline.

<table style="width:100%; border-collapse:collapse; margin:1.5rem 0; font-size:0.95rem;">
  <thead>
    <tr style="background:#f5f5f0; border-bottom:2px solid #2175C5;">
      <th style="text-align:left; padding:0.75rem; color:#5f6368;">Default</th>
      <th style="text-align:left; padding:0.75rem; color:#5f6368;">What leaves the Mac</th>
      <th style="text-align:left; padding:0.75rem; color:#5f6368;">How to turn it off</th>
    </tr>
  </thead>
  <tbody>
    <tr style="border-bottom:1px solid #e5e7eb;">
      <td style="padding:0.75rem;"><strong>Telemetry (on since 0.15.0)</strong></td>
      <td style="padding:0.75rem;">Anonymous, metadata-only events; the engine sends batches to PostHog Cloud (US)</td>
      <td style="padding:0.75rem;"><code>rapid-mlx telemetry off</code>, <code>RAPID_MLX_TELEMETRY=0</code> or <code>DO_NOT_TRACK=1</code></td>
    </tr>
    <tr style="border-bottom:1px solid #e5e7eb;">
      <td style="padding:0.75rem;"><strong>Chat harness web tools</strong></td>
      <td style="padding:0.75rem;">Weather, web search and <code>web_fetch</code> queries, available to tool-capable models by default</td>
      <td style="padding:0.75rem;"><code>RAPID_MLX_CHAT_WEB_TOOLS=0</code> before starting a profile</td>
    </tr>
    <tr style="border-bottom:1px solid #e5e7eb;">
      <td style="padding:0.75rem;"><strong>Model downloads</strong></td>
      <td style="padding:0.75rem;">Weights fetched from Hugging Face on first use</td>
      <td style="padding:0.75rem;">Pre-stage models, then run offline</td>
    </tr>
  </tbody>
</table>

None of this is hidden; the project documents it in its [telemetry page](https://rapidmlx.com/docs/telemetry.html) and README. The release notes are also candid that the terminal harness's network policy is heuristic and terminal commands are not sandboxed.

The lesson is general. "Local" describes where the model runs. "Air-gapped" describes what the whole system can reach. Our [air-gapped AI service](/service/air-gapped-ai) exists because those are different claims, and only the second survives an audit.

## How many models does Rapid-MLX support?

Rapid-MLX's bundled registry, [`aliases.json`](https://github.com/raullenchai/Rapid-MLX/blob/main/rapid_mlx/aliases.json), listed 251 aliases on November 7, 2026, covering text, vision, image, video and embedding models, plus 53 audio aliases in a separate registry. An alias is a model at a given quantization, so distinct models are fewer.

Aliases are pinned. All 251 bundled aliases carry an exact Hugging Face revision hash, and a missing pin fails rather than silently switching weights. For a regulated team that is the more useful feature than the count: it means the model you approved is the model that runs.

Version 1.0.2 also accepts a pasted Hugging Face link and checks whether the model fits the Mac's memory before downloading, so the registry is a starting point rather than a ceiling.

## Where does a Mac fit in an enterprise AI stack?

A Mac running Rapid-MLX fits three roles: a developer workstation that keeps proprietary code off third-party APIs, an edge node with no network, and a cheap test bench for trying an open model on your own tasks before buying GPUs.

It is not a fleet server. The project's own LFM3 measurements on an M5 Max reached 951.5 tokens per second aggregate at 8 concurrent requests on a 1.7B model. Serving hundreds of employees on mid-size models still belongs on datacenter GPUs running vLLM or TGI.

Our guide to the [self-hosted LLM serving stack](/blog/self-hosted-llm-serving-stack-ollama-vllm-tgi) covers that split runtime by runtime. And [on-device agents](/blog/on-device-ai-agents-enterprise-next-moat) are where a fast Mac server earns its place.

One more caution from the security side. Rapid-MLX keeps a prefix cache and saves it to disk on shutdown, which is good for speed. Shared prompt caches are also the surface in the [cache side-channel research](/blog/cache-side-channel-attack-local-llm-on-premise-isolation) we covered in September.

## How does ibl.ai use a local endpoint like Rapid-MLX?

On ibl.ai you own all the code and the data, the platform is model-agnostic, there is no per-seat pricing, and you can deploy anywhere. An OpenAI-compatible endpoint on a Mac is one more model the platform can call.

Each agent's [LLM is chosen per agent](/docs/os/agent-settings/llm-selection) in [Agentic OS](/product/agentic-os), so a coding agent can run against a local Rapid-MLX server while others stay on a cloud model. Switching back is a setting, not a migration.

That is the real effect of releases like this one. Open-source inference on hardware you already own keeps getting faster, which keeps pushing [open-weight prices down](/blog/open-weight-models-price-floor-enterprise-ai). A stack you own can take each gain as it lands.

ibl.ai is family-owned and operated from New York, NY, and our engineers work alongside your team to wire local and private models into the systems you already run.

*Sources: release dates, features and the idle-memory figures from the Rapid-MLX [v1.0.0](https://github.com/raullenchai/Rapid-MLX/releases/tag/v1.0.0) and [v1.0.2](https://github.com/raullenchai/Rapid-MLX/releases/tag/v1.0.2) release notes; the 4x and 1.5x claims, telemetry and web-tool defaults from the [README](https://github.com/raullenchai/Rapid-MLX); the M5 Max comparison from [rapidmlx.com/compare](https://rapidmlx.com/compare); per-chip medians from the project's [speed-by-chip table](https://github.com/raullenchai/Rapid-MLX/blob/main/docs/benchmarks/speed-by-chip.md); alias counts from our count of the bundled registry. The brief cited 149 tok/s for Qwen3.6-35B on an M5 Max; no Rapid-MLX source we found shows that figure, and the project's published M5 Max number is 173.4 tok/s. All benchmarks are the project's own or community-submitted.*

## Why does owning the AI stack matter?

**ibl.ai is the agentic AI platform where you own all the code and the data. You self-host the entire stack inside your own perimeter, run it model-agnostic across any LLM and switch anytime, and pay by usage with no per-seat pricing — so you can deploy anywhere: your cloud, on-premise, GovCloud, or fully air-gapped.**

- **You own all the code and the data.** Full source code under a perpetual license, running on your infrastructure. Not API access to someone else's platform — the stack itself is yours.
- **Model-agnostic.** Run any LLM — Claude, GPT, Gemini, Llama, Command, or your own fine-tune — and switch providers without rewriting the platform.
- **No per-seat pricing.** Usage-based billing against a budget cap you set. Cost tracks what your organization actually uses, not how many people you employ.
- **Deploy anywhere.** Your cloud, your VPC, on-premise, GovCloud, or a fully air-gapped network with no outbound connectivity.

1.6M+ users across 400+ organizations run the platform this way, including NVIDIA, MIT, and Syracuse University.

ibl.ai is family-owned and operated from New York, NY — a U.S.-headquartered, domestically-owned long-term partner, not a vendor that sells licenses and moves on.