On-device LLM updates 2026, and the 3 Swift lines that point any of them at your Mac

Three streams shipped at once. First, Apple's Foundation Models framework, introduced at WWDC 2025 and matured through Q1 and Q2 2026, exposes the ~3B parameter on-device foundation model behind Apple Intelligence to any Swift app through a stable API, with guided generation that constrains output to a Swift type at the token level. Second, MLX became the default inference framework for Apple Silicon after three dedicated WWDC 2025 sessions, then landed inside Ollama on March 30, 2026 (v0.19), with measured prefill of 1,810 tokens per second and decode of 112 tokens per second on M5 Max running Qwen3.5-35B-A3B in NVFP4 quantization. Third, the open-weight calendar concentrated into a 48-hour cluster on April 28 and 29: Xiaomi MiMo-V2.5-Pro (1.02T MoE, MIT), NVIDIA Nemotron 3 Nano Omni (30B / 3B active, NVIDIA Open Model Agreement), IBM Granite 4.1 (3B / 8B / 30B dense, Apache 2.0), and Mistral Medium 3.5 (128B dense, Modified MIT). OpenBMB then shipped MiniCPM-V 4.6 1.3B under Apache 2.0 on May 11. Five entries in three streams. Everything else in the on-device space this year is a point release inside one of those three.

Yes, the practical change is that every Apple Silicon Mac running macOS 26 Tahoe ships with a built-in LLM that a third-party app can call without bundling weights, without a network round-trip, and without standing up its own inference server. The Foundation Models framework wraps the same ~3B parameter on-device model that powers Siri, Writing Tools, and notification summaries, and exposes it through Swift. The interesting engineering work is the guided generation feature, which is constrained decoding at the token level against a Swift @Generable type. You declare the shape you want, the model is forced to produce it. The constraint is what makes the framework usable for tool calls and structured extraction, not just text generation. The model is small (~3B), not designed as a general-knowledge chatbot. For the cases where 3B is enough, the integration is one Swift import; for the cases where you need a frontier model, you fall back to the Anthropic-compatible-proxy path described below.

It is the difference between Apple Silicon being a place you can technically run a model and a place you actively prefer for inference. MLX is designed around the unified memory architecture: the CPU and GPU share the same physical RAM, so tensor operations cross the boundary with zero-copy, eliminating the CPU-to-GPU transfer overhead that dominates traditional frameworks on discrete GPUs. The 2026 milestone was Ollama v0.19 on March 30, 2026, which made MLX the inference backend on Apple Silicon, not just an optional flag. On M5 series chips the GPU Neural Accelerators apply to both time-to-first-token and tokens per second; Ollama's own blog post on the MLX backend reports prefill jumping from 1,154 to 1,810 tok/s and decode from 58 to 112 tok/s on M5 Max running Qwen3.5-35B-A3B in NVFP4. Those are not theoretical numbers from a research paper, they are the numbers the runtime hits when you `ollama serve` and point a client at localhost:11434. The same generation of changes landed in raw mlx-lm, in LM Studio, and in third-party clients that wrap MLX directly.

The realistic Mac targets sort by parameter count and memory budget. For 64GB Apple Silicon, IBM Granite 4.1 (3B/8B/30B dense, Apache 2.0, up to 512K context) is the cleanest pick, both for license simplicity and because the 30B dense fits with room for a long context. Mistral Medium 3.5 (128B dense, 256K context, Modified MIT) is borderline on 64GB but comfortable on 128GB and benchmarks at 77.6% on SWE-Bench Verified, which makes it the realistic coding pick. NVIDIA Nemotron 3 Nano Omni (30B / 3B active multimodal, runs in 25GB of RAM) is the most efficient multimodal entry, useful when you want the model to read what is on screen as part of an agent loop. Llama 4 70B Q4_K_M sits in roughly 40GB and runs at 25-32 tok/s on M5 Max, which is interactive for chat. Xiaomi MiMo-V2.5-Pro (1.02T MoE, 42B active) is the highest-capability open release but its 42B active footprint is tight on 64GB; it is a 128GB-and-up target. OpenBMB MiniCPM-V 4.6 1.3B is too small to be the planner in an agent loop but is the right always-on screen reader for a Mac that already has a frontier model behind a proxy.

Most articles in this space stop after listing the models or the runtimes. The model is necessary, the runtime is necessary, and neither one is sufficient. To actually drive your real Mac you need an agent loop that reads the macOS accessibility tree, plans multi-tool sequences, and clicks, types, scrolls inside arbitrary apps. The model layer is the brain, the agent loop is the body. The 2026 picture is that the brain layer matured fast and the body layer is still where most of the engineering work has to land. Fazm is one specific implementation of that body layer: a native Swift app that wraps the Claude Code agent loop via ACP, exposes nine macos-use tools that read and act on the accessibility tree, and lets you swap the brain to any Anthropic-Messages-compatible endpoint, including an on-device proxy in front of an MLX runtime or a local Ollama server. The integration surface is small enough to read in one sitting; see the next question for the exact line numbers.

Lines 527 to 530 of Desktop/Sources/Chat/ACPBridge.swift. Three lines, plus the comment. The block reads the `customApiEndpoint` string from UserDefaults and, if it is non-empty, writes it to `env["ANTHROPIC_BASE_URL"]` on the spawned ACP subprocess. That is the entire mechanism. Anything that speaks the Anthropic Messages API on a local port (LiteLLM in front of Ollama, LiteLLM in front of vLLM, an MLX-backed proxy, an Apple Foundation Models adapter wrapped in an Anthropic shim) becomes the brain of the agent. The ACP subprocess is the unmodified Claude Code agent loop running locally as a Node process; it sends every model request to ANTHROPIC_BASE_URL, gets back tool calls, executes them against the local macos-use and playwright MCP servers, and feeds the responses back to the model. The code is open at github.com/m13v/fazm and the path is stable; verify it yourself.

It depends which definition of offline you mean. If offline means no inference traffic leaves the Mac, yes: pull weights once, point a local proxy at a local MLX or llama.cpp or Ollama server, and the entire agent loop runs without touching the network. The accessibility tree, the tool execution, the chat state, the personal context profile, all of that already stays local in Fazm by design. If offline means no software downloads ever again, no: the agent loop itself, Claude Code via ACP, is Node code that you keep updated through normal channels, and the bundled binaries (Node, ffmpeg, cloudflared) live in Fazm's resources directory and get updated through app releases. The middle ground that most people land on is hybrid: a local 8B or 30B model for high-frequency, low-stakes turns (reading the screen, picking the next click) and a frontier model for hard reasoning, with the proxy in the middle deciding which one to call. The proxy is where you spend the engineering effort; the agent loop just sends requests.

Two things. First, none of the major on-device runtimes ship an Anthropic Messages API natively; you still have to put a shim (most commonly LiteLLM) between the runtime and any agent that expects the Anthropic shape. The conversion is mechanical but it adds a process and a port. Ollama in particular has been adding `ollama launch <integration>` targets all year (`pi`, `hermes`, `kimi-cli`, `cline`, `opencode`, `claude-desktop`) but not a native Anthropic surface on top of its own API. Second, none of the on-device model releases come with the tool-use training that frontier hosted models have. Open-weight models will produce a tool call when you ask, but the depth of multi-step planning is noticeably weaker, especially under the kinds of long-horizon prompts that an agent generates. The practical implication: a hybrid setup is still the right answer for actual agent work in mid-2026, with the local model handling the high-volume cheap turns and a frontier model handling the planning turns.

Each entry traces to a primary source. Apple Foundation Models updates: machinelearning.apple.com/research/apple-foundation-models-2025-updates and the Foundation Models page in developer.apple.com/documentation. MLX as the WWDC 2025 default and the Apple Silicon throughput numbers: the Ollama blog post 'Ollama is now powered by MLX on Apple Silicon in preview' on ollama.com/blog/mlx. Ollama 2026 release-by-release changelog: github.com/ollama/ollama/releases. The April 28 to 29 open-weight cluster: each model's Hugging Face page (XiaomiMiMo/MiMo-V2.5-Pro, nvidia/Nemotron-3-Nano-Omni-30B, ibm-granite/granite-4.1, mistralai/Mistral-Medium-3.5). MiniCPM-V 4.6: huggingface.co/openbmb/MiniCPM-V-4_6. Fazm's three-line block: github.com/m13v/fazm at Desktop/Sources/Chat/ACPBridge.swift, lines 527 to 530 (verify against the current commit on main; the path is stable but line numbers shift with refactors).