llama.cpp release April 2026, read as a swap-in backend for a Mac agent

Roughly the b8800 through b8925 build range. The ones that matter for a Mac agent are b8920 (metal: print GPU description on server boot, so you can verify which GPU is doing the work), b8922 (WebGPU flash attention enhancements with a fallback tile path for older browsers), a prefix-caching fix in the server for Anthropic-style clients, a new --endpoint option on llama-cli so it can act as a client to a remote llama-server, and b8925 (a structured-output parser fix). The Hexagon and KleidiAI work in b8914 and b8924 is not load-bearing on an Apple Silicon Mac.

Fazm reads one UserDefaults key named customApiEndpoint. The ACP bridge picks that value up at process start and exports it as the ANTHROPIC_BASE_URL environment variable for the Claude agent subprocess it spawns. The actual code path is in Desktop/Sources/Chat/ACPBridge.swift at lines 380 and 381. If your local llama-server exposes an Anthropic-compatible surface (or sits behind a shim that does), put that URL in Fazm Settings under Advanced, AI Chat, Custom API Endpoint. Fazm itself does not care whether the other side is Anthropic's real API or a loopback address.

Not out of the box. Fazm ships with a cloud Claude backend by default because Claude still beats every local model at long-horizon desktop automation. The custom endpoint is a hook for users who want to experiment or who have compliance constraints. The reads from your Mac (accessibility tree, file index, audio transcription) are already on-device. What crosses the network is the model request. Pointing customApiEndpoint at 127.0.0.1 moves that leg to your machine too. This is why the v2.3.2 changelog (April 16, 2026) rewrote the privacy language from 'nothing leaves your device' to 'local-first' - honest, not marketing.

Because on a Mac you often have more than one GPU path available (the integrated Apple GPU, an eGPU, or in Asahi Linux setups, a discrete card). Up until b8920, llama.cpp booted silently if Metal initialization succeeded. After b8920 the server prints the GPU name, core count, and a few capability flags at startup. If you are wiring this into Fazm via a reverse proxy and throughput drops, that one log line tells you whether llama-server actually picked up the M-series GPU or fell back to a slower path. This is the single most useful debugging affordance the April builds added for Mac users.

The April 2026 weekly report flags a prefix-caching fix in the server for the Anthropic API path. Before the fix, repeated long-system-prompt calls (exactly the kind of traffic a desktop agent generates, because the accessibility tree snapshot is prepended every turn) were cache-missing when the client used the Anthropic-style content array shape. After the fix, the prefix cache recognizes the stable system block across turns. For a Mac agent that sends a 3-10 KB tree every message, the difference is measurable - cache hits on the prefix skip the prefill step entirely.

It turns llama-cli into a thin client that talks to a remote llama-server over HTTP instead of loading the model locally. Useful if you keep a big-RAM machine on your network running the server and want a laptop-side CLI that chats to it. For Fazm, this is adjacent rather than direct - Fazm goes through the ACP bridge, not llama-cli. But if you want to sanity-check your llama-server is working before wiring it into Fazm, llama-cli --endpoint http://127.0.0.1:8080 is the one-liner.

On an M3 Max or M4 Max with 48 to 96 GB of unified memory, you can run a Q4_K_M quant of a 70B class model with reasonable prompt-processing speed. For the specific shape of Fazm's traffic (short system context, a 2-10 KB accessibility tree, tool-call-heavy turns), latency matters more than raw throughput because every user turn is round-tripped. A 30B class instruct-tuned model at Q5 is usually the sweet spot in practice. Below 13B, tool-calling accuracy on structured UI trees drops off fast.

Only if you run llama.cpp's WebGPU backend in a browser, which is not the path a native Mac agent takes. The Metal backend is still the right target on Apple Silicon. The WebGPU work is interesting for hosted demo pages, not for a signed, notarized Mac app talking to a loopback server.

The Claude agent Fazm spawns speaks the Anthropic content-array shape, so pointing it at a raw OpenAI-schema endpoint will return 400s for the first tool-heavy turn (content blocks with tool_use will not deserialize). You need a shim in between - something like LiteLLM's Anthropic-compat layer, or one of the small OSS proxies that translate between the two. The address you put in customApiEndpoint should be the shim, not llama-server directly. The fact that llama.cpp's April builds fix prefix caching specifically for the Anthropic API path is what makes this round-trip cheap enough to be worth doing.

Because the cheapest thing a local model can consume is structured text. Fazm reads the real macOS accessibility tree via AXUIElement for whatever app is under the cursor and passes that tree in. A 7 KB tree is a few thousand tokens. A screenshot of the same screen, even at tight JPEG quality, is 10x the token cost after vision encoding plus significantly more latency. Screenshot-based Mac agents can technically swap their cloud backend for a local model too, but the vision step destroys the latency budget well before the language model does. With an accessibility-tree pipeline, local inference is actually tractable.