A local AI that can use your other apps, not just run on your Mac

A local model means a neural net running on your hardware (Ollama, LM Studio, MLX, Foundry Local). A local host AI is the model plus everything around it: permissions, IO, and the ability to reach other apps on the same machine. If your model is local but it cannot read what is in Mail or type into Keynote, then your local host AI is a fancy chat window. Fazm's answer is that the local model calls a Mac-native host that talks to the OS through Accessibility APIs.

Three reasons. First, the AX tree is a structured string graph; screenshots are pixels that have to be re-read by a vision model on every turn. Second, the AX tree already has roles (AXButton, AXTextField, AXStaticText) so the agent can target elements semantically instead of guessing what a button looks like. Third, it is cheap: one AX call is a local function, a screenshot is a PNG plus an inference round trip. Fazm uses screenshots only as a fallback for apps that do not expose AX properly (Qt, some Electron builds).

Desktop/Sources/AppState.swift in the open-source Fazm repo. Function names: testAccessibilityPermission (lines 433-463), confirmAccessibilityBrokenViaFinder (lines 468-485), probeAccessibilityViaEventTap (lines 490-504). The public retry timer that wraps them all is schedulAccessibilityRetry (lines 376-400). Everything is MIT licensed.

On Tahoe, each process has a cached view of which TCC permissions it owns. When the user toggles Accessibility for Fazm in System Settings while Fazm is running, AXIsProcessTrustedWithOptions starts returning true immediately, but the live AX calls continue returning cannotComplete or apiDisabled until Fazm restarts. Fazm sidesteps the cache entirely by testing with a real AX call and, if that is ambiguous, a CGEvent.tapCreate call which checks the live TCC database. If both come back bad it asks the user to relaunch.

By default, no. The AX tree never leaves your Mac. When you explicitly ask for screen context, ScreenCaptureManager uses CGWindowListCreateImage to capture the target window and feeds it into the model. You can see the captures in /tmp/fazm.log. If you pair Fazm with a local-inference backend like Ollama or LM Studio, even that round trip stays on your host.

Any app that implements AX. That includes Apple's whole family (Finder, Mail, Notes, Calendar, Messages, Xcode, Numbers, Keynote, Pages, System Settings), Safari and Chrome (including extension UIs), native Electron apps when they remember to set accessibility flags, Slack, Discord, Notion, Linear, Figma, Zoom, VS Code, Terminal, iTerm, and so on. The floating control bar detects the frontmost app and pulls its window tree regardless. Qt and raw SDL apps are the usual misses.

Ollama gives you a model endpoint on localhost:11434. It does not know anything about your running apps. You could wire pyatspi or pyobjc on top, deal with the AX framework in Python, negotiate TCC permissions per process, and build a retry loop for macOS 26 Tahoe. Fazm does all of that in Swift and ships it as a signed DMG. The model half is interchangeable; Fazm talks to Claude by default but it can point at any local endpoint. The host half is the unique work.

Fazm keeps running but the agent loses its hands. You can still chat and the app can still render screen captures, but it cannot click, type, or read widget text. A floating hint appears in the control bar and a settings pane has a one-click tccutil reset flow that triggers an NSAlert and relaunches the app cleanly. The code path is resetAccessibilityPermissionDirect in AppState.swift line 544.

Yes. Fazm's desktop is MIT-licensed at github.com/mediar-ai/fazm. Every file and line number cited on this page exists in that repo on 2026-04-19. The three functions that form the three-stage permission probe are all in Desktop/Sources/AppState.swift. The screen capture fallback is in Desktop/Sources/FloatingControlBar/ScreenCaptureManager.swift. The floating UI that reacts to the isAccessibilityBroken flag is next to it.