The open source local desktop agent question is mostly a permission-probe question

The lie the operating system tells

The macOS accessibility API has one entry point everyone reaches for first: AXIsProcessTrusted(). It returns a Bool. The Bool is supposed to mean "this process can call accessibility APIs against other apps." In practice, on macOS 26 (Tahoe), and on Sequoia after an OS update or an app re-sign, that Bool is cached per process and the cache goes stale. The function returns true. The next AX call fails. The user grants permission again. The function still returns true. The next AX call still fails. The agent looks broken. The agent is not broken. The cache is.

A second-order failure is layered on top of the first. The error code that comes back from a failing AX call is AXError.cannotComplete, and it is ambiguous on purpose. Sometimes it means the permission is broken. Sometimes it means the frontmost app does not implement accessibility at all, which is true of most Qt apps, most OpenGL apps, some Electron builds, and PyMOL. An agent that treats every cannotComplete as a permission failure will nag the user for permission every time they tab to PyMOL. An agent that treats every cannotComplete as a foreign-app problem will silently stop working the day TCC drifts.

The four probes, in order

The probe lives in Desktop/Sources/AppState.swift between line 310 and line 540. The names of the methods are unglamorous:checkAccessibilityPermission, testAccessibilityPermission, confirmAccessibilityBrokenViaFinder, and probeAccessibilityViaEventTap. They run as a sequence, with short-circuit exits at each step.

Probe 1, the cheap one

Call AXIsProcessTrusted(). If it returns false on a process that previously had permission, do not believe it. The cache is the most likely culprit, and there is a cheaper way to confirm than re-prompting the user. Skip ahead to probe 4. If it returns true, do not believe that either. Run probe 2.

Probe 2, the real AX call

Get the frontmost app from NSWorkspace.shared.frontmostApplication, build an AXUIElement for it, and try to read kAXFocusedWindowAttribute. The four meaningful return codes are: success (everything is fine), noValue (the app has no windows, also fine), apiDisabled (system-wide AX is off, unambiguous failure), and cannotComplete (ambiguous, run probe 3). notImplemented and attributeUnsupported are also treated as fine because they come from healthy AX calls against apps that just do not expose a focused window attribute.

Probe 3, the Finder sanity check

When probe 2 returned cannotComplete, the question is whether the failure was caused by the permission or by the foreign app. The cheapest way to find out is to repeat the same call against an app that is known to implement accessibility correctly. Apple ships one: Finder. If Finder succeeds, the original failure was app-specific (probably PyMOL or a Qt build), the permission is fine, and the agent should keep going. If Finder also fails, the permission is the problem. If Finder is not even running, fall through to probe 4.

Probe 4, the CGEvent tap escape hatch

This is the part the existing playbooks do not mention. CGEvent tap creation is governed by the same accessibility entitlement as AX calls, but unlike AXIsProcessTrusted it does not consult the per-process cache. It hits the live TCC database. Calling CGEvent.tapCreate with .listenOnly on mouseMoved is a probe, not a real subscription: the tap is invalidated immediately. If it returned a non-nil port the entitlement is live and AX will recover after a process restart. If it returned nil the permission is genuinely missing and the user has to grant it. Fazm only runs this probe when the previous step had said the process previously had permission, to avoid triggering the "app was prevented from modifying other apps" system notification on every polling cycle for a process that has just been installed.

What the agent does after the probe agrees

A clean four-step probe is necessary, not sufficient. The probe gives the agent a trustworthy yes-or-no on whether AX is alive. Once that is settled, the agent still needs to actually drive applications. Fazm hands that work to a small set of MCP-style tool families: native macOS apps go through mcp__macos-use__* (Finder, Settings, Mail), the browser goes through Playwright attached to the user's real running Chrome (not a headless instance), WhatsApp uses mcp__whatsapp__* on the native macOS Catalyst app, and Telegram is driven by Python telethon scripts because that is faster and steadier than browser automation.

The routing rules live in Desktop/Sources/Chat/ChatPrompts.swift and they are part of the system prompt the agent receives every turn. The interesting consequence: when the agent picks up a new app, it does not have to be retrained, it has to be added to the routing block. New tool families plug in via ~/.fazm/mcp-servers.json, the user-side MCP server registry that landed in v2.4.0 on 2026-04-20 and is read by Desktop/Sources/MCPServerManager.swift.

The shape of an open source local desktop agent on macOS, then, is two layers: a probe that decides whether the OS will let the agent see the screen at all, and a router that decides which tool family talks to which surface. Most of the comparison energy in this category is spent on the second layer. Most of the failure is in the first.

What to grep for when picking an agent

You can audit any open source local desktop agent on macOS in fifteen minutes if you know what to look for. Clone the repo. Grep for these strings:

• AXIsProcessTrusted: the entry point. One hit is the bare minimum. Zero hits means the project does not even ask whether it has permission.
• kAXFocusedWindowAttribute or AXUIElementCopyAttributeValue: the real call. If grep returns hits inside a permission-check function, the project is doing a functional test, not just trusting the cache.
• CGEvent.tapCreate or CGEventTapCreate: the escape hatch. A repo with this in a permission probe knows about the macOS 26 cache problem and has handled it.
• cannotComplete: the ambiguous error. A repo that branches on this and has a Finder fallback or equivalent cross-app sanity check is built by someone who has been bitten in production.

The hit count for those four strings, plus a quick read of the function that contains them, is a faster signal of whether the agent will survive on a real Mac than any feature comparison.