How to Set Up a Desktop AI Agent on Mac, Including the Third Permission Nobody Tells You About

The two-permission answer is wrong

The generic setup guide for a Mac desktop AI agent is: download, open System Settings, toggle Screen Recording, toggle Microphone, relaunch. That is correct for agents that work by taking a screenshot, sending the image to a vision model, and clicking on pixel coordinates the model returns.

It is not correct for agents that read the real UI tree. Fazm, Agent! by Anthropic, and Manus all need a third permission called Accessibility. Accessibility is a separate pane in System Settings from Privacy and Security, Screen Recording. Granting Screen Recording does not grant Accessibility. Without it, every AXUIElement call the agent makes returns nothing.

The reason most setup guides skip Accessibility is that they are written for the screenshot class of agents. Those do not need the UI tree because they are operating on pixels. If you are picking an agent, the permission list tells you which architecture you are getting.

What each permission actually unlocks

The three permissions map to three different pipelines inside the agent. One permission, one pipeline. Missing one quietly disables that pipeline.

The setup flow, in order

Order matters here. Granting Accessibility before the app has been launched once produces a TCC entry with no app icon, and some macOS builds get confused about matching the entry to the app bundle later. The safe order is launch first, grant second, relaunch third.

The hidden third failure mode: TCC cache drift on macOS 26

Here is the thing the setup guides never cover. On macOS 26 Tahoe, Apple moved accessibility permission checks behind a per-process cache. The first time a process calls AXIsProcessTrusted(), the result is cached for the lifetime of that process. That sounds like a performance optimization and it is, but it interacts badly with app updates, app re-signs, macOS upgrades, and any event that invalidates the TCC entry.

When the cache goes stale you end up in a state where the toggle in System Settings reads ON, the TCC database on disk has your app marked as allowed, and the running process still thinks it is not trusted. Every AX call fails silently. Most agents misreport this as "permission denied" and send you back to System Settings, which already looks correct, which makes you think the agent is broken.

The fix is to bypass the cache. Fazm does it with a probe that creates a live CGEvent tap. Event tap creation hits the live TCC database, not the cache. If the tap succeeds, the permission is actually granted and the cache is lying.

What a stale-permission session actually looks like

If you want to see the failure mode without triggering it on your own machine, here is the sequence of log lines Fazm writes when it detects and recovers from a stale TCC entry. Every line has a real emitter in AppState.swift.

The numbers behind the recovery

Fazm does not loop forever. The retry window is deliberately small so the user gets a fast answer either way. These constants live in AppState.swift at the top of the accessibility block.

Why Accessibility, not Screen Recording, is the critical permission

Screen Recording gives the agent a picture. Accessibility gives the agent a sentence. Those are not the same thing.

When Fazm reads the UI with the Accessibility API, it gets a structured tree where every element has a role (button, text field, menu item), a title ("Send", "To:", "Save As..."), a position, a size, and a list of valid actions it can perform. A screenshot has none of that. A screenshot has pixels. The model has to re-recognize the UI from those pixels every single turn.

The live recovery flow

When the retry timer finishes without recovering, Fazm does not just log an error. The app shows a modal with a Quit and Reopen button, spawns a detached sh subprocess that sleeps 1 second then runs open <bundlePath>, then terminates the current process. The new process starts with a fresh TCC cache.

When to suspect a stale permission instead of a broken agent

There are four signals that tell you the permission is stale, not missing, so you can skip re-granting and go straight to a relaunch.

• The toggle in System Settings, Privacy and Security, Accessibility is already on for the app.
• The agent worked yesterday and stopped working after you installed an app update or upgraded macOS.
• Other accessibility-aware apps (Raycast, BetterTouchTool) also started behaving oddly around the same time.
• The agent's logs show AXError.cannotComplete or AXError.apiDisabled instead of "permission denied".

In all four cases the answer is: quit the app, reopen it. If that does not work, toggle Accessibility off and on. If that still does not work, remove and re-add the app in the Accessibility pane.

0 skills installed on first launch

One thing that differentiates a consumer desktop agent setup from a developer framework setup is what happens between clicking Done and the first useful task. Fazm ships 17 bundled Claude skills (pdf, docx, xlsx, pptx, video-edit, frontend-design, canvas-design, doc-coauthoring, deep-research, travel-planner, web-scraping, social-autoposter, ai-browser-profile, google-workspace-setup, telegram, find-skills, plus onboarding helpers) and installs them to ~/.claude/skills/ on first launch with a SHA-256 checksum update loop.

That matters for setup because the first real task you give the agent does not have to be a generic one. You can open the onboarding flow, pick a skill ("help me plan a trip", "convert this .docx to PDF"), and it already has a working runtime for that task. No pip install. No MCP server wiring. The setup is done when the permissions are green.

Setup FAQ