Examples of business process automation, organized by the UI layer the target app lives in

Because industry grouping (sales, HR, finance, supply chain) is useful for picking motivation but useless for picking tooling. Two 'accounts payable' automations can look identical on paper and have completely different feasibility: one runs entirely in QuickBooks Online (API-native, any iPaaS works), one runs in QuickBooks Desktop for Mac (no public API, no Zapier connector, no UiPath activity pack that reliably reads the tree). The UI layer decides whether the automation can be built at all. The department it sits inside does not.

Layer 1 is API-native: the target app has a documented public API or a vendor-supported connector in Zapier, Make, n8n, Workato, or Power Automate Cloud. Layer 2 is browser-UI: the target is a web app with no public API, but you can drive its HTML through Playwright, Selenium, or a browser-first RPA like Bardeen. Layer 3 is desktop-only: the target is a native macOS or Windows app with no public API and often no web equivalent. Layer 3 is where most 2026 BPA listicles go quiet, and it is where most real small-business workflows actually run.

A reconciliation bookkeeper pulls a statement PDF from an online banking dashboard (Layer 2, browser), opens it in Preview, copies the transaction table into Numbers, applies a category lookup against a master sheet, then exports a PDF of the categorized report and attaches it to a Mail message to the accountant. Zero of those four apps (Preview, Numbers, Keynote, macOS Mail) ship a public automation API. All of them expose a full accessibility tree. The accessibility tree is what `mcp-server-macos-use` (the binary wired into Fazm at `acp-bridge/src/index.ts` line 1057) reads and drives. No screenshots, no OCR, no pixel guessing.

Apple's AXUIElement API exposes every onscreen element as a tree node with a role (AXButton, AXTextField, AXMenuItem, AXRow, AXCell, AXStaticText), a label, x/y coordinates, width/height, and visibility. The `mcp-server-macos-use` Swift binary (version 0.1.17, BSL 1.1, source at github.com/mediar-ai/mcp-server-macos-use) traverses that tree on demand, writes a structured dump to `/tmp/macos-use/<timestamp>_<action>.txt`, and lets the model find elements by role and partial text match rather than by pixel coordinate. Each line has the format `[AXButton (button)] "Open" x:680 y:520 w:80 h:30 visible` (documented in `llms.txt` line 115). Input goes through CGEvent, indistinguishable from a real mouse or keyboard at the OS level.

Six tools: `open_application_and_traverse` (launch or focus an app, read its tree), `click_and_traverse` (click a button, then re-read the tree, optionally typing and pressing a key in the same call), `type_and_traverse` (type into the focused field), `press_key_and_traverse` (fire a key with optional modifiers), `scroll_and_traverse` (scroll by N lines at a coordinate), and `refresh_traversal` (re-read without acting). Every example in the section 'Twelve desktop-only BPA examples the other listicles skip' decomposes into a short sequence of these six calls. The spec is pinned in `/Users/matthewdi/mcp-server-macos-use/llms.txt` lines 19 to 75.

Two failure modes. First, brittleness: a 1-pixel UI shift, a theme change, a language change, or a different monitor scale invalidates the template match. Accessibility trees are addressed by role and label, not by pixel location, so none of those invalidate the match. Second, token cost: returning a screenshot to an LLM for every step costs thousands of tokens. Returning a structured tree line (`[AXButton (button)] "Open" x:680 y:520 w:80 h:30 visible`) costs dozens. For a ten-step workflow, that is a 100x cost delta. The Fazm chat prompt routes intentionally: `macos-use` tools for desktop apps, Playwright for web pages, `capture_screenshot` only when the user literally asks 'what does this look like'. The routing rule is in `Desktop/Sources/Chat/ChatPrompts.swift` line 60.

Two mechanisms, both in `mcp-server-macos-use`. InputGuard blocks keyboard and mouse input from the user during the action so a stray click does not land in the middle of an automated click sequence. A 30-second watchdog prevents a permanent lockout if the server crashes mid-action, and pressing Escape cancels immediately. A floating overlay tells the user what is being driven, and the mouse cursor position is saved and restored around each action so the pointer does not visibly jump. All four are documented in `llms.txt` lines 112 to 114 of the same file.

Yes, and it is the single feature that makes multi-app BPA examples practical. When `click_and_traverse` causes a different app to become frontmost (a classic case: clicking a mailto: link opens Mail, clicking a PDF attachment opens Preview, clicking 'Open in Numbers' opens Numbers), the server detects the new frontmost process, re-reads the tree of the new app automatically, and returns the new tree. The caller never has to track process IDs by hand. Documented in `llms.txt` line 112 under 'Cross-app handoff'.

Functionally yes, packaging no. The primitive set (click, type, press, scroll, read) matches every RPA vendor's primitive set. The packaging is a consumer Mac app: install the .app, grant accessibility permission once, then describe the workflow in chat. No activity library to learn, no studio canvas to wire, no orchestrator to deploy, no bot runner license to buy. The trade is expressiveness for reach: you give up the visual canvas of UiPath Studio and get a chat surface that covers every app on the Mac including the ones UiPath never built a connector for.

API automation is more reliable when an API exists. If Salesforce or Stripe or Linear exposes an endpoint, use the endpoint. Accessibility-API automation is strictly a fallback for apps that have no endpoint. Within that fallback, it is more reliable than pixel-based RPA because the tree is a structured contract the OS maintains, not an image the UI designer can accidentally break with a theme refresh. The failure modes are different: APIs fail on rate limits and auth; accessibility automation fails when an app renders its UI in Canvas or WebGL and skips the accessibility tree entirely (rare; some games, some chart-heavy analytics tools). For the long tail of business apps (Office for Mac, iWork, Mail, Finder, accounting desktop software, Figma desktop, Slack, Discord, Outlook, Adobe CC), the accessibility tree is populated and usable.

Yes. Grant accessibility permission to any AX-aware tool (Script Editor, Accessibility Inspector, or Fazm itself), trigger a traversal, and inspect `/tmp/macos-use/`. Every file there is plain text, one element per line, in the documented format. Run `rg -n AXButton /tmp/macos-use/*.txt` after a Fazm session and you will see dozens of matches. Same format documented in the public llms.txt at github.com/mediar-ai/mcp-server-macos-use.

Fazm is a consumer Mac app whose chat surface routes every request to the right layer. Web requests go to Playwright (attached to real Chrome via the Playwright MCP Bridge extension, ID `mmlmfjhmonkocbjadbfplnigmagldckm`). Desktop requests go to `mcp-server-macos-use`. API requests go to the skill for the relevant service (`telegram` skill uses the telethon API, `google-workspace` skill uses a bundled Python MCP, etc.). The routing rules are explicit in `Desktop/Sources/Chat/ChatPrompts.swift` starting at line 55 under the `<tools>` block. One session can use all three layers in a single chat turn.