AI agents and legacy desktop apps: what the accessibility tree actually returns.

On macOS, often yes. The accessibility API was introduced in Mac OS X 10.2 (2002), so most apps shipped after that, even ones written in Carbon or older AppKit, register at least a partial NSAccessibility tree. Fazm reaches them by calling AXUIElementCreateApplication(pid) on the app's process and walking kAXChildrenAttribute. Where it fails: apps that opted out of accessibility entirely (some Qt builds without the AT-SPI bridge, most Tk/Python apps, OpenGL or Metal canvases, ancient Carbon dialog boxes that draw their own controls). Those return AXError.cannotComplete from every call. The honest workflow on macOS is AX-first, vision fallback when AX is empty, and explicit disambiguation when the error code is ambiguous.

AXError.cannotComplete is what AXUIElementCopyAttributeValue returns when the call cannot be answered, but the OS does not tell you why. The same code is returned for at least three different conditions: the calling process has lost accessibility trust (the user revoked it, or the macOS 26 TCC cache went stale after an app re-sign), the target legacy app never implemented NSAccessibility, or the target app is alive but unresponsive. An agent that retries on cannotComplete will spin on a Tk app forever. An agent that maps it to 'permission broken' will nag the user every time they open Blender or PyMOL. You have to disambiguate before you act.

It re-runs the same AX call against a control app that is always installed and always implements accessibility correctly: Finder. The function is confirmAccessibilityBrokenViaFinder() in Desktop/Sources/AppState.swift around line 517. If Finder also returns cannotComplete, the permission is genuinely stuck and the agent surfaces a Quit and Reopen dialog. If Finder succeeds, the original failure was app-specific, the permission is fine, and the agent logs the suspect app as AX-incompatible and falls back to vision. There is also a second probe via CGEvent.tapCreate, used when Finder is not running, because event-tap creation reads the live TCC database and bypasses the per-process AXIsProcessTrusted cache.

Anything that ships as a native AppKit or SwiftUI app, including older versions. Apple's own legacy bundles (older Pages, Keynote, Numbers, Aperture from before it was discontinued, classic iWork) all expose usable trees. Most third-party native apps that were on the Mac App Store before 2018 do too: BBEdit, MarsEdit, Things, OmniGraffle, Coda, Transmit, classic Photoshop CS5 era builds. Java apps work if the user installed Java with the JavaAccessBridge enabled, which is the default for current Adoptium and Azul builds. Carbon apps built before Apple deprecated it work as long as they did not draw their own control widgets. The general pattern: if VoiceOver can read the app, an accessibility-tree agent can drive it.

Five categories. First, Qt builds where the developer did not link the QAccessible AT-SPI bridge (most Linux-cross-compiled scientific tools fall here). Second, Tk and Tcl apps including PyMOL, IDLE, and most Python tutorial apps. Third, OpenGL or Metal canvases that draw all their UI as pixels (Blender, many CAD tools, game engines). Fourth, Electron apps below ~v25 without --force-renderer-accessibility, which expose mostly generic AXGroup nodes you cannot dispatch on. Fifth, very old Carbon apps (think pre-2008 shareware) that draw control widgets via QuickDraw and never registered an AX role. For all five, the right move is: detect the AX gap, stop calling AX on this window, capture pixels with ScreenCaptureKit, route through OCR or a vision model.

Three concrete spots in github.com/m13v/fazm. AppState.swift around lines 480 to 512 is testAccessibilityPermission(): the switch statement that maps each AXError to a concrete decision. AppState.swift around lines 514 to 534 is confirmAccessibilityBrokenViaFinder(): the control-app probe. AppState.swift around lines 536 to 553 is probeAccessibilityViaEventTap(): the live-TCC tie-breaker that bypasses the stale per-process cache. The retry timer that fires every five seconds for three attempts is right above. PermissionsPage.swift in MainWindow renders the user-facing flow. All Swift, no obfuscation, you can clone the repo and trace one accessibility check from the timer firing to the AXError code being mapped.

On the apps where AX is empty, yes, but not in the way most blog posts frame it. Screenshot agents (Anthropic Computer Use, OpenAI Operator) read pixels and predict coordinates. They handle Tk and OpenGL apps because every other approach fails on those. They also burn tokens, lag on multi-monitor coordinate math, and break when the app's UI shifts between the screenshot and the click. On the apps where AX is rich, screenshot agents are slower and more expensive than AX agents for no benefit. The right architecture is hybrid: AX-first when the tree has roles you can dispatch on, vision when AX returns nothing useful. The disambiguation pattern is what makes hybrid switching possible without a heuristic that gets it wrong.

Every macos-use traversal writes a flat text file to /tmp/macos-use/<timestamp>_<tool>.txt. For a working native app the file starts with a header like '# Pages — 412 elements (0.21s)' followed by one line per element in the format [AXRole (subrole)] "title" x:N y:N w:W h:H. For a Tk app, the same file is a few lines: an AXApplication node, an AXWindow, and that is it, because Tk never registered children. The empty-tree signature is the trigger to fall back to vision. The full path is logged in the macos-use response so you can read what the model saw on any turn.

Conceptually yes, the function names change. Windows UI Automation is closer to a single coherent API, and most Win32, WinUI, and WPF apps participate including very old ones. The big Windows-specific quirk is the elevation asymmetry: an unelevated UIA client cannot read elements in elevated processes. Linux AT-SPI works only when the toolkit (GTK, Qt with the bridge plug-in) opts in, which makes it the most fragmented surface. Fazm is macOS only today (14.0+), so the code in this guide is the macOS path. The shape of the problem, three boundaries crossed at once and an ambiguous error code, is the same on all three OSes.

Cost and latency. A vision model pass on a 4K screenshot is hundreds of tokens minimum and adds a network round-trip you do not need on a native AppKit app where the AX tree gives you exact frames in points. Running the legacy paths with the screenshot fallback also means inheriting the screenshot model's failure modes (coordinate drift, modal interception, multi-monitor confusion) on apps that did not need any of that. AX-first with explicit fallback is faster, cheaper, and more deterministic on the workloads where it works, which on macOS is most of the apps a small business actually uses.