Chatting About Work vs Doing the Work

Most AI assistants are glorified chat interfaces. You ask a question, you get a response, and then you still have to go do the thing yourself. Upload a spreadsheet, ask for analysis, get a summary - but you're the one updating the cells.

That's a fundamentally different experience from an AI that controls your apps directly. One that clicks the buttons, fills the forms, navigates the menus, and completes the task end to end while you watch or do something else entirely.

The Accessibility Layer Makes It Possible

On macOS, the accessibility API exposes every element of every app - buttons, text fields, dropdowns, menus. An AI agent that uses this API can interact with any application the same way you would, except faster and without getting bored.

This isn't screen scraping or fragile pixel matching. The accessibility tree provides structured, semantic information about what's on screen. The agent knows it's clicking a "Send" button, not just clicking at coordinates (340, 520).

Why This Gap Exists

Building a chat interface is relatively straightforward. Building an agent that reliably controls arbitrary applications is hard. You need to handle loading states, unexpected dialogs, app-specific quirks, and the countless edge cases that come with real software.

That difficulty is exactly why the gap between "chat about it" and "do it" is so wide. Most products stay on the chat side because it's safer and easier to ship.

What Changes When AI Acts

When your AI agent can actually execute tasks, the interaction model flips. Instead of copying an AI's suggestion and pasting it into the right app, you just say what you want done. The agent handles the clicks, the typing, the navigation between apps.

That's not an incremental improvement on chat. It's a category change.

Fazm is an open source macOS AI agent. Open source on GitHub.