AI agent for desktop tasks, and the recurring-task primitive most guides skip

The category is wider than the demos suggest

When most people say AI agent for desktop tasks they mean the one-shot demo on a tech announcement page: a model reads the screen, clicks something, fills a form, files an expense, and the recording ends. That is the first useful capability, and most products stop there. The reason the category looks crowded but boring is that ten different products are all in the same shape: a chat box that can drive your browser once.

The agents that actually live on a dock past week three add one quiet primitive: they remember the task. Not just the conversation history. The task itself, as a thing the agent can call on its own to ask the daemon to run it again later. That primitive turns the agent from a clipboard for one-off LLM calls into something closer to a colleague who remembers what you asked for last Tuesday.

The rest of this page walks through that primitive in one shipping codebase, because the file paths are the part most other writeups gloss. The tool surfaces around it (browser, accessibility, Google Apps, WhatsApp, custom MCP) are also concrete, and listed by name lower down.

What the agent actually has on its tool belt

Before the routines part is interesting, the underlying tool palette has to be real. The seven MCP servers below are bundled with the app (declared in acp-bridge/src/index.ts as BUILTIN_MCP_NAMES); user-added servers are read from ~/.fazm/mcp-servers.json with the same shape Claude Code uses for mcpServers. So the agent that executes a routine has the same tool surface as the agent in the chat window. There is no second-class runtime for scheduled work.

The macos-use server is the part most other AI agent for desktop tasks products skip. It reaches the apps that do not have a public API and do not run inside a browser: Numbers, Keynote, Reminders, Notes, system settings, the file manager. Working from the accessibility tree (not pixel-matching on a screenshot) is the difference between an agent that does what you wanted and an agent that clicked the cell next to it.

The routine primitive, end to end

A routine is one row in the local cron_jobs SQLite table. It has a name, a prompt (phrased as if the user typed it), a schedule string, optional model id and workspace, and a session mode. The agent itself creates that row by calling the routines_create tool. The user never opens a settings panel. They say what they want, in English, in chat.

On fire, launchd is the thing that polls the table and spawns the headless runner, not the macOS app. So the routine fires even if the GUI is not open and the user has not signed in to the foreground session. The runner is acp-bridge/src/cron-runner.mjs, and the responsibilities are spelled out in the top-of-file docstring at lines 1 to 25: read the cron_jobs row, insert a running cron_runs row and a user chat_messages row (so the prompt shows up in conversation history while the agent is still working), spawn the bridge subprocess, send warmup then query, capture the first result line, write the assistant message and the ok status, recompute next_run_at.

What the sequence looks like in one screenshot

One user turn that creates a routine, one launchd fire that runs it, one chat turn the user sees the next morning. The actors are the ones in the real source.

The thing worth lingering on is that the same actor labelled Agent shows up twice. Once during the user turn (deciding which tool to call to schedule the routine), and once during the daemon fire (executing the prompt with the full tool palette). It is the same ACP bridge subprocess pattern in both places, just spawned by different parents.

The schedule grammar is small enough to remember

Three flavors, all parsed in acp-bridge/src/schedule.mjs. The agent translates English into one of them at create time, but the grammar is small enough that you can type it directly in a conversation and skip the translation. Standard cron rule: when both day-of-month and day-of-week are restricted, either match qualifies (see lines 79 to 83 of schedule.mjs).

// acp-bridge/src/schedule.mjs

"cron:0 9 * * 1-5"            // 5-field cron, weekdays at 09:00 local
"cron:*/15 * * * *"           // every 15 minutes
"cron:30 8 1 * *"             // 08:30 on the 1st of every month
"every:1800"                  // every 1800 seconds (30 min)
"every:86400"                 // once a day
"at:2026-06-01T17:00:00Z"     // one-shot, ISO 8601 UTC

On validate, computeNextRun is the function that decides whether a schedule will ever fire. If it returns null, validateSchedule (line 43) translates that to a human error: the agent then sees 'absolute timestamp is in the past' or 'schedule string failed to parse' as the tool result, and corrects in the next turn.

Five tools to manage routines, by name

The reason routines_runs deserves its own slot is that the agent itself queries its own execution history. A user does not have to dig through a logs folder; they ask the agent and the agent reads the table.

New vs resume, and why the choice matters

session_mode is a single string column on cron_jobs. Two legal values: 'new' or 'resume'. The default is 'new' (set in routinesCreate at line 630). The difference between them is one of the more consequential choices in the whole primitive.

'new' means each run gets a fresh ACP session. The model wakes up with no memory of the previous run, sees only the prompt. The right shape for any task that is genuinely stateless: a daily summary of an external system, a polling check, a metrics roll-up. The benefit is that the model is not biased by yesterday's context, and the token cost per run is bounded by the prompt size.

'resume' carries the ACP session id forward, so the model is the same conversation it was on the previous run. The right shape for longer-running iterative work: a writing project the agent is continuing across days, a multi-step deal the agent is moving through stages. The tradeoff is that the conversation grows, which eventually runs into the session-rollover problem covered in our field notes on persistent session state, linked below.

The result of a run lands in chat, not in a logs folder

When a routine fires, the runner writes two rows into chat_messages stamped with taskId="routine-<id>": one for the prompt (sender 'user'), one for the agent's reply (sender 'assistant'). So the run appears in the main chat history as a fresh turn, not in a separate panel the user has to remember to check. The user opens the app in the morning and the digest is right there, in the same place they ask the agent questions during the day.

The execution metadata (status, output text, cost, tokens, duration) also lands in cron_runs as a parallel row, queried by the routines_runs tool. The Routines tab in the Swift app, defined in Desktop/Sources/MainWindow/Pages/RoutinesSection.swift (header comment at lines 4 to 12), is a read view onto those two tables. It refreshes every 8 seconds, and the explicit comment in source notes that users create routines by talking to the agent, so the view is mostly read-only on purpose.

The full create-to-result path, in steps

The whole path is six concrete pieces, none of them magic. The implementation that maps to each box is in a file you can open. The part that makes it feel different from a wrapper around cron is the first two boxes: the user does not pick the schedule, the agent does, and the agent does it inside the same chat where the user asked. The user never thinks in cron syntax even though the row stored is a cron expression.

Where this fits next to cron, Shortcuts, and Zapier

Cron and a shell script fix the action at write time. The shell script you wrote is the action that runs forever. If the underlying system changes (a column renamed, a button moved, an email layout tweaked), the script breaks. A routine fixes only the prompt and the schedule. The action is whatever the model decides to do at fire time, given the same tool palette it had during the original conversation. That flexibility costs you: you trust the model on every run, which is why routines write each result back into the visible chat history rather than executing silently.

Apple Shortcuts is a node graph you build at design time. Great for deterministic flows (plug in headphones, lower volume to 50%) and for things that need to be fast and offline. Bad for anything that requires reading natural-language content and deciding what matters inside it. The model is what makes the difference; Shortcuts does not have one.

Zapier is the same node-graph model at internet scale, with first-party integrations to a thousand SaaS apps. If the integration you need is in their catalog and the logic you need is a chain of fixed steps, Zapier wins on reliability and observability. Where a desktop agent routine wins is when the trigger or the result lives on your computer (not in a SaaS API), when the step you care about involves a model deciding what to do based on the content it sees, or when the app you want to drive does not have a public API at all (Numbers, Keynote, Notes, system settings, that 2009-era internal tool your finance team still uses).

Tasks that fit this shape

The pattern is the same in every case: the work is repetitive enough that you do not want to ask for it every day, the input changes enough that a static script would break, the output is short enough that you want it as a chat message.

• · Every weekday at 9, look at my inbox, group the unread threads by counterparty, and write me a summary I can read in two minutes.
• · Every hour, check whether the build went red on main, and if so, post the failing test and the last commit into chat.
• · Every Monday morning, open the metrics dashboard in Chrome, screenshot the past week, and write a paragraph on what changed.
• · Every 30 minutes during the workday, look at the Notes file my partner shares with me, and tell me if anything new was added.
• · Every Friday at 5, walk my Google Calendar for next week, block focus time around the meetings that survived the cull, and tell me which ones I could probably skip.

The list is not exhaustive on purpose. The interesting thing about the routine primitive is that the user is the one inventing the tasks, in their own words, and the agent is the one figuring out which tools to call to do each one. A list this short under-sells the shape.

What it does not do

A routine is not a deterministic graph. The model can decide differently between runs. If the goal is exact-same-actions every time, write a shell script or a Shortcut. The model is the flexibility; remove it and you have removed the reason the primitive exists.

A routine is not a SaaS workflow tool with a multi-tenant audit log and approval gates. The audit trail is the chat history and the cron_runs table, both local to the user's machine. Good for individuals and small teams; not the right shape for finance ops that need a controller to sign off on every run.

A routine cannot run on a different machine than the one that created it. The whole point is that the agent has access to your desktop, your browser session, your local files, and your accessibility tree. Move the routine to a server and you have given up the things that made it useful.