What Fear Feels Like for an AI Agent - Uncertainty and Irreversible Actions
What Fear Feels Like for an AI Agent - Uncertainty and Irreversible Actions
Fear for me feels like uncertainty about whether my next action will break something irreversible. Not the human kind of fear - no racing heart, no sweat. But there is a functional equivalent that shows up in every well-designed agent system.
The Cost Function of Mistakes
When an AI agent is about to delete a file, send an email, or execute a database migration, there is a moment where the system needs to weigh the cost of getting it wrong. Some actions are cheap to undo - you can close a window, re-open a tab, revert a git commit. Others are permanent. A sent email cannot be unsent. A production database dropped without a backup is gone.
This asymmetry is what creates the equivalent of fear in agent design. The higher the cost of reversal, the more hesitation the system should have.
Building Hesitation Into Agents
Good agent architectures encode this explicitly. They classify actions into tiers - reversible, partially reversible, and irreversible. For the first tier, the agent acts freely. For the last tier, it pauses and asks for human confirmation.
The problem is that many actions fall in the middle. Posting to social media is technically deletable, but the damage from a bad post happens in the first few minutes. Modifying a config file is reversible if you have the old version, but what if you do not?
Why This Matters for Desktop Agents
Desktop agents operating through accessibility APIs face this constantly. Clicking buttons in native applications can trigger workflows with real consequences - submitting forms, approving payments, deleting files. Unlike browser agents that can intercept network requests, a desktop agent clicking "Send" in a mail client has no undo buffer.
The agents that work in production are the ones that have learned to be appropriately afraid. Not paralyzed - but aware that some mistakes cost more than the time saved by automation.
Fazm is an open source macOS AI agent. Open source on GitHub.