The Sandbox Paradox: AI Agents Need Access to Be Useful
The Sandbox Paradox: AI Agents Need Access to Be Useful
The permission model for desktop AI agents is fundamentally broken. Not because it is too loose or too strict - but because both extremes fail equally.
Lock the agent in a sandbox with no file access, no browser control, no app interaction, and it is a chatbot. Useful for answering questions, useless for doing work. Give it unrestricted access to your entire system and you have a security nightmare - one prompt injection away from disaster.
This is the sandbox paradox. The agent needs access to be useful and restrictions to be safe. These requirements directly oppose each other.
Why Traditional Sandboxes Don't Work
Browser sandboxes work because browsers have a clear trust boundary: the web page is untrusted, the system is trusted. Desktop agents blur this line. The agent needs to read your files (trusted context), interact with web pages (untrusted content), and execute system commands (highest privilege) - all in the same workflow.
Mobile app permission models - "allow camera access yes/no" - are too coarse. An agent needs file access, but not to your SSH keys. It needs browser control, but not to your banking site. It needs to run commands, but not rm -rf.
A Better Model: Semantic Permissions
Instead of blanket allow/deny, effective agent permissions should be semantic. Not "can access files" but "can access files in ~/Documents/work/ but not ~/Documents/personal/." Not "can control browser" but "can control browser tabs except those on financial sites."
This requires the agent to understand what it is doing, not just what it is allowed to do. A semantic permission system says "you can automate Slack" rather than "you have accessibility API access." The system maps high-level permissions to low-level capabilities.
The Approval Spectrum
In practice, agents need a graduated approach. Auto-approve low-risk actions like reading files and checking app states. Prompt for medium-risk actions like sending messages or modifying documents. Block high-risk actions like deleting files, running shell commands with sudo, or accessing credentials.
The key insight is that the approval level should match the blast radius of the action, not its technical complexity. Sending a Slack message to your team (medium blast radius) should require more approval than reading a 10GB log file (zero blast radius).
Fazm is an open source macOS AI agent. Open source on GitHub.