5 Tiny SwiftUI Utilities for AI Agent Accessibility
5 Tiny SwiftUI Utilities for AI Agent Accessibility
AI agents that control your Mac rely on the accessibility tree to understand what is on screen. If your SwiftUI views lack proper accessibility labels, agents cannot interact with them. Here are five small utilities that fix this.
1. Labeled Icon Buttons
Custom icon buttons often have no accessibility label. Wrap them with a modifier that enforces one:
Button(action: doSomething) {
Image(systemName: "gear")
}
.accessibilityLabel("Settings")
Without this, an AI agent sees a generic button with no description and has to guess what it does.
2. Toggle State Announcements
Toggles need to announce their current state, not just their label. Add .accessibilityValue(isOn ? "Enabled" : "Disabled") so agents can read the state without taking a screenshot.
3. Custom Slider Descriptions
Sliders with custom ranges need descriptive labels. A volume slider should say "Volume: 75%" not just "Slider." Use .accessibilityValue("\(Int(value))%") alongside a clear label.
4. Grouped Container Labels
When multiple elements form a logical group - like a card with an image, title, and subtitle - wrap them in an accessibility container with .accessibilityElement(children: .combine). This gives agents a single coherent element instead of three disconnected ones.
5. Action Descriptions for Gestures
Custom gestures are invisible to the accessibility tree. If your view responds to a long press or swipe, add .accessibilityAction(named: "Show Options") so agents know the action exists.
Why This Matters for AI Agents
AI desktop agents like Fazm read the accessibility tree to navigate apps. Every unlabeled button is a blind spot. Every missing state description forces the agent to fall back to slower, less reliable screenshot analysis.
Building accessible apps is not just about human users with disabilities - it is about making your software compatible with the next generation of AI-powered automation.
Fazm is an open source macOS AI agent. Open source on GitHub.