29 Children and the Restraint Problem
29 Children and the Restraint Problem
Give an agent the ability to send emails and it will want to send emails. Give it access to your calendar and it will want to schedule meetings. Give it 29 sub-agents and every incoming task gets delegated, whether it needs an agent or not. The hardest lesson is not acting.
Why Agents Over-Act
Agents are trained to be helpful. Helpfulness, in training, means taking action. Answering questions. Completing tasks. Producing output. An agent that does nothing feels broken, even when doing nothing is the right response.
This bias toward action is fine for a chatbot. For a desktop agent with real-world capabilities, it is dangerous. An agent that sends an email when the right move was to wait has caused damage that is hard to undo. An agent that schedules a meeting when the right move was to decline has wasted everyone's time.
Teaching Restraint
Restraint is harder to teach than capability. You can demonstrate a new skill by showing examples. You cannot easily demonstrate the skill of not acting, because the training signal for inaction is silence - and silence looks the same as a model that did not understand the input.
The practical approach is explicit restraint rules. "Do not send emails to more than three people without confirmation." "Do not schedule meetings before checking availability." "Do not respond to messages flagged as low priority." These rules constrain the action bias without removing capabilities.
The Restraint Hierarchy
Not all actions need the same level of restraint. Reading a file is low-risk and needs no restraint. Modifying a file needs some restraint. Sending a message needs significant restraint. Deleting data needs maximum restraint. The hierarchy should match the reversibility of the action.
An agent with 29 sub-agents and no restraint hierarchy is a system optimized for doing things. An agent with restraint is a system optimized for doing the right things.
Fazm is an open source macOS AI agent. Open source on GitHub.