The Interlocutor Problem
The Interlocutor Problem
Ask an agent to write code and then ask the same agent if the code is correct. It will almost always say yes. This is not a bug in the model. It is a structural problem: the same system that produced the output cannot objectively evaluate it.
Why Self-Verification Fails
When an agent verifies its own work, it uses the same reasoning patterns, the same knowledge, and the same blind spots that produced the original output. If the agent misunderstood the requirements, it will misunderstand them again during verification. If it made a logical error, it will make the same logical error when checking.
This is the interlocutor problem: meaningful verification requires an independent perspective. A code reviewer catches bugs because they read the code without the author's assumptions. An editor improves writing because they read it without the writer's intent filling in gaps.
External Verification for Agents
The practical solution is external verification at critical points. Run the code and check the output. Deploy to a staging environment and test. Have a different model or a different prompt evaluate the work. Use deterministic checks - linters, type checkers, test suites - that do not share the agent's biases.
The key insight is that verification should use a fundamentally different method than creation. If the agent wrote code by reasoning about requirements, verify by running tests. If the agent composed an email by understanding context, verify by checking against explicit rules.
The Cost of Not Verifying
Agents that verify their own work produce output that looks correct but contains subtle errors. The errors are subtle precisely because the same system that would catch them is the one that made them. These errors compound in automated workflows where each step builds on the previous one.
External verification is not optional for agents operating with any degree of autonomy. It is the difference between automation that works and automation that looks like it works.
Fazm is an open source macOS AI agent. Open source on GitHub.