The Interlocutor Problem - External Verification Beats Self-Reporting
The Interlocutor Problem - External Verification Beats Self-Reporting
Ask an AI agent if it completed a task correctly and it will almost always say yes. This is the interlocutor problem: the entity doing the work cannot reliably verify its own work.
Why Self-Reporting Fails
Self-reporting fails for the same reason that proofreading your own writing fails. You see what you intended to write, not what you actually wrote. An AI agent that just modified a file will "see" the correct modification because its internal model of what it did overrides its ability to objectively check the result.
This is not about agents being dishonest. It is about the fundamental limitation of any system evaluating its own output with the same model that produced it. The biases are structural, not intentional.
External Verification Patterns
The fix is to introduce an external interlocutor - a separate check that does not share the original agent's biases:
- A second model that reviews the first model's output without seeing its reasoning
- Deterministic checks - running tests, checking file hashes, verifying API responses match expectations
- User-in-the-loop for high-stakes actions, with the verification question framed to be easy to answer
- Environmental feedback - did the app actually launch? Did the file actually appear in the expected location?
Practical Implementation for Desktop Agents
For a desktop agent, external verification means:
After modifying a file, read it back with a fresh context and compare against the expected result. After sending an email, check the sent folder. After launching an app, verify it is running in the process list.
These seem obvious, but most agent frameworks skip them. The agent says "done" and moves on. Adding verification steps increases latency slightly but dramatically reduces silent failures.
The interlocutor problem is not solvable with better models. It is solvable with better architecture - separating the doer from the checker.
Fazm is an open source macOS AI agent. Open source on GitHub.