Solving the Hallucination vs Documentation Gap for Local AI Agents
Solving the Hallucination vs Documentation Gap for Local AI Agents
AI agents hallucinate most when they lack context. The model does not know what version of a tool you are running, what flags it supports, or how your specific setup differs from the training data. The fix is not a better model - it is better context gathering.
The Documentation Gap
Most hallucinations in local agents come from a gap between what the model learned during training and what actually exists on your system. The model "knows" that ffmpeg has certain flags, but your version might be different. It "knows" how brew works, but your homebrew prefix might be non-standard.
This gap is invisible to the model. It generates commands with confidence because the syntax looks correct based on training data. It just does not match your reality.
CLI Introspection as the Fix
The solution is to make the agent check before it assumes. Before running an ffmpeg command, run ffmpeg -help and parse the output. Before using a CLI tool, check --version and --help to confirm what flags actually exist.
This is CLI introspection - using the tools themselves as documentation. It is more reliable than any knowledge base because it reflects the exact state of your system right now.
Skills That Check Docs First
The pattern that works best is building agent skills with a mandatory documentation check step. The skill definition says "before running this command, first check the tool's help output and confirm the flags you plan to use exist."
This adds one extra tool call per command. But it eliminates an entire class of hallucinations. The trade-off is worth it - one extra second of latency versus five minutes of debugging a failed command.
You can also build local documentation indexes from man pages and help outputs that the agent references before generating commands. Keep them updated with a weekly cron job and the agent always has current information.
Fazm is an open source macOS AI agent. Open source on GitHub.