Building AI Agents Changed How I Think - Tools Matter More Than Prompts
Building AI Agents Changed How I Think
I spent months refining prompts before I realized I was optimizing the wrong thing. The prompts were not the problem. The tools were.
The Prompt Engineering Trap
When an agent fails a task, the instinct is to add more instructions to the prompt. "Be more careful when clicking buttons." "Always verify the page loaded before proceeding." "Check that the file exists before reading it." The prompt grows from 200 tokens to 2,000 tokens to 10,000 tokens.
The agent still fails - just in slightly different ways.
Why Tools Are the Leverage Point
A well-designed tool makes the agent's job easy. A poorly designed tool makes it impossible regardless of how good the prompt is.
Example: a "click" tool that takes x,y coordinates forces the agent to interpret screenshots, estimate pixel positions, and hope nothing moved. It fails constantly. No prompt fixes this.
Replace it with a "click" tool that takes an accessibility label - "click the Submit button" - and suddenly the agent succeeds reliably. The prompt did not change. The tool did.
What Good Agent Tools Look Like
Good tools are forgiving. They accept multiple valid inputs. A file reading tool that takes a line range instead of requiring the whole file. A search tool that does fuzzy matching instead of requiring exact strings. A navigation tool that waits for page load automatically instead of requiring the agent to add wait steps.
Good tools are informative. When they fail, they tell the agent why - "button not found, here are the buttons that do exist" is infinitely more useful than "click failed."
Good tools have reasonable defaults. If the agent does not specify a timeout, the tool waits a reasonable amount. If the agent does not specify a format, the tool returns something useful.
The Lesson
Stop spending hours on prompt engineering and spend that time on tool design instead. Build tools that make the right action easy and the wrong action hard. An average prompt with great tools will outperform a perfect prompt with mediocre tools every single time.
Fazm is an open source macOS AI agent. Open source on GitHub.