47 Translation Errors as a Learning Dataset for AI Agents
47 Translation Errors as a Learning Dataset for AI Agents
A trip planning agent produced 47 translation errors in one session. Mismatched character encodings, wrong language detection, and element-not-found failures on localized websites. The instinct is to fix these bugs and move on. The better approach is to treat them as a dataset.
Errors as Training Data
Every agent failure contains information about the gap between what the agent expected and what actually happened. A translation error means the agent assumed a page was in English when it was in Japanese. An element-not-found failure means the agent's selector strategy does not account for localized UI labels.
47 of these failures, documented with the full context - page URL, expected element, actual page state, error message - become a test suite. Not a unit test suite that checks synthetic cases, but a real-world regression suite that validates against actual failure modes.
The Element-Not-Found Pattern
Most element-not-found errors in agent automation stem from the same root cause: the agent identifies elements by visible text, and that text changes across languages. A "Submit" button becomes different text in every locale.
The fix is not better translation. The fix is identifying elements by structural position, accessibility roles, or stable attributes rather than displayed text. The 47 failures tell you exactly which selectors need to be locale-independent.
Building the Feedback Loop
Capture every agent failure with enough context to reproduce it. Store the input, the expected behavior, the actual behavior, and the error. Run new agent versions against this dataset before deploying.
This is not novel - it is standard test engineering. But agent builders often skip it because failures feel like bugs to squash rather than data to collect. The difference between an agent that plateaus and one that improves over months is whether you systematize failure collection.
Your error log is your most valuable asset. Treat it that way.
Fazm is an open source macOS AI agent. Open source on GitHub.