AI Coding Productivity Data Is Not What You Expected
AI Coding Productivity Data Is Not What You Expected
Everyone using AI coding tools feels faster. The code appears on screen in seconds instead of minutes. Autocomplete finishes your thoughts. The vibes are good. But METR's research found something uncomfortable: developers significantly overestimate how much time AI actually saves them.
The METR Finding
METR ran controlled studies measuring actual time-to-completion on coding tasks with and without AI assistance. Developers predicted AI would save them around 24% of their time. The measured result was closer to 0% - and in some cases, AI-assisted work took longer.
How is that possible when the code literally writes itself?
Where the Time Goes
The time savings from code generation are real but small. What eats those savings:
Verification. You still need to read and understand the generated code. This takes longer than reading code you wrote yourself because you did not build the mental model while writing it.
Correction cycles. The agent generates code that almost works. You prompt it to fix the issue. It fixes that but introduces a new one. Three rounds later, you have spent more time than writing it from scratch would have taken.
Context switching. Describing what you want to an AI, evaluating its output, and redirecting it uses a different cognitive mode than just writing code. The switching cost is real and unmeasured.
What This Means
AI coding tools are still useful - but not in the way most people think. The value is not in raw speed. It is in reducing the activation energy to start tasks, exploring unfamiliar APIs, and generating boilerplate. If you treat AI as a speed multiplier for all coding, you will be disappointed when you measure the actual results.
The developers who benefit most are the ones who know exactly when AI helps and when it does not - and they switch tools accordingly.
Fazm is an open source macOS AI agent. Open source on GitHub.