AI feels faster. A careful study says it made experts slower. I went looking in my own week.
There's a study I can't stop thinking about, because it accuses me of something and I'm not sure it's wrong. In 2025, METR ran a proper randomised trial: experienced open-source developers, working on their own large codebases, with AI tools allowed on some tasks and forbidden on others. The developers felt the AI made them about 20% faster. The stopwatch said they were 19% slower. I use Claude Code every day and I would have bet the same way they did. So I spent a week paying honest attention to my own work, and I don't love what I noticed.
What the study actually measured
Let's be precise, because this result gets flattened into "AI is useless" and it says nothing of the kind. METR took 16 seasoned developers and 246 real issues on projects averaging over a million lines of code that they'd maintained for years. Each task was randomly assigned AI-allowed or not. Tooling was Cursor Pro with the frontier Claude models of the time. The finding: allowing AI increased completion time by 19%.
The forecasting gap is the part that should make everyone quiet. Beforehand, the developers predicted a 24% speedup. Afterward — having lived the slowdown — they still believed AI had sped them up by about 20%. Economics experts had predicted 39% faster. ML experts, 38%. Everyone who guessed, guessed the wrong direction. The feeling of speed was near-universal and near-universally wrong.
Read the fine print before you weaponise it
METR was scrupulous about what this does not show, and I want to honour that. It's not "AI never helps." The conditions were specifically the ones where AI's edge is smallest: genuine experts, in codebases they know cold, with high quality bars, where a plausible-but-slightly-wrong suggestion costs more to correct than to have never seen. That's the worst case for a coding assistant. Greenfield work, unfamiliar languages, boilerplate, one-off scripts, a junior climbing a new codebase — none of that was tested, and that's where the wins plausibly live.
And there's a genuine update: in February 2026 METR said it now believes developers are probably more sped up in 2026 than they were in 2025, as agentic tools matured — while being admirably blunt that "our data is only very weak evidence for the size of this increase." Even better, the reason they had to redesign the experiment is telling: developers refused to participate if it meant sometimes working without AI, and 30–50% quietly declined to submit exactly the tasks they most wanted AI for. The clean trial fell apart because people valued the tool too much to give it up. That's not nothing.
The most replicated finding in all of this isn't "AI is slow." It's that how fast AI feels is a bad estimator of how fast it is — and the gap runs in the flattering direction.
What I actually caught myself doing
I didn't run a lab. This is n=1 and unscientific and I won't pretend otherwise — no fabricated stopwatch numbers here. But watching honestly, the pattern rang true in a way that stung. On the parts of this blog I know intimately — the theme, my custom modules — I'd hand a small change to the agent, get something 90% right, and then spend real minutes reading it, spotting the subtle wrong turn, and steering it back to how I'd have just written it. It felt fast the whole time, because I was always busy, always watching output stream. But "busy" and "fast" aren't the same thing, and the study named the difference.
Where it plainly won: the stuff I didn't want to hold in my head. A gnarly bit of Twig I'd have had to look up, a batch of research I'd never have done by hand, boilerplate for a module I'd have resented typing. On unfamiliar ground, the agent is a genuine multiplier. On my home turf, it's often a very confident pair-programmer I have to double-check — and the checking is the tax nobody feels while they're paying it.
The honest takeaway
The corroborating data doesn't say stop — GitClear's analysis of hundreds of millions of changed lines shows the second-order cost instead: more code shipped, but duplication creeping up and refactoring dropping off. Faster to write is not the same as faster to ship and maintain. So I'm not quitting Claude Code; it earns its keep on exactly the work the METR conditions excluded. What I've changed is smaller and, I think, more useful: I've stopped trusting the feeling. When a task is squarely in territory I know cold, I now ask whether I'm actually saving time or just enjoying the sensation of momentum. Often the honest answer is that I'd have been done already. The tool is real. The feeling of speed is the thing to distrust.