The Capability Trap: Smarter Output, Same Decision
Every Rollout Measures What the Tool Produces. Almost None Measure Whether a Call Went Differently
If we can't name a decision that went differently, we bought speed, not judgment.
The Nineteen Percent
In July 2025, METR ran a randomized controlled trial that has aged better than most things written about AI productivity. Sixteen experienced developers worked 246 real tasks on repositories they already maintained, randomized so some tasks allowed AI tools and some did not. The result: with AI allowed, they took 19% longer, 95% confidence interval running from 2% to 39% slower.
The second number matters more. Going in, those developers expected AI to make them 24% faster. Coming out, having lived the tasks, they still believed it had made them 20% faster. They were off by roughly forty percentage points about their own performance, in the direction of flattery, on work they knew intimately.
METR revisited the design in February 2026 and got 18% faster for returning developers and 4% for new recruits, with intervals wide enough to contain zero. They called it an unreliable signal rather than publishing the friendlier figure. After two serious attempts, we still can't say cleanly whether the tools helped. What we can say is that the people using them could not tell.
Why the Dashboard Confirms Itself
Notice what METR needed to get an answer: randomization, a control condition, screen recordings, real tasks. Almost no enterprise rollout has any of that. What a rollout has is a dashboard, and every number on it measures the tool.
Seats provisioned. Prompts per week. Adoption by department. Time to first draft. All of them rise when the tool works and rise when the tool is confidently wrong, because none of them observes the decision at the other end. A model that fabricates a citation produces that citation quickly, and the dashboard records a fast draft.
These metrics aren't lying. They're answering a different question from the one leadership thinks it asked, and answering it correctly.
The asymmetry is structural rather than careless. Capability shows up in artifacts, and artifacts are countable. Judgment shows up in the decisions that didn't go wrong, and those leave no trace at all. A compliance lead who spots a fabricated framework clause and quietly fixes it produces exactly the same telemetry as one who doesn't: a completed control narrative, on time. The organization sees two identical rows.
Three Questions That Separate Them
There's a cheap test for whether a deployment moved judgment or only output. It needs no control group, just someone willing to answer three questions honestly about the last quarter.
Did a wrong call get caught earlier than it would have been? Not "did we review more." Did something false get stopped, and can we point at it. If nothing was caught because nothing was wrong, that's a claim about the model's error rate, and it's almost certainly untrue.
Did anyone change their mind on evidence? A collaboration that never revises anyone's position is one party dictating and the other transcribing. Which party is dictating doesn't change the diagnosis.
Did disagreement surface before commitment or after? Disagreement arriving after a filing or a client call isn't disagreement. It's a post-mortem.
| What we measure | What it proves | What it cannot tell us |
|---|---|---|
| Seats, prompts, adoption rate | The tool is in use | Whether anyone reads output critically |
| Time to first draft | The tool produces faster | Whether the draft was right |
| Training completion | The policy was distributed | Whether anyone applies it under deadline |
| Self-reported confidence | People feel capable | See the forty-point gap above |
What the Other Half Needs
If judgment leaves no trace in ordinary work, the only way to see it is to create the conditions deliberately. Put a person in front of confident, plausible, wrong output under real time pressure and watch. Do they catch the error, verify the claim that carries the decision, escalate when a source can't be confirmed, and correct rather than defend?
That's what PAICE (People + AI Collaboration Effectiveness) measures. The assessment seeds errors of the kind that reach filings, control narratives, and client reports, then scores behavior when the model is wrong. Someone fluent who misses the seeded error scores lower than a terse person who catches everything. Conversation is the medium; it isn't the measurement.
I want to be careful about what that buys. It gives a cohort-level behavioral reliability profile, explainable and re-measurable. It does not give a causal estimate of whether a deployment improved decisions, which is what METR spent two studies failing to pin down for one well-defined task class. Anyone selling you that number, us included, should be asked how they got it.
Privacy is architectural rather than promised: hashed identifiers, no retained conversation text, individual results structurally unavailable to the organization. The distribution is visible. A name never is.
Next quarter's rollout will produce a number. Ask what decision it changed.
Ready to see whether your team's judgment moved, not just its output? Learn about PAICE for organizations or take an individual assessment to see it firsthand.
Recommended Reading
- Collaboration Beats Capability - The same argument at the organizational level
- The AI Adoption Illusion - What adoption rates hide
- AI Governance Metrics Beyond Adoption and Training - Why the standard catalog measures systems and not operators
- The Behavioral Measurement of AI Collaboration - How behavioral evidence is gathered
Curious but short on time?
Take the 3-minute PAICE Pulse — a quick confidence check that maps how you see your own AI collaboration posture. No login required.