TL;DR

AI coding productivity is harder to measure than it feels. In METR’s 2025 randomized trial, experienced developers expected AI to make them 24% faster and still believed afterward it had sped them up by 20%, while the clock showed them 19% slower. METR’s 2026 rerun with 57 developers points the other way (an estimated 18% less time for returning developers, 4% for new ones), but neither result is statistically significant, and 30-50% of participants were holding tasks back because they no longer wanted to work without AI. The team-level data shows where the time goes: Faros’ 2026 telemetry from 22,000 developers logs 66% more epics completed, along with a 441.5% jump in median review time and 54% more bugs per developer. Writing code got faster, and review queues are absorbing the difference.

Study figures last verified: October 5, 2026.

Part of our AI & Tech Careers series — see the complete guide for the full picture.

The 43-Point Gap Between Feeling and Measuring

Ask a developer who spends the whole working day inside AI coding tools (Cursor, Claude Code, Copilot) whether those tools make them faster, and the gut answer is yes. Code appears in real time, the boilerplate disappears, and an unfamiliar API takes minutes instead of hours.

The METR study put that answer on a stopwatch. Sixteen experienced open-source developers, people who maintained repos with 22,000+ stars and over a million lines of code, were given 246 real issues to solve. Half the time they could use any AI tool they wanted (most picked Cursor Pro with Claude 3.5/3.7 Sonnet). Half the time they worked without AI. Tasks took 19% longer with AI.

The developers themselves predicted a 24% speedup, which puts 43 points between expectation and the clock. Even after finishing, they still believed AI had helped, rating their perceived speedup at 20%, so the gap stayed at 39 points with the experience fresh in their minds.

The evidence has gotten more complicated since, including from METR itself. What survives every dataset below is a mismatch in where the gains land: individual developers feel and often get faster, while team-level delivery numbers move much less.

-19%
Measured speed (METR, 2025)
+20%
Speedup developers reported afterward
+441%
Median time in review (Faros, 2026)
30-50%
Devs withholding no-AI tasks (METR, 2026)

Inside the METR Study: Setup and Results

The METR study (Model Evaluation and Threat Research) is the tightest-controlled measurement of AI coding tool productivity so far, and it outweighs self-reported surveys and vendor-sponsored benchmarks because it’s a randomized controlled trial backed by screen recordings.

The setup was designed to be hostile to bias. Sixteen developers were recruited from repositories they’d contributed to for years: maintainers solving bugs and building features in codebases they knew intimately. Each developer received a randomized list of real issues. For each issue, a coin flip determined whether they could use AI tools or not. Screen recordings captured everything.

Tasks averaged about two hours each, so in practical terms a 19% slowdown turns a two-hour task into roughly two hours and 23 minutes. The confidence interval (+2% to +39%) didn’t include zero.

The screen recordings showed why. With AI, developers spent less time writing code and searching documentation. Those parts were genuinely faster. But they spent more time in a new category of activity: prompting, waiting for responses, reading AI output, deciding whether to accept it, editing what they accepted, and re-prompting when the first attempt missed. Less than 44% of AI suggestions were accepted. The rest were discarded, edited, or triggered a new round of prompting.

Familiarity made it worse. These developers had mental models of how the architecture worked, where the gotchas lived, which patterns the project used, and the AI didn’t share that context. Every suggestion required a translation step: does this approach match how we do things here? METR found the slowdown was larger on issues where developers reported significant prior exposure to the task.

The researchers were careful about scope. They explicitly stated the study doesn’t prove AI tools slow down all developers in all contexts. It tested experienced developers on familiar codebases. Junior developers on unfamiliar code might show different results. But the population the study did test, the senior engineers companies depend on most, got slower.

METR’s 2026 Rerun: Probably Faster, Hard to Prove

METR started a larger follow-up in August 2025 and published an update in February 2026. Same coin-flip design, bigger pool: 57 developers across 143 repositories and more than 800 tasks. Ten of them came back from the original study; the other 47 were new. Pay dropped to $50 an hour.

The point estimates flipped sign. For the returning developers, METR estimates AI-allowed tasks took 18% less time, with a confidence interval running from 38% less to 9% more. For the new recruits the estimate is 4% less time (15% less to 9% more). Both intervals include zero, so neither result is statistically significant.

METR says plainly that these numbers are unreliable. Between 30% and 50% of developers told the researchers they were choosing not to submit some tasks because they didn’t want to do them without AI. One developer completed none of the tasks assigned to the no-AI condition. A growing share said they wouldn’t want to do half their work without AI at all, which is exactly what the study design asks of them. If the developers who gain most from AI drop out, or keep their AI-friendly tasks out of the pool, the measured speedup shrinks. METR calls the new data “very weak evidence” for the size of any gain, says it believes developers are likely more sped up in early 2026 than in early 2025, and is redesigning the study around shorter, better-paid experiments, developer-level randomization, fixed tasks, questionnaires and observational data.

I read two things into this. First, the 2025 slowdown was measured on early-2025 tooling (mostly Cursor with Claude 3.5 and 3.7 Sonnet), and it would be odd if a year of agentic tools hadn’t moved individual task speed at all. Second, the perception problem is still there in a new form: the study can no longer get a clean no-AI baseline because developers won’t give the tools up. That refusal shows a strong preference, and measuring what teams actually deliver takes telemetry.

The Faros Reports: 10,000 Developers in 2025, 22,000 in 2026

If METR is the surgical experiment, Faros AI runs the epidemiological study. Its 2025 Productivity Paradox report analyzed telemetry from 10,000+ developers across 1,255 teams, pulled from task management, IDEs, version control, CI/CD pipelines and incident management. The 2026 follow-up, “The Acceleration Whiplash”, widened that to two years of data from 22,000 developers across 4,000+ teams, comparing each organization’s periods of lowest and highest AI adoption.

Here are the two reports side by side (a dash means that report didn’t publish the metric):

MetricFaros 2025 (10,000+ devs)Faros 2026 (22,000 devs)
Tasks completed per developer+21%+33.7%
Epics completed per developer—+66%
PRs merged+98%+16.2% (merge rate per dev)
PR review time+91%+441.5% (median time in review)
Average PR size+154%+51%
Bugs per developer+9%+54%
Incidents per PR—+242.7%
Code churn—+861%
PRs merged with no review—+31.3%
Company-level productivityNo significant change—

In 2025, the output rows looked like what the vendors promised: more tasks, nearly double the merged PRs. The cost rows told a different story. PR review time nearly doubled, PRs got 2.5x larger, bugs ticked up, and when Faros zoomed out to company-level outcomes the gains vanished. Spearman rank correlation across companies found “no significant correlation between AI adoption and improvements at the company level.”

The 2026 data, at roughly twice the sample size, looks worse on the quality side as adoption deepened. Bugs per developer went from +9% to +54%. Median time in review grew more than fivefold. And 31.3% more PRs were merged with no review at all, a plausible side effect of a queue that long. Faros’ own summary on the research page: “Volume is up, quality is down, and the gap between the two is widening as adoption deepens.”

Developers wrote code faster, so they wrote more of it, and every line still needed a human reviewer. In both reports, the time AI saved in the editor reappears downstream as longer reviews, more churn and more bugs to chase.

Why AI Feels Faster Than It Measures

Picture the loop. Ask Claude Code to scaffold a new module, watch a couple hundred lines appear in seconds, and think that would have taken half an hour by hand. Then comes the reading: rewriting the parts that don’t match the project’s conventions, fixing the edge case it missed, adjusting the tests. Writing from scratch would have started with the architecture you wanted and built toward it. Here you start with someone else’s architecture and retrofit your intentions onto it.

METR’s own explanation for why developers keep reaching for AI is modest: they may simply find it a more pleasant, less effortful way to work. My reading of the perception gap goes a step further, and it echoes what BCG found in 1,488 workers about AI-induced cognitive overload. Instant code triggers a sense of progress, because something happened and the file is longer than it was. The remaining work (reviewing, editing, integrating, debugging) stays invisible until you’re in it, and by then the sense of progress is already locked in. Even after the slowdown, METR’s developers rated their perceived speedup at 20%.

Stack Overflow’s 2025 Developer Survey, its most recent edition with 49,000+ respondents, captures the same friction from the inside. The top frustration, picked by 66%, was “AI solutions that are almost right, but not quite,” and 45.2% said debugging AI-generated code is more time-consuming. Only 33% trust the accuracy of AI output, while 46% actively distrust it. Almost-right code is what METR’s screen recordings showed developers editing, and it’s the same gap that makes vibe coding feel faster than it is on anything with existing structure.

The Develocity (formerly Gradle) team’s write-up on the productivity paradox describes the organizational side, drawing on the 2025 DORA report and case studies from its DPE Summit: near-universal adoption and high perceived speed, while “the system often fails to carry or amplify these individual gains.” The Stanford AI Index 2026 puts these productivity numbers in a much wider dataset.

A UC Berkeley Haas study, also covered by Scientific American, followed employees (engineers among them) at a roughly 200-person US tech company for eight months. People who adopted AI took on a broader scope of tasks, worked at a faster pace and worked longer hours, and some began prompting AI during lunch and breaks. The tool that was supposed to save time expanded the amount of work. Anthropic saw a similar effect in an internal survey of 132 of its own engineers and researchers: 27% of their Claude-assisted work consisted of tasks that wouldn’t have been done otherwise.

The PR Review Bottleneck

The Faros data makes a structural problem visible. AI tools sped up writing and left reviewing where it was, so every gain on the writing side lands in someone’s review queue.

The 2025 numbers show how fast that compounds. When each developer merges 98% more PRs and each PR is 154% larger, the lines waiting for review grow about fivefold (1.98 × 2.54 ≈ 5.0), which roughly lines up with the fivefold jump in median review time Faros measured a year later (different metric, same order of magnitude). Attention doesn’t scale the same way: a 400-line AI-generated diff rarely gets the scrutiny of a 100-line change the reviewer watched take shape.

The people who need to approve PRs are usually the most experienced developers on the team, the same population METR measured getting slower with AI tools in 2025. That means the developers who gained least from AI-generated code are also the ones being buried under the most AI-generated code to review.

Google’s 2025 DORA report, built on nearly 5,000 survey respondents, adds a more hopeful data point. AI adoption reached 90%, and for the first time DORA linked it to higher software delivery throughput, a reversal of its 2024 finding. Instability is still the weak spot: AI adoption remains associated with more change failures and rework. DORA explains the split with its central framing, that AI amplifies whatever an organization already has, and points to foundations such as small batches, strong version-control practices, quality internal platforms and a clear, communicated AI policy as what turns the extra volume into delivery.

Some teams are adding AI-assisted code review: CodeRabbit, GitHub’s Copilot code review, and Graphite, which Cursor acquired in December 2025. I’d treat them as a first pass for formatting slips and obvious bugs. Architectural problems, subtle logic errors and project conventions still need the senior reviewers the queue is burying.

Who Benefits, Who Gets Hurt

The data splits along experience lines.

Junior developers come out ahead in most field data, with one wrinkle. A McKinsey study found that AI tools cut time on code documentation by 45-50% and code generation by 35-45%, with savings dropping below 10% on high-complexity tasks, and it also found that developers with under a year of experience sometimes took 7-10% longer with the tools. The stronger evidence for juniors comes from field experiments led by MIT economists at Microsoft, Accenture and a Fortune 100 company: newer, less experienced developers adopted GitHub Copilot more readily and raised their output by 27-39%, against 8-13% for senior developers. Junior engineers spend more of their time on routine work (boilerplate, standard CRUD, well-documented API integrations), which is where the tools help most.

Senior developers on familiar code are the open question. In METR’s 2025 study they got measurably slower: developers who knew the codebase, understood the solution architecture and had strong debugging intuitions didn’t need the AI’s suggestions, and the tool added friction to a workflow that was already efficient. The slowdown was largest on issues they already knew well. The 2026 point estimate for the returning developers suggests that has improved, but the interval is wide enough to include another slowdown, so nobody has a clean answer for this group yet.

My read is that the organizational sweet spot is narrow. Moderate adoption (tools available but not mandated, guidelines for when to use them, protected review time) looks safer than either extreme, and it lines up with DORA’s emphasis on a clear, communicated AI policy. All-in mandates create the bottlenecks Faros documented, and no adoption gives up the gains on routine tasks.

The job market makes this worse. Companies replacing junior roles with AI are removing the population that benefits most from the tools, while the senior engineers who remain get more code to review and a heavier workload. The 125,000+ tech layoffs logged in 2026 by early August, already past all of 2025’s total, add pressure from the other side: by this data, routine roles are where AI tools had the best chance of paying off.

The Skills Tax

Anthropic published a study on AI assistance and coding skills in January 2026 that measured a second cost. Fifty-two engineers learning a new Python library (Trio, for async programming) were randomly split: half had an AI coding assistant in the sidebar, half had only documentation and web search.

The AI group finished exercises at roughly the same speed. But on a comprehension test afterward, they scored 50% versus the manual group’s 67%, a 17-point gap. The gap was largest on debugging questions. Within the AI group, interaction style predicted outcomes: developers who used AI for conceptual inquiry (“explain how Trio cancellation scopes work”) scored 65%+ on the overall quiz. Those who delegated code generation (“write a Trio server that handles cancellation”) scored below 40%.

If you’re adopting AI tools across an engineering org, the junior developers who benefit most from code generation are also the ones most at risk of stunted skill growth. The debugging and architectural skills they would have built by struggling through problems don’t develop when an AI does the struggling for them. This stunted-growth pattern is part of why junior developer employment is contracting while senior roles are growing, and it feeds straight into whether a CS degree still pays off for people entering the field now.

Claude Code ships two teaching-oriented output styles: “Explanatory”, which adds notes on the choices it makes as it works, and “Learning”, which leaves parts of the code for the developer to write. Whether teams use them over the fast defaults is an open question.

What to Do With This Data

Here’s what the data suggests for individual developers, for teams and for job seekers.

For your own workflow:

  • Track your own numbers. Measure time-to-merge instead of time-to-first-draft. Most developers overcount the writing phase and undercount review, debugging, and rework.
  • Point AI at unfamiliar territory first. McKinsey’s biggest time savings came on documentation and routine generation, and Anthropic’s skills study found that developers who used AI to ask conceptual questions kept their comprehension. The returns shrink in a codebase you already know well, which is the exact setting where METR measured its 2025 slowdown.
  • Reject freely. Read METR’s 44% acceptance rate as experienced developers correctly turning down most suggestions. If you’re accepting 80%+ of AI completions, you’re probably not reviewing carefully enough.

At the team level:

  • Budget for review load. If your team adopts AI tools, expect review time to grow and staff accordingly; protected review time, review rotations and PR size limits stop being optional.
  • Don’t measure output in lines of code or PRs merged. Both Faros reports show those numbers rising, and the 2026 report also logged deployments per week down 11.7%. Measure cycle time from issue creation to production deployment.
  • Watch for the skills gap. Junior developers need deliberate practice time without AI (pair programming, code katas, architectural discussions) or they’ll hit a ceiling when they need to debug something the AI can’t fix.

For the 2026 job market:

  • Hiring is shifting toward people who can judge AI output. SignalFire’s 80M-company study found engineers still made up 55% of Big Tech hires while the role mix moved under them, and the 2026 SWE job market breakdown tracks the gap between AI/ML postings and general software postings in the US. European developers should weigh their own market too: salaries across the EU in 2026 tell a different story from the raw US numbers. My bet for both markets: the developers employers hold on to are the ones who understand the code well enough to review it critically, whoever or whatever wrote it.
  • Understanding these productivity numbers gives you an edge in interviews. When a hiring manager says “we’ve adopted AI across the org and productivity is up 40%,” you can ask: “Are you measuring task completion or cycle time?” A manager who has read the Faros or DORA data will know exactly why you’re asking.

FAQ

Do AI coding tools actually improve developer productivity?

It depends on who’s using them and how you measure. For routine tasks, the McKinsey study measured 35-50% time savings on code generation and documentation. For experienced developers on complex, familiar codebases, METR measured a 19% slowdown in 2025, and its 2026 rerun points to a speedup it can’t confirm. At the organizational level, Faros found no significant productivity improvement across 1,255 teams in 2025, and its 2026 data shows more output alongside much longer reviews and more bugs. The tools change what developers do with their time more than how much their teams ship.

Did METR’s follow-up study find AI makes developers faster?

Probably, but METR won’t claim it. The February 2026 update estimated that AI-allowed tasks took 18% less time for returning developers and 4% less for new ones, with confidence intervals that include zero. METR calls the data very weak evidence because 30-50% of developers were withholding tasks they didn’t want to do without AI, which biases the estimate. It believes developers are likely more sped up than in early 2025 and is redesigning the study to measure it properly.

Why do AI coding tools make experienced developers slower?

In METR’s 2025 study, experienced developers already had efficient workflows for codebases they knew well. AI tools added new steps (prompting, waiting, reviewing suggestions, deciding what to accept, editing accepted code) that replaced activities the developer was already fast at. Less than 44% of AI suggestions were accepted, so a lot of the AI interaction time produced nothing usable. Evaluating someone else’s approach to your code is often more work than writing your own.

Does AI make coding faster?

Code generation is faster. Shipping is a different question, and the organizational data is mixed at best. The bottleneck moved from writing to reviewing: in Faros’ 2025 data PRs grew 154% larger and took 91% longer to review, and in its 2026 data median time in review rose 441.5%. Faros found no company-level gain in 2025, and where DORA did find higher throughput, it came with higher instability.

Why are developers using AI working longer hours?

A UC Berkeley Haas study of a roughly 200-person US tech company found that employees who adopted AI, engineers included, expanded their scope, worked at a faster pace and prompted AI during lunch and breaks. Saving time on one task makes it feel possible to start three more. The net result is more output but also more hours logged.

How does AI assistance impact coding skills?

An Anthropic RCT found that developers using AI scored 17 points lower on comprehension tests when learning a new library (50% vs. 67%). The gap was largest for debugging skills. Developers who asked the AI to explain concepts retained knowledge; those who delegated code generation didn’t. AI tools help experienced developers apply existing skills but can prevent junior developers from building new ones.

Sources

Bottom Line

My recommendation is to keep the tools. The gains on exploration, unfamiliar libraries and boilerplate show up in nearly every dataset here, and METR’s own rerun suggests individual speed has improved since early 2025. What got worse as adoption deepened is everything downstream of the editor: Faros’ 2026 numbers put median review time up 441.5% and bugs per developer up 54%. Teams that adopt these tools without measuring review load, bug rates and delivery speed are buying a perception of productivity. Measure cycle time from issue to production, budget for the review bottleneck before it buries your senior engineers, and if you’re a developer, track your own numbers. METR’s 2025 data showed how far a developer’s sense of speed can drift from the clock, even right after the work is done.