The Rejected PRMerge Rate Metrics

Measuring Reviewer Load on Teams Using AI Coding Assistants

Four metrics reveal whether AI coding assistants create a review bottleneck.

Cover illustration for “Measuring Reviewer Load on Teams Using AI Coding Assistants”

To measure reviewer load on teams using AI coding assistants, teams need to track more than pull requests and lines of code. Those measures show how much code enters the system. Four signals reveal the difference, namely PR size growth, segmented review cycle time, bug-per-PR rates, and cognitive demand per review. Together, they expose a verification bottleneck before it becomes a delivery crisis.

Why PR count and lines of code no longer measure what teams think they measure

The standard measurement frame was built for a bottleneck in implementation. Code generation and verification were roughly coupled, with one developer producing a PR that one reviewer could assess within a manageable queue. AI coding assistants break that coupling because one engineer can run several agents in parallel, each producing another diff that needs attention. PR count, lines of code, and seat utilization record the increase in generation without measuring the judgment required to verify it. As a result, they can overstate team capacity even when the underlying counts are accurate.

This explains how a team can report faster implementation and more merged PRs without improving end-to-end delivery. The Productivity-Reliability Paradox paper documents a near-doubling of merged pull requests coinciding with a near-equivalent rise in review time and flat organizational delivery metrics. Faros telemetry found that the median time in PR review is up 441% where AI adoption is high. These signals are not contradictory, because implementation speed and delivery speed describe different parts of the system. DORA's 2025 State of AI-Assisted Software Development report supplies the broader explanation that AI amplifies downstream conditions, allowing strong feedback loops to absorb volume while exposing weak review processes. Measurement therefore needs to establish whether verification capacity increased with generation.

What AI-generated code actually does to reviewer work

When a reviewer opens a problematic human-written PR, awkward naming, inconsistent style, or obvious shortcuts can provide an initial map of where to investigate. AI-generated code can lose those surface cues while keeping the errors they would have pointed to. It may be idiomatic and structurally tidy while misunderstanding a requirement or handling an edge case incorrectly. The reviewer must look past how well the implementation seems to fit together and check whether it solves the actual problem. Clean presentation does not establish correct intent.

Faros's 2026 AI Engineering Report describes this burden as “the tax falling on senior engineers”. Reconstructing intent requires the reviewer to establish what the code was meant to accomplish before judging whether its logic achieves that result. That is more demanding than following visible trouble spots through an otherwise understandable change. The mechanical burden compounds the reasoning burden because PRs under high AI adoption are larger and touch more files.

Faros data shows that review time is still up nearly 200% where AI adoption is high, even when AI review agents do the mechanical checks. Automation can reduce part of the work while leaving the harder evaluation in place. Measuring that remaining work requires attention to its depth and duration.

Where the load actually concentrates and who absorbs it

Jazzband illustrates how review demand can overwhelm a model built on shared participation. The project covered 71 open-source projects with over 150 million PyPI downloads a month. In March 2026, it announced that it was sunsetting operations because AI-generated spam PRs and issues had made its open membership and shared push access model untenable. The announcement also said that long-standing issues with how a single maintainer ran the project played a part. Only a small fraction of AI-generated PRs met project standards, and reviewer capacity could not absorb the volume without undermining the trust model.

Context, risk ownership, and authorization boundaries sit with a small group of engineers, so a longer queue does not make every reviewer interchangeable. Less-experienced reviewers cannot simply take on security-sensitive changes or changes that affect the architecture when they lack the knowledge and authority needed to review them. The resulting pressure falls on senior engineers who must choose between delaying work and approving it superficially. Both choices reduce the value that faster generation was meant to provide.

The Productivity-Reliability Paradox paper found no measurable improvement in organizational DORA metrics despite individual-level gains. Deployment frequency, lead time, and change failure rate therefore support the same diagnosis at the system level. Additional output does not become additional delivery when the people qualified to verify it remain the constraint.

Why PR size growth is the first signal to track

Median PR size is the most direct starting point because it describes the unit of work entering review. AI agents left unconstrained produce larger diffs, and Faros telemetry found PRs are 51% larger under high AI adoption. That increase changes what a reviewer must hold in mind during an assessment. Size is also visible before queue depth grows, review cycle time lengthens, or defects escape.

The cost of that growth is not linear. A reviewer who can confidently evaluate a 200-line diff cannot simply extend the same confidence to a 600-line diff. Larger changes demand more context holding, more cross-file reasoning, and more passes through the implementation. Counting lines remains useful here, but their role changes from an output measure to a warning about review demand.

Track median PR size separately for AI-assisted and non-assisted PRs because an overall median can conceal a widening gap between them. Human correction and rework provide useful companion signals, including additional commits after review, significant human edits, and repeated agent attempts. These show where apparent generation speed is being paid back through correction.

What review cycle time reveals when it is segmented correctly

Review cycle time becomes useful when it identifies where work waits and whose capacity it consumes. The average across an organization can look fine while AI-assisted PRs wait much longer for a qualified reviewer. Many fast, small human PRs can pull that average down and conceal the queue that matters. Segmenting by PR type and reviewer makes the constraint visible. Without that separation, the metric describes elapsed time without locating the bottleneck.

Review pickup time is the first useful distinction within that timeline. It measures how long an AI-assisted or agent-authored PR waits before a qualified reviewer begins. If generation accelerates while pickup time increases, the organization is creating work faster than its verification system can absorb it. This connects the earlier size signal to an observable capacity problem. The larger input is no longer merely a potential burden because it is now waiting for scarce attention.

A PR approved in one round consumes a different amount of attention from one requiring three rounds of correction, even when their total elapsed times look similar. Post-merge correction also matters because hotfixes, reverts, and immediately reopened work move verification costs beyond the PR's own timeline. A short review cycle can therefore underreport total effort.

How bug-per-PR rates expose what review is actually catching — and missing

Size and cycle time show the volume and movement of work, but neither establishes whether review succeeds. Bugs per PR show how well review catches errors, especially when defects pass review and reach production. Faros telemetry found bugs per PR are substantially higher under high AI adoption. A higher-defect input stream is arriving at higher volume. Review must keep up with both changes to keep working as well as it did.

Logic and correctness issues deserve particular attention because they match the failures that clean-looking code can conceal. Business logic errors, misconfigurations, and incorrect edge-case handling take more than a check of style or structure to find. They require the reviewer to connect the implementation to its intended behavior. When these defects escape while reviewer queues and demand rise, they contribute to the load picture by indicating that throughput pressure is reducing review depth.

Escaped defects are high-signal but lagging evidence because the code has already passed through review. Rework PRs provide an earlier proxy by identifying changes opened to fix or undo recently merged code without waiting for a production incident. Acceptance and retention therefore belong together, since merging code does not establish that it survives without being rewritten, reverted, or replaced. DORA's 2025 addition of Rework Rate as a fifth metric recognizes this gap in traditional delivery measurement. A fast approval followed by corrective work is not the same outcome as a change that remains useful.

Measuring cognitive demand per review, which no standard dashboard exposes

Cognitive demand matters because review quality depends on the reasoning a reviewer can sustain. Qodo's 2026 State of AI Code Quality Report found that 36% of developers said review takes the same time but takes more mental effort. Time-on-task misses even more of the work when reviewers switch contexts or give superficial approvals under pressure from the queue. Ignoring cognitive demand risks optimizing speed while the capacity for careful judgment deteriorates.

Observable proxies make this burden more tractable, although none measures mental effort directly. Review rounds before approval show where confidence required repeated evaluation or substantial correction. Reviewer comment depth compares substantive comments about logic, correctness, and architecture with mechanical comments about style and naming. Time to first meaningful comment after pickup can expose the period spent building context before a reviewer can contribute. A reviewer concentration index adds the distributional view by showing how many PRs a single senior engineer reviews per week relative to the team.

These signals are most meaningful when related to human review effort per accepted change. Rounds, comments, requested changes, and human time all contribute to that effort, including work spent on submissions that are rejected or heavily reworked. Reviews per day and active queue depth for each senior engineer help show whether growing output is drawing disproportionately on the same people. Baselines also need attention now because AI assistance increasingly becomes a default, making clean comparisons between assisted and unassisted review work harder to establish.

Putting the four signals together into a reviewer load dashboard

Diagram: Four Signals of Reviewer Load — and What Each Exposes. Visualizes: Visualize a four-stage sequence showing how the signals build on one another to diagnose a verification bottleneck.

The four signals form a sequence. PR size growth shows that the input is changing, while segmented cycle time shows whether qualified reviewers can absorb it. Rising bug-per-PR rates then reveal whether throughput pressure is compromising review depth. Cognitive demand proxies show whether the people maintaining that throughput can sustain the effort. Their combined value is the ability to distinguish increased activity from reliable delivery.

Implementation should begin with an audit of what the current toolchain already exposes. GitHub's August 2026 enterprise roundup featured the Copilot Impact Dashboard, originally released July 22, 2026, to help engineering leaders understand AI activity and its downstream effects. Qodo's AI Code Review Platform ranked first for F1 score on Martian's Code Review Bench in March 2026. Its multi-repository context and review-quality signals make it one option to evaluate for inputs to the cognitive demand proxy layer. Tooling can supply evidence, but it does not decide who is qualified to review a change or how much judgment that person can absorb.

Instrument PR size and the assisted-versus-unassisted baseline first, then connect them to pickup time, review rounds, and reviewer-level queues. Track defects and rework too, to check whether changes that pass review stay in the codebase. Read cognitive demand proxies across that sequence to check whether reviewers can keep up, since fast approvals alone do not show that they can. Faros telemetry showing 31% more PRs are merged with no review at all under high AI adoption demonstrates the cost of treating reviewer attention as freely scalable. Early measurement preserves the option to adjust PR scope rules, rotate review responsibility where qualifications permit, or gate agentic output before the quality cost compounds.

Notes

  1. The Productivity-Reliability Paradox: Specification-Driven Governance for AI-Augmented Software Development

    Provided the finding of a near-doubling of merged pull requests alongside rising review time and flat organizational delivery metrics, and the absence of measurable DORA improvement despite individual-level gains.

Every new piece from The Rejected PR, as it's published.

Follow

Marcus Calderón

More from The Rejected PR

  1. November 19, 2026Defining Merge Rate as an AI Coding Quality Signal8 min read →
  2. November 3, 2026Implicit Coding Conventions AI Tools Consistently Violate8 min read →
  3. October 9, 2026Why AI-Generated PRs Fail Code Review9 min read →