The Rejected PRAI PR Rejections

Reviewer Feedback Patterns on AI-Written Code

AI code fails through four predictable patterns reviewers must learn to spot.

Cover illustration for “Reviewer Feedback Patterns on AI-Written Code”

AI-generated code fails differently than human-written code, not merely more often. The defects cluster around four predictable patterns: logic errors, hallucinated APIs, security anti-patterns, and context blindness. Recognizing these patterns is what allows a reviewer to allocate attention where it will actually catch failures.

Why AI-generated code surfaces a different mix of defects than human-written code

The most dangerous property of AI-generated code is that it looks finished. It arrives syntactically polished and consistently formatted, which signals competence to a reviewer trained on human output. A junior developer's rough code announces its own risk profile. It is hesitant, inconsistent, and prompts closer inspection. AI code does the opposite. It is confidently wrong, and the confidence is structural rather than incidental.

The mechanism underneath this difference is straightforward. MIT CSAIL researchers found that every company's codebase is "kind of different and unique," making proprietary conventions "fundamentally out of distribution" for AI models. The model cannot be wrong in the ways a human who knows the system would be wrong, because the model has never seen the system.

This distinction matters for how review time gets spent. AI code fails in ways that correlate with training-data frequency and context absence. The review habits calibrated for the first set of failures do not transfer to the second.

Logic errors and correctness failures: where AI code breaks most often and most subtly

Diagram: AI Code Defect Patterns: Where Failures Concentrate. Visualizes: Visualize four distinct defect categories in AI-generated code as a ranked or layered breakdown, showing each pattern and its core detection challenge.

Logic and correctness errors are the largest defect category in AI-generated code, and the hardest to catch with automated static analysis. A linter can flag an unused variable. It cannot determine whether a function handles the empty-list case correctly. These errors require understanding intent, not just syntax, which is exactly what a reviewer contributes and a static tool cannot.

CodeRabbit's analysis places logic and correctness as the single largest gap between AI and human code, ahead of security and performance. Null inputs, boundary conditions, error propagation, retry logic, partial failures. These are the scenarios that test coverage tends to miss as well, because tests written to spec often encode the same happy-path assumptions the model made.

The detection difficulty is what makes this category distinctive. Queen's University's study of 278,790 code review conversations found human reviewers exchange significantly more rounds when reviewing AI-generated code than human-written code. That is a behavioral signal: logic problems require more back-and-forth to resolve because they do not present as obvious breaks. The error is only visible when a reviewer traces what happens when the second API call fails, or when the input list is empty, or when the timestamp crosses a timezone boundary. Logic errors in AI code do not announce themselves. The reviewer has to slow down and trace execution paths rather than skim for visible gaps.

Hallucinated APIs: the failure mode that looks like working code until runtime

Hallucinated API calls are the failure mode with no real analog in human-written code. The model generates a call to a function that does not exist, uses a signature that never existed in that library version, or references an interface that was deprecated three releases ago. The call reads as idiomatic. It follows the grammatical conventions of the language precisely. It is fiction with correct syntax.

CodeAnt AI's review pipeline guide identifies hallucinated APIs as a distinct failure mode requiring its own detection strategy, not covered by traditional diff review. TypeScript strict mode catches some cases at compile time, because the type checker can verify that a named export exists or that an argument count matches. Python, JavaScript, and Ruby are less forgiving in the opposite sense: they leave hallucinated calls undetected until runtime, when the interpreter finally reaches the line and discovers the function was never defined.

Why does this survive review? A reviewer who does not know the specific library version in use cannot spot it by reading alone. Cross-file blindness compounds the problem. The function may be defined, overridden, or removed elsewhere in the repository, outside the diff the reviewer is examining. You need either automated API-existence checking against the actual dependency graph, or a reviewer who validates calls against the real dependency manifest instead of trusting the code's internal plausibility.

Security anti-patterns: AI code reproduces known vulnerabilities at a higher rate than human code

Security failures in AI-generated code are not generic. They are patterned, and the pattern traces directly to training data. The model does not reason about security intent. It generates statistically representative code. Historically, a great deal of shipped code has been insecure, which means the statistical representation of code is also insecure.

CodeRabbit's December 2025 analysis found AI-generated code is by a large margin more likely to introduce XSS vulnerabilities and insecure object references than human developers. These specific vulnerabilities are instructional, because they are not exotic. A human developer who has internalized secure coding practices avoids them by default. A model that learned from the median of historical code reproduces them at the median rate of historical code, which is higher than the rate a trained human produces.

The review implication is that security review of AI code cannot be treated as a checklist pass. Traditional SAST tools catch known, deterministic security patterns: a SQL string built by concatenation, a hardcoded credential. Whether an object reference is safe depends on the authorization model, not just the line of code. Qodo's 2026 analysis identifies security as a domain requiring specialized review agents with focused evaluation rubrics: injection risk, authorization and authentication boundaries, secrets exposure. The check must reason about whether the code's behavior matches the intended authorization model, not whether it matches a known vulnerability signature.

Context blindness: why AI code breaks conventions and architectural rules the model cannot see

Context blindness is not a bug category like XSS. It is the root condition that produces AI code that is locally coherent but globally wrong. The model generates code that violates architectural decisions, team conventions, and system-level invariants it was never shown. These are not errors of logic so much as errors of absence: the answer is correct for a different codebase.

CodeAnt AI's guide cites MIT CSAIL research finding that proprietary codebases are "fundamentally out of distribution" for AI models. The model generates reasonable general-purpose code that violates the specific rules of this codebase. The AI queries a service's database directly when the team convention is that Service A always goes through Service B's API. It wraps HTTP calls without the team's custom retry client. It adds a field to a data model without updating the schema migration that the codebase's deployment process depends on. Each individual decision is defensible in isolation. Each one breaks an invariant the model never saw.

Context blindness also explains false-positive review noise, which is the same condition inverted. AI reviewers flag code as wrong when the code embeds an intentional design decision the AI cannot see: a buffer-based over-fetch strategy, a deliberate UTC offset, an existing feature implemented elsewhere in the file. Queen's University's empirical study found that human reviewers provide feedback including understanding, testing, and knowledge transfer that AI agents do not. That is not a failure of AI capability so much as a statement of where context lives. Human reviewers hold context the AI lacks and transmit it through review. The reviewer's job under these conditions is not just bug-spotting. It is context-carrying: bringing architectural and convention knowledge actively to the review, because the tool generating the code did not have it.

How AI review feedback itself compounds the problem when it lacks codebase context

The same context blindness that produces defects in AI-written code produces a different but related failure in AI review feedback. When AI code review tools operate without full codebase context, they generate a predictable set of false-positive patterns that bury real findings and train developers to ignore AI feedback. The false positives are not random. They follow recognizable shapes. A context-blind reviewer flags code that was already fixed in a later commit, because it is evaluating an older diff snapshot rather than current HEAD. It suggests features that already exist elsewhere in the file or in imported modules. It raises concerns about scenarios that cannot occur given the deployment environment. The hardest false positive to dismiss is the one where the AI flags a real edge case without knowing the code already accounts for it at the system level. The reasoning sounds technically correct but misses the design invariant, which makes the comment both credible and wrong.

Queen's University's large-scale study quantified the consequence. AI agent code suggestions are adopted at 16.6%, significantly lower than suggestions from human reviewers. Over half of unadopted AI suggestions are either incorrect or addressed through alternative fixes by developers. The review gate degrades not because the AI is always wrong, but because the signal-to-noise ratio makes triage expensive. Qodo's 2026 analysis frames attention as finite: without severity-driven triage, critical issues get buried under cosmetic noise, and developer trust in AI review drops even when the AI is technically accurate. The leverage point is not more AI output. It is triage.

What a reviewer actually needs to do differently when the code is AI-generated

Knowing which defect patterns AI code concentrates lets a reviewer allocate attention strategically rather than treating every line with equal suspicion. The patterns are specific enough to support a checklist, but the checklist is not a mechanical pass. Each item directs attention to a place where the failure mode is known to concentrate. For hallucinated APIs, validate calls against the actual dependency manifest. SAST with real dependency graph awareness catches calls to functions that do not exist in the declared version of a library, which is exactly the check that reading the diff cannot perform.

For security anti-patterns, apply specialized security review at the authorization and injection boundary. Reason about intent rather than pattern-matching to known CVEs. The question is not whether the code contains a recognizable vulnerability signature, but whether its behavior matches the intended authorization model.

For context blindness, bring architectural and convention knowledge actively to the review. Queen's University's findings confirm human oversight remains critical for suggestion quality and contextual feedback that AI agents lack. The implication is not to reduce human review but to redirect it toward what humans uniquely contribute: the context that lives in team memory rather than in the repository.

Qodo's 2026 prediction on severity-driven triage points in the same direction. Structuring review output as Action Required, Recommended, and Minor prevents critical findings from being buried and gives developers a clear policy for what must be resolved before merge. The specialist-agent model applies the same logic to tooling: a correctness agent, a security agent, a performance agent, an observability agent, a requirements agent, and a standards agent, each with focused rubrics, produce less noise than a single generalist pass. The pattern holds at every level: targeted attention beats uniform suspicion.

Why review pipelines designed for human-speed code production are structurally insufficient for AI-volume output

The volume of AI-generated code has broken the assumption that human reviewers can function as the primary quality gate. GitHub reports nearly 1 billion commits in 2025 with a large surge in generative AI projects. Google and Microsoft report AI-generated code now comprises roughly a third of new code at their companies. The defect patterns in AI code make it a compound problem: more code, harder-to-spot failures, and a review process that was never designed for this rate.

The efficiency metrics are misleading if read in isolation. Queen's University's study found that agent-involved collaboration patterns, especially under Gradual AI Adoption and Rapid AI Agent Adoption, are associated with faster review decisions. But these efficiency gains do not translate into better review quality. Most LLM or AI agent collaboration patterns show significantly higher review quality risk than human-only review. Speed and quality are not the same problem, and optimizing for the first without measuring the second produces a pipeline that fails faster.

The teams that adapt will treat context as a required input to review, assembled before the diff opens, not as whatever the reviewer happens to remember. Sourcegraph's evaluation of automated code review tools identifies the practical architecture most teams converge on: deterministic static analysis as the high-confidence compliance gate, combined with AI-native review for semantic, context-dependent findings earlier in the pipeline. Neither alone is sufficient. The deterministic layer catches what can be certified. The contextual layer catches what must be understood.

Review is a system with feedback loops, not a one-off checkpoint. The defect patterns AI code concentrates require that the system be designed around those patterns, not retrofitted to catch them by chance. Knowing the patterns is the prerequisite. Redesigning the pipeline around them is the work.

Notes

  1. 1 Human-AI Synergy in Agentic Code Review

    Provided empirical data on AI code review conversation volumes, suggestion adoption rates (16.6%), review quality risk, and the ~1 billion GitHub commits figure that the article cites throughout.

  2. A Survey of Bugs in AI-Generated Code

    Informed the article's characterization of logic errors, hallucinated APIs, and security anti-patterns as the dominant defect categories in AI-generated code.

Every new piece from The Rejected PR, as it's published.

Follow

Priya Sundaresan

More from The Rejected PR

  1. October 9, 2026Why AI-Generated PRs Fail Code Review9 min read →
  2. November 3, 2026Implicit Coding Conventions AI Tools Consistently Violate8 min read →
  3. October 31, 2026Ghost PRs and Silent AI PR Abandonment7 min read →