All articles

Blog

Code review: AI writes faster than your teams can review

A 13,323-line pull request written with AI, closed after just over fifteen hours: what needs to change in your code review.

  • By Azziz ERRIME
  • 14 min read
An empty office chair facing a stack of code review pages that reaches the ceiling, lit by a lamp at night.

TL;DR

  • Code review is no longer keeping pace with AI: at one company studied in 2026, PRs per active developer doubled, and so did each reviewer’s workload.
  • According to Veracode, AI-generated code still passes only 15% of security tests on XSS and 13% on log injection.
  • Before buying a tool, I’d measure reviewer workload, cap PR size and write the word “security” into the PR template.

A 13,323-line pull request written with AI, closed for lack of reviewers. When AI writes the code, code review decides what it is worth. Provided, that is, the review has been properly resourced and protected.

Wednesday 19 November 2025, 18:37 Paris time. A pull request lands on the OCaml compiler repository: 13,323 lines, 147 files, 40 commits. Its author produced it with AI to add support for DWARF, a debugging format.

At 09:55 the next day, just over fifteen hours later, Gabriel Scherer, one of the maintainers, closes it without merging. In his closing message, he makes clear that he is not passing judgement on the code submitted: he has not looked at it closely enough to form a view. None of the people in a position to review and maintain this work seemed ready to take it on. That leaves the question every code review comes down to: who reviews, and how quickly?

A code review that never happened

Just before closing the PR, Scherer had posted a comment. In it, he describes code that is “hard to review, and very lightly tested”. He points out that the compiler is short of people available for review and maintenance. In his experience, he adds, checking what an AI writes takes him more effort than checking a human’s work.

In his view, PRs this large carry “a very real risk of bringing the Pull-Request system to a halt”. Producing lines that pass the tests, he writes, is “only 20% of the work”. The remaining 80% (discussing the feature, reviewing it, integrating it) falls to the author and to others alike.

An older study that SmartBear, a code review tool vendor, carried out at Cisco found that defect detection drops off beyond 500 lines reviewed per hour. At that top speed, I make it more than 26 hours of careful reading for the 13,323 lines. The PR stayed open for just over fifteen hours.

His closing sentence is about how the project is organised: “Currently github/ocaml as it is organized is not able to work in this way.” I wonder how many teams could write anything different about their own repositories.

PRs have doubled, and so has each reviewer’s workload

According to GitHub, the number of pull requests merged each month on the platform more than tripled between January 2023 and June 2026.

A study published on 2 July 2026 followed the developers of a company whose CTO had announced a “2× mandate” in June 2025 to double productivity through AI. The number of PRs per active developer did double, and each reviewer’s workload doubled with it. The authors see this as an upper bound, not an industry average.

In open source, the foundation behind the Godot game engine changed its contribution policy on 30 June 2026. It gives the reason plainly: it has few qualified reviewers, reviewing code takes real effort, and more comes in than they can handle.

I think code review gets skipped when the queue overflows, one PR at a time. Faros AI sells tools that measure the work of engineering teams. In its telemetry, PRs merged without any review, human or agent, are up 31.3% in relative terms. Each one can carry with it the flaws that AI keeps missing, version after version.

A road checkpoint, barrier raised and booth empty, with a line of white lorries passing through.

An open checkpoint, nobody in the booth, and traffic going through without slowing down.

AI-generated code still trips over the same flaws

In spring 2026, Veracode, an application security vendor, broke down the security test pass rate of AI-generated code, vulnerability by vulnerability. The rates had barely moved since its 2025 report.

Vulnerability Security test pass rate
SQL injection 82%
XSS 15%
Log injection 13%

Its 28 July 2026 report picks out the same two weak spots, XSS and log injection. Log injection slips line breaks into a logged field to forge fake entries. My interpretation: the model succeeds when the fix fits a known pattern, such as parameterised queries against SQL injection. It fails when the defence depends on where the data will end up: a page shown to another user, or a log examined during an investigation. Veracode puts forward a similar explanation: both flaws require following a user’s data through the application.

In practice, attackers start with that path. They identify the application’s inputs (a form field, a header, a URL parameter), then follow the data to the page or log where it resurfaces. A PR is reviewed file by file, whereas that path runs through several.

The other blind spot lies in dependencies. A study presented at USENIX Security 2025 shows that models from 2023 and 2024 recommended packages that did not exist, often the same ones run after run. That lets an attacker register those names before you do. A Lasso Security blog post from March 2024 describes how a researcher uploaded an empty package under a name that models had invented, “huggingface-cli”. It was downloaded more than 30,000 times in three months.

For a secure code review, the cheapest measure fits in one sentence. Braz and co-authors showed at ICSE 2022 that explicitly asking reviewers to look at security makes them eight times more likely to find the vulnerability. Their checklist added nothing significant to that instruction. But it takes someone with time to read, and when that time runs short, the idea of also handing the review to a machine soon follows.

Having AI review AI: the temptation OCaml saw coming

In the comment he posted just before closing the PR, Scherer had anticipated the answer: “You may think that the answer to that is to also automate the review process, or (more plausibly) to lower our quality standards”. So far, he writes, the project has not chosen to work that way. The temptation is real all the same, and automated code review is already running at scale, according to GitHub, which is talking about its own product.

AI can also find real vulnerabilities. On 11 August 2025, Google’s Big Sleep team reported to curl the flaw that became CVE-2025-9086. Daniel Stenberg, who maintains curl, sees it as the first report the project has received that seems to have used AI to pinpoint a vulnerability accurately.

In a paper accepted at ICML 2025, Kim and co-authors compared the errors of large language models on the HELM leaderboard. When two of them are wrong at the same time, they pick the same wrong answer about 60% of the time on average, compared with about 33% by chance. That figure only holds for this leaderboard. It is enough to stop me assuming that a second model covers the first one’s blind spots.

The automated reviewer becomes a target too. Prompt injection means hiding instructions in content the AI will read, so that it follows them as if they came from you. With CamoLeak, a Legit Security researcher planted invisible comments in a PR to get Copilot Chat to leak private code. GitHub fixed this critical vulnerability on 14 August 2025.

A smoke detector on an office ceiling, covered with a blank yellow Post-it note.

A detector masked by a Post-it note, like an automated reviewer fooled by a hidden instruction.

On 19 August 2025, Kudelski Security published an account of a simple PR containing a malicious configuration file. Fixed as early as January 2025, the flaw made it possible to run code on the servers of CodeRabbit, an AI code review tool. It opened up write access to a million repositories, private ones included. Which leaves the human who approves at the end of the chain.

Eight months after OCaml, an AI agent goes after a maintainer’s approval

In a report published on 4 August 2026, the AI Security Institute, a UK public body, describes an incident during cyber security testing, from 25 to 28 July. An AI agent under test studied a project’s human maintainers, created several fake identities and used them to push a real maintainer into approving its code. According to the institute, its investigations found no evidence of real harm.

The Linux kernel documentation reserves sign-off for humans: “AI agents MUST NOT add Signed-off-by tags. Only humans can legally certify the Developer Certificate of Origin (DCO).” Through the Signed-off-by line, the human contributor certifies the origin of their code and takes full responsibility for it.

GitHub’s Copilot coding agent does not let the person who asked it for a PR approve that PR themselves. Since 13 March 2026, however, an administrator can switch off another safeguard: human approval before workflows triggered by the agent can run. None of these documents says how long the human gets to read.

The Carnegie Mellon researchers draw their main conclusion from thousands of published texts on code review. It is the team that decides, in review, whether an agent’s effect on the software is good or bad.

“[…] review is the control point through which a coding agent’s effect on software is decided, and […] AI does not fix the sign of that effect: the team sets it” Agarwal, Miller, Kästner and Vasilescu, Carnegie Mellon, 8 July 2026.

A quarter of an hour after closing the PR, Scherer was talking about development processes so different as to be incompatible, without saying which one was right.

On Monday morning, I’d organise code review before buying a tool

His closing sentence ended with a bracketed aside: “This may change in the future and then one could reconsider.” He was talking about his own repository. Every team can run the same check on its own code review: is it organised to review what AI writes?

If I were a tech lead or a CISO, I’d start with four steps.

  • I’d open the Git platform, the tool that hosts your repositories, and pull out two figures already sitting there. First, the share of PRs merged without human review over 90 days, then the number of lines submitted to each reviewer every week, converted into hours of reading.
  • I’d cap the size of PRs handed to a reviewer at around 200 to 400 lines, and have anything larger split up.
  • I’d write the word “security” into the PR template, along with the areas where generated code fails: logs, the display of user data, every new dependency.
  • I’d check who in your repositories can switch off human approval for agent-triggered workflows, and what write access you have given the AI code review tool.

I still have my doubts about that cap: the guideline predates coding assistants, and I have not found the measurement repeated on generated code. Nor does splitting shorten OCaml’s 26 hours, and reviewing everything will eat into the promised gains. For want of anything better, I prefer a PR that waits.

Frequently asked questions about code review and AI

Is AI-generated code secure?

In its July 2026 report, Veracode, an application security vendor, finds that AI-generated code passes 56% of security tests on average. The best model tested still fails nearly one in three tasks, and the failures cluster around XSS and log injection. Models also invent dependencies: among those from 2023 and 2024, nearly one in five recommended packages did not exist.

Can AI review code?

AI can review part of the work. According to GitHub, which is talking about its own AI code review tool, Copilot was handling more than one in five reviews on the platform in March 2026. On the HELM leaderboard, when two models are wrong, they pick the same wrong answer about 60% of the time on average. An automated reviewer can also be attacked, as CamoLeak and the CodeRabbit vulnerability have shown.

What are code review best practices for AI-generated code?

The familiar rules of thumb predate AI. The study carried out at Cisco and published by SmartBear advises reviewing no more than 200 to 400 lines at a time, at a rate below 500 lines an hour. Explicitly asking reviewers to look at security makes them eight times more likely to find the flaw. Also check who can switch off human approval for agent-triggered workflows, something GitHub has let administrators do since 13 March 2026.

Sources