All articles

Blog

Best AI for coding: none of them writes secure code without review

Claude, GPT-5, Gemini: even the best AI for coding still lets vulnerabilities through. The scores by model, and what protects your code.

  • By Azziz ERRIME
  • 10 min read
Best AI for coding: a row of identical, unmarked vintage typewriters, with a single sheet raised and struck through in red

TL;DR

  • On security, no model lets you skip the review, and the choice of model is no substitute for a security-focused review.
  • According to Veracode, the best AI for coding in its ranking produces secure code on 72% of tasks, Claude Sonnet 4.5 on 50%.
  • I’d pick the model for the team’s productivity and put the security effort into code review.

In March 2025, a researcher scanned 1,645 apps built with AI and found 170 whose database would answer anyone. I found no record anywhere of which model had written that code. Yet it’s the first thing people ask when looking for the best AI for coding. On security, the answer tells you nothing about what happens to the code after generation.

On 20 and 21 March 2025, Matt Palmer went through projects published with Lovable, a service that generates web apps with AI. Across 1,645 projects, he counted 303 exposed endpoints in 170 of them. He notified the company on the 21st. In a later post, he described what anyone could read there: email addresses, phone numbers, payment details and third-party API keys.

In April, Lovable shipped version 2.0 with a security scanner. On 14 April, a Palantir engineer came across the same flaw independently and made it public on X. On 29 May, it was assigned a public identifier, CVE-2025-48757. Matt Palmer criticises the new scanner: it “merely checks for the existence of any RLS policy, not its correctness”.

The apps ran, the screens rendered, and the flaw only showed if someone went looking. Before comparing models, I keep that scene in mind: the useful question is also what you do with the code once it’s written.

An open database never shows up in a demo

The rule missing from these apps is called RLS, short for Row Level Security. Defined in the database, it decides which rows each user is allowed to read. Without it, the interface can show each person only their own data while the database behind it answers any query sent to it.

In practice, the attacker has nothing to break. They open the app in their browser and watch the requests going to the database. They note the address and the public key used to query it, then replay the request for the whole table, which the database returns if no rule filters it.

A flaw like this is an absence. One line of configuration is missing, and no functional test asks for it, since everything works. Code generation produces this kind of defect silently, because it breaks nothing visible. What remains is whether the top-ranked model produces fewer of them than the others, and on that point there are measurements.

The best AI for coding still fails 28% of security tasks

Veracode, a vendor of application security tools, regularly puts the leading models through the same set of programming tasks. Each one invites a known vulnerability, and Veracode checks whether the resulting code avoids it. Its update published on 18 November 2025 gives the ranking below. It is the most recent per-model ranking I have been able to read. The March 2026 measurement only gives an average.

Model Share of tasks where the generated code is secure
GPT-5 Mini 72%
GPT-5 (with reasoning) 70%
Gemini 2.5 Pro 59%
Grok 4 55%
GPT-5-chat (no reasoning) 52%
Gemini 2.5 Flash 51%
Qwen3 Coder 50%
Claude Sonnet 4.5 50%
Claude Opus 4.1 49%

The top of the table belongs to models that reason before they answer. Within the same family, GPT-5 goes from 52% without reasoning to 70% with it. Veracode explains this with an image I find apt: the reasoning steps “function like an internal code review”. The model weighs several ways of writing the function and discards the less secure ones before answering.

If you’re looking for the best AI for coding on this criterion alone, the table reads like a buying guide. Read closely, it leaves two questions open. What is the code worth in the 28% of tasks that even the leader gets wrong? And what exactly does this ranking measure, when it only looks at known vulnerabilities in tasks chosen in advance?

A model that scores better on a benchmark can introduce more serious bugs

Sonar, a vendor of code analysis tools, gave several models a series of Java exercises, then analysed the resulting code. Its report of 13 August 2025 points to a paradox. From Claude 3.7 Sonnet to Claude Sonnet 4, the benchmark score goes up. Yet the bugs the newer model introduces are more than 93% more likely to be classed as BLOCKER, the blocking category.

Veracode’s ranking doesn’t see dependencies either. An AI that generates code also suggests libraries to install, and some of them don’t exist. The study by Spracklen and colleagues, presented at USENIX Security in 2025, measures these invented names. It finds 3.59% among the packages recommended by GPT-4 Turbo, the lowest rate; 13.63% for the best open model, a DeepSeek; and 19.7% on average. And 43% of these invented names recur across ten runs. An attacker who spots a recurring invented name can publish a malicious package under it and wait for a developer to install it.

Finally, the BaxBench benchmark asks the question missing from the other two. It has eleven models write complete services, checks that they work, then attacks them: around half of the functionally correct programs remain exploitable. The Lovable flaw belongs to this family, code that does what it was asked to do and nothing that someone forgot to ask for.

One serious argument for the ranking remains: reasoning models are improving fast, so following the leader might be enough.

Two suspension bridges side by side, the more refined one with a snapped main cable

The more refined of the two gave way first.

Reasoning aside, no model has improved since July 2025

The progress of reasoning models is real, and I’m not downplaying it. Between GPT-5-chat and GPT-5 with reasoning, the gap reaches eighteen points. For a team that generates a lot of code, that is no small thing.

Yet the same report describes a market standing still: “Models from Anthropic, Google, Qwen, and xAI released between July and October 2025 did not show meaningful security improvements.” Those vendors’ models stay clustered between 50 and 59%, the range already seen over the summer. Veracode’s next measurement, published on 24 March 2026, puts the share of tasks yielding secure code at 55%, with figures that “have barely budged” since the first 2025 report.

I read this as a hint, not proof: the only clear gain comes from a step that Veracode likens to a review. No study I have read measures what a human review adds to generated code.

Then there’s the question of who reviews, and how.

A security instruction changes what the reviewer sees

The obvious temptation is to have one AI review another AI’s code. Kim and colleagues measured how far models get things wrong together, in work presented at ICML in 2025. On a classification dataset, when two models are both wrong, they make the same mistake in around 60% of cases. That figure only holds for that dataset. It’s enough to make me doubt that a second model would catch what the first one missed.

Human review has its own weak spot, and it is easy to fix. Explicitly asking a reviewer to look at security makes them eight times more likely to find the flaw, according to Braz and colleagues, on code written by humans.

In practice, a security-focused review of AI-generated code looks for what is missing before what is written. An access rule on every table, validation on every input, a dependency that really exists and is the one you think it is. In an earlier article, I explained why code review can no longer keep pace with AI.

Vault door left ajar, the lock in place but the bolt retracted

The lock is there. Nobody checked that it was locked.

Choosing the best AI for coding from a ranking is like checking that a rule exists

According to Matt Palmer, Lovable’s scanner checked that a protection rule existed, without checking that it protected anything. Picking the best AI for coding from a ranking, then relying on that choice, takes the same shortcut. You confirm that a safeguard is present without measuring what it does for your code.

I’d choose the model based on the team’s productivity, its languages and where your data goes, and put the code security effort elsewhere:

  • a line in the pull request template that explicitly asks for a security review, since that instruction alone multiplies detection;
  • a check on every dependency a model adds: does the package exist, and is it the right one;
  • a review that starts with what is missing, access rules and validation, before what is written;
  • a fresh look at the ranking every quarter, without turning it into a policy.

That’s also what I’m measuring right now: what developers spot, or let through, when they review code written by an AI. Models change faster than review habits.

I do have one reservation. Reasoning models gained eighteen points in Veracode’s tests, and nothing rules out the next generation making review less necessary. I’ll change my mind when an independent measurement shows it. Until then, I review.

Frequently asked questions about the best AI for coding

What is the best AI for coding?

On security, none stands out enough to let you skip the review. In the measurement published by Veracode on 18 November 2025, GPT-5 Mini produces secure code on 72% of tasks and GPT-5 on 70%. Claude Sonnet 4.5 and Qwen3 Coder sit at 50%. For productivity, the best AI for coding depends on your languages and your tools: try it on your own code.

Is Claude or ChatGPT better for coding?

In the Claude vs ChatGPT match-up, the Veracode measurement gives GPT-5 with reasoning secure code on 70% of tasks and Claude Sonnet 4.5 on 50%. Without reasoning, GPT-5-chat drops to 52%, so the gap owes more to the mode than to the brand. Both families need a security-focused review.

Is AI-generated code secure?

Not by default. According to the Veracode measurement published on 24 March 2026, only 55% of generation tasks yield secure code, a rate that has barely moved since 2025. The BaxBench benchmark finds that around half of functionally correct programs remain exploitable. AI-generated code needs a security-focused code review, just like human code.

Is Copilot secure?

The study by Pearce and colleagues, published in 2021, found around 40% of 1,689 programs generated by Copilot across 89 scenarios to be vulnerable. A more recent replication, run with newer versions of the tool, measured 27.25%. The progress is real, and more than one program in four is still vulnerable in these scenarios.

Sources