Gemini 4 Carbon leads AI benchmark by one point as Stanford finds LLM claims expire

A one-point lead on the intelligence index

The gap at the top is one point. Gemini 4 Carbon scored 59 on Intelligence Index V5. That is one ahead of Claude Opus 5.5 and two clear of OpenAI's GPT-6 Astra.

Buyers who pick models off a leaderboard now have a one-point gap to interpret. They do not have a durable lead. Google has not said when wider access to Gemini 4 Carbon will arrive.

Tom's Hardware reported that the model ranked first in four of the nine evaluations in the suite. Its cost per task on the Intelligence Index test came in at $7.76. The Hacker News reported that Gemini 4 Carbon led benchmarks for finding, exploiting and patching vulnerabilities. Ars Technica reported that Google announced the model with a staged release that could take a while.

Google says Carbon gained 89 points on WebDev Arena. It now ranks fourth there.

Why the lead is fragile

Benchmarks age. Owen Queen and James Zou, both at Stanford, tested 50 widely cited claims about large language models against 20 models.

They found that 44% of those claims had already expired. Among the 22 findings with an estimated expiration date, the average lifespan was 15.7 months. After one year, 16% had expired. After two, the figure climbed to 39%. Eight developers released the models between July 2024 and September 2026. Claims about what models cannot do failed fastest. Five of nine reasoning-related findings were classed as expired. None had held up or grown stronger.

Results also vary inside a single release window. Among models shipped between July and September 2026, expired findings ranged from 11 for DeepSeek V4 Pro to 30 for GPT-6.1 Sol.

The fabricated-citation test is the clearest case. GPT-3.5 made up 55% of its references in a 2023 study by Walters and Wilder. On the same prompts, GPT-6.1 Sol fabricated 0% and Claude Sonnet 5.5 came in at 1%. Qwen3.7-Flash still fabricated 69%.

Queen and Zou draw a narrow conclusion. Findings about LLMs should be presented as observations about specific models at a specific point, not as general truths.

Benchmarks do not settle programming or knowledge work

A one-point lead on a composite index is not a lead on every task.

Ars Technica reported that Gemini 4 Carbon beats GPT-6 Astra, Claude Opus 5.5 and Claude Fable 5.1 in almost every Google test, but only by a few points. Artificial Analysis says the model is not as good as Claude Code with Claude Opus 5.5 at programming. It also falls behind Grok 4.7, Muse Wonder and Kimi K3.1 Pro on knowledge work. On AA-Omniscience, which tracks knowledge and hallucination, Gemini 4 Carbon scored 54. That is the best result in the comparison. It is still far from consistent recall on a scale running from -100 to +100.

Price is the other half of the comparison. At $7.76 per task, Gemini 4 Carbon undercuts Claude Opus 5.5 and Sonnet 5.5, both max with fallback.

It costs about six times more than GPT-6.1 Sol (max). It costs about 13 times more than the open-weight DeepSeek V4.1 Flash 1017, which scores 32 on the Intelligence Index. Claude Fable 5.1 sits at $12.99 per task. At lower price points, Chinese open-weight models like GLM 5.3-Flash and DeepSeek V4 Pro take over the Pareto frontier, according to Arena.ai figures cited by Tom's Hardware.

Access, security claims and what is still unknown

Access remains limited. Google is offering Gemini 4 Carbon to cyber defenders first through its Fairwind Program. Wider availability will come sometime after.

The security claims are the loudest part of the pitch. Google says the model is twice as likely to generate code free of vulnerabilities as Argon. The Chrome team claims it is 10 times better at producing high-quality patches than non-Google models. Ars Technica reported that Google claims a "significant leap" in vulnerability detection, validation and remediation. Google also talked up its collaboration with the U.S. government on model screening and pre-release testing.

For now the leaderboard has a new name at the top, by a single point.