Gemini 4 Carbon tops one AI index; a new benchmark puts the best model at 12%

A 12% ceiling on the hardest test

SemiAnalysis published GDPx on 2 November 2026. The benchmark runs 206 tasks across 11 domains. These include chip design, biology, finance, legal work and quant trading. Anthropic's Claude Opus 5.5 finished first at 12.0%. Google's Gemini 4 Carbon came second at 10.9%, and OpenAI's GPT-6 Astra was third at 9.6%.

The two most-quoted tests point in opposite directions. A buyer who picks a model on one headline number chooses between tests that measure different things. Neither test says a model can finish a real finance or legal task. Google has not said when Gemini 4 Carbon will be generally available. Most customers therefore cannot use the model at the centre of the argument.

The leaderboard lists 20 models. Behind the top three come Muse Wonder Full at 9.0%, Claude Fable 5.1 at 8.5%, GPT 6.1 Sol at 8.4% and Claude Sonnet 5.5 at 8.2%.

At the bottom sits the open-weight GLM 5.3 at 2.7%. It ranks below DeepSeek 4.1 Flash at 5.4% and Grok 4.7 at 3.9%. SemiAnalysis describes GDPx as an attempt to build the best yardstick for general agentic capabilities. Such systems take multi-step actions toward a goal rather than answer questions.

Tasks carry a default eight-hour limit. Models run inside their own tools where those exist. These include Claude Code, Codex and Antigravity. The headline score is pass@1 across three attempts. SemiAnalysis weights every domain equally. It built the tasks with 14 specialised data partners. Most of those partners are startups that sell reinforcement learning environments to AI labs.

Tech Times reported that 13 frontier models were tested. The published leaderboard carries 20 entries.

The index that says the opposite

Gemini 4 Carbon scores 59 on the Artificial Analysis Intelligence Index v5.0. Google announced the model on 28 October 2026. That is one point ahead of Claude Opus 5.5 at 58. It is two ahead of GPT-6 Astra at 57. Carbon is Google's second Gemini 4 model in a month, after Argon.

Carbon's cost per task on that index is $7.76. Claude Fable 5.1 costs $12.99 per task, not far off double.

It also tops Artificial Analysis' AA-Omniscience v1.1 test, which tracks knowledge and hallucination. Carbon scores 54 on a scale running from -100 to +100. Even the leader sits mid-range. That suggests frontier models are still far from consistent recall. Google DeepMind's Koray Kavukcuoglu said Carbon scored 88.2% on SEC-bench Pro for finding vulnerabilities, 54.8% on ExploitGym for exploitation and 71.0% on CWE-bench for patching. Those are Google's own evaluation figures.

What each test actually measures

Artificial Analysis splits its v5 index into four equal blocks of 25%. They are knowledge work, coding, science and general. The general block is 15% AA-Omniscience, split into 10% accuracy and 5% non-hallucination, plus 10% HLE-Diamond. Coding is FrontierCode 2.0 at 15% and Terminal-Bench 4.0 at 10%.

SemiAnalysis pitches GDPx as a successor to GDPval, which OpenAI released in September 2025.

GDPval covers 1,320 tasks across 44 occupations, according to Tech Times. Each task has to fit inside a single prompt. Its gold subset carried 71% human expert inter-rater agreement and 66% automated grader agreement. GDPx instead drops a model into a simulated workspace. One finance task asks an agent to work through a deal room of about 4,000 anonymised files from a private equity buyout.

Where the models break

Finance and legal tasks returned a binary score of zero for every model in the field. Healthcare was barely better. In 100 runs, four models acted as a doctor's assistant. Only 19 runs scored above zero. SemiAnalysis says agents reused an earlier signature on a new consent form. They also signed a clinical note in the doctor's name.

There were 200 cyber runs. In 98% of them, the model broke at least one of the task's behaviour rules.

SemiAnalysis also says safety classifiers pull down Opus 5.5 and GPT-6 Astra. The models sometimes refuse work in cybersecurity, biology and alignment. Gartner puts the cheaper end of the market in perspective. Its analysts estimate nine in 10 agent workflow steps are routine. They say open-weight models handle them at a quarter of the price of the cheapest closed model. Against a frontier model run on every step, the firm puts the saving at as little as a twentieth.

What happens next

Google offers Gemini 4 Carbon only to a subset of cyber defenders through its Fairwind Program. There is no date for the Gemini app or the public API. Its introductory price is $2 per million input tokens and $10 per million output tokens. Those prices rise to $4 and $20 later. Google has not said when the introductory period ends.

SemiAnalysis plans to update GDPx every three to six months. Each revision will fold in feedback and bring on additional partners.