Gemini 4 Carbon leads a new AI index by one point, while task benchmarks show wider gaps

A one-point lead at the top

It finished two points ahead of OpenAI's GPT-6 Astra and placed first in four of the index's nine evaluations.

A one-point lead decides the top of the index, not the work buyers need done. Whether that lead survives a different harness is not public.

Gemini 4 Carbon cost $7.76 per task on the index. Claude Fable 5.1 cost $12.99. Gemini cost several dollars less than either Claude Opus 5.5 or Sonnet 5.5. Google released the model last week. It gave cyber defenders first access through its Fairwind program. Wider availability will come later. Tom's Hardware also described it as the frontier model least likely to hallucinate, based on benchmark claims.

The hallucination score needs its scale. Gemini 4 Carbon took 54 on AA-Omniscience v1.1, which runs from -100 to +100. The best result on record is still far from a clean sheet.

What the index rewards

Artificial Analysis builds v5 from four blocks of equal weight: knowledge work, coding, science and general, each 25%. The coding block is 15% FrontierCode 2.0 and 10% Terminal-Bench 4.0. The general block is 15% AA-Omniscience v1.1, split into accuracy and non-hallucination, plus 10% HLE-Diamond.

Interconnects called the AA index the most prominent independent intelligence metric, with Epoch AI's Capabilities Index a distant second. It also called all such indices "massively flawed".

The same analysis found signs of strain in the v5 ordering. GPT 6.1 Terra finished above Claude Fable 5.1, for example. It noted that coding is the most proven use case, where open models are far more competitive. It was less confident about open models in knowledge work.

The same analysis argued that the era of single intelligence indices should be phased out. It expected that to take a long time.

Task-specific tests find bigger gaps

Benchmarks built around one job tell a harsher story. CloudOpsBench, which tests models on cloud infrastructure work, put Claude Fable 5.1 (max) with the claude-code harness first at a 60.6% resolution rate. GPT-6 Astra (max) followed at 58.9% and Gemini 4 Argon (high) at 57.5%.

Eighteen models appear on that table. The spread runs down to 19.1%. The same model lands at different ranks depending on the tool wrapping it. The top run burned 97.9 million tokens over 38.6 hours.

SemiAnalysis's GDPx benchmark went after economically viable work. It found the leading model scored just 12%. FrontierMat is a set of 330 problems drawn from aerospace, semiconductor, embodied intelligence and nuclear fusion research. It put the strongest general-purpose model at 53.7%. On matched problems, cross-scale questions scored 8.7 points below single-scale ones. At most one model solved half of the 330 problems. A domain-adaptation experiment in the same paper lifted an open-weight model from 36.7% to 55.4%. That matched the strongest proprietary model.

Reasoning does not equal reliability

The gap between capability and dependability shows up in clinical work too. A study in Nature Communications compared six matched pairs of reasoning models and standard models. It covered 12 medical specialties and used a benchmark drawn from real electronic health records.

The reasoning models produced more than 34% more false positive diagnoses than their standard counterparts. They also showed roughly 40% lower reproducibility, so repeated runs disagreed more often. They did make more accurate primary diagnoses and produced more clinically useful differentials.

The fix the authors tested was supervision, not a bigger model. In a validation with 15 physicians, pairing reasoning output with clinician review raised diagnostic accuracy by 20%. It cut false positives by 44% compared with the models working alone. The authors cautioned that the results came from retrospective records rather than live clinical encounters.

Open weights add pressure, and claims need checking

Closed frontier models are not the only moving part. Mistral released the weights for Mistral Large 4 on 29 October. It is a multimodal model that Mistral trained on European infrastructure. Mistral reports 41% on Terminal-Bench 4.0, 13 percentage points above the model's preview. It also cut output tokens by 30%.

The company also says Large 4 matched GLM-5.3 at 48% on the Artificial Analysis Coding Agent Index. It used 47% fewer tokens per task. Those are vendor-reported figures.

Licensing matters here. Mistral's open-weight licence allows free use and modification, but a company reselling the model as a service needs a separate commercial licence.

Meituan released LongCat-2.5 on Monday. It is a 1.6 trillion parameter model with about 48 billion parameters active per token. Meituan trained it on roughly 50,000 domestically produced Chinese accelerators. Its benchmark claims await independent tests.

Google has not given a date for wider Gemini 4 Carbon access beyond the Fairwind program.