AI weather models are getting better at forecasting, but the hype runs ahead of the evidence

The dossier's most recent dated weather material is not a forecast at all. A study reported by the South China Morning Post on 26 October found Chinese video AI models beat US rivals on visual quality while lagging on safeguards, which is a useful reminder that "AI model quality" is a bundle of separate claims, not one score. Nothing in the dossier dates a new global weather-model release inside the last 72 hours.

The announcements that do exist cluster in late October and early November, and most come from vendors.

Envision said its Tianji weather foundation model topped the European Centre for Medium-Range Weather Forecasts' AI Weather Quest, according to a Kyodo News wire item carried on 2 November. Germany launched a global weather model powered by AI, per a Radar.am report dated 1 November. Vaisala completed its acquisition of AI weather company Atmo, The Manila Times reported on 29 October. Each is a company or government statement about its own system, and none of the three gives an independent error score in the dossier text.

What the benchmarks actually measure

Where the dossier does carry numbers, they come from research rather than marketing. A paper submitted to arXiv on 29 October, FABench, builds a backtesting environment for forecasting agents and poses 287 questions across nine domains in its first release. Evaluating agents built on eight recent models, the authors found retrieval improved the Brier score of all eight in every retrieval setting and lifted the leading agents to market consensus on 54 questions linked to Polymarket, with the best three scoring slightly above it. That is a careful, bounded result. It says retrieval helps, not that AI beats physics.

Two other papers sharpen the picture. QUAL, submitted on 30 October, reports that aligning quantiles across a pool of time-series foundation models cut average member MASE by 7.4% and CRPS by 9.8% on GIFT-Eval, with no training or model modification. And a paper on web world models, also submitted on 29 October, found that in a matched human evaluation of 3,000 predictions from ten frozen open models, 86.1% were judged semantically plausible, yet several models showed asymmetric authorisation behaviour, combining low false-safe execution with poor recall of cases that should have triggered a clarification request. Plausible is not the same as safe, and the authors say so.

"Nine in 10 agent workflow steps are routine, and open-weight models handle them at parity for a quarter of the price of the cheapest closed model," Gartner's Mark Petty and Raymond Paquet wrote in a document titled "Open-Weight Models Will Run 90% of Enterprise Agentic Workflow Steps," as quoted by The Register on 3 November.

That Gartner note is about agents, not weather, but it matters for anyone budgeting a forecast pipeline. The analysts put the frontier-model saving at as little as a twentieth in the best case, then admit current open models "can be more verbose and retry more often, so the saving per task against a frontier model is closer to five or six times." The Register published that on 3 November. If cheap open models do most of the routine numerical work, the expensive ones get reserved for the hard cases, and weather forecasting is full of hard cases.

The infrastructure underneath

Forecasting quality also depends on what goes into the model. On 19 October, CNN reported that Trump administration cuts to satellites will set back US weather forecasting by a decade, a story the outlet updated on 27 October. That is the constraint no benchmark addresses: an AI model trained on observations cannot outperform the observations it is fed, and a decade of degraded satellite input is not something a better architecture fixes.

Nvidia, meanwhile, is pushing its Earth-2 open models into European deployments, according to a company blog post dated 22 October, and separately published a post on 22 October about European financial services organisations adopting open models to build proprietary intelligence. Both are vendor blogs, so treat the framing accordingly, but they show where the compute and the distribution are heading. IBM Research published its own take on ensemble forecasting with Granite time-series models on Hugging Face on 22 October, which lines up with the QUAL paper's finding that combining models beats picking one.

The sceptical case is not hard to find. Nathan Lambert asked on Interconnects on 30 October whether he had been wrong about open models and cyber risks, a question worth reading next to Bruce Schneier's 27 October post on hiding capabilities in AI models and his 26 October post on bias in the Qwen3-Coder model. Capability claims and safety claims tend to arrive from the same press offices. The dossier also carries a 20 October Interconnects roundup on DeepSeek V4.1 Flash 1017 and new regulatory controls in the US and China, which is the policy backdrop for all of this.

For weather specifically, the honest summary is short. Several organisations say their AI models beat established benchmarks. One peer-reviewed-style benchmark, FABench, shows retrieval helps forecasting agents reach market consensus. One safety paper shows plausible predictions do not guarantee correct decisions. Nobody in this dossier publishes an independent, like-for-like verification of the weather claims themselves. Until someone does, the announcements are inputs to a story, not the conclusion of one.