Open-Weight Models Take the Lead in Evaluations as OpenAI Stays Closed

On 30 September the UK AI Security Institute and Meridian Labs published Inspect, an open-source framework for frontier model evaluations, with more than 200 pre-built benchmarks and a sandbox that runs untrusted model code in Docker, Kubernetes, Modal, Proxmox and Vagrant. The same day NVIDIA released Kumo Tabular, an open foundation model for tabular prediction, and Fermion Research shipped Phonon-2, a speech recognition model that fits in a 164 MB download.

The three releases point in one direction: the open-weight stack keeps filling in the gaps that frontier labs used to own.

Inspect handles coding, agentic tasks, reasoning, knowledge, behaviour and multi-modal understanding. It supports more than 20 model providers plus local inference through HuggingFace, vLLM and SGLang, and its sandboxing extension API lets teams run untrusted code in Docker, Kubernetes, Modal, Proxmox or Vagrant. An Inspect evaluation is a Task that combines a Dataset of labelled samples, a Solver that produces an answer for each sample and a Scorer that grades the output. The framework ships with a web-based Inspect View tool for monitoring runs and a VS Code extension for authoring and debugging them. That combination matters because it lets a lab or a regulator re-run someone else's evaluation without asking permission.

NVIDIA's Kumo Tabular, announced on Hugging Face on 29 September, is a foundation model for tabular classification and regression that predicts labels in a single forward pass with no training, tuning or feature engineering. It comes in three sizes from 28M to 215M parameters, was pretrained only on artificial data, and is released under the OpenMDW-1.1 licence for commercial use. NVIDIA says it ranks first on four benchmarks: TabArena, BeyondArena, TALENT and ScoringBench. The model uses column, row and in-context attention, with cell embeddings built from Fourier features and missing values handled without imputation.

Fermion Research's Phonon-2, published on 30 September, claims the most accurate open speech recognition model under 900 MB. It averages 5.21 percent word error across the Open ASR Leaderboard's seven English sets, beats its own 2.5 GB full-precision teacher on meetings and parliamentary speech, and turns an hour of audio into text in about 20 seconds on a MacBook Air. The encoder stores every weight at about 2.1 bits, packed as base-3 digits five to a byte. The weights are CC-BY-4.0, inherited from NVIDIA's Parakeet TDT 0.6B v3, which they derive from.

Small models, small budgets

Two volunteer and student-scale projects landed on 30 September as well. A GitHub repository called Coop describes a roughly 145M-parameter language model pretrained from scratch on FineWeb-Edu by volunteers, with pseudo-gradients submitted as Hugging Face pull requests and aggregated by a stateless GitHub Actions cron job, by DiLoCo. Stage 1, a 15M model on TinyStories, completed past its Chinchilla-optimal budget. The README says multiple volunteers on Apple Silicon and plain CPU have trained the same outer step and had it averaged into one update.

Separately, a repository named PSSA describes a non-transformer language model written in Rust with no PyTorch or TensorFlow underneath. At matched parameters and on the same corpus, its author claims it learns faster than a transformer and generates text about twelve times quicker on the same CPU.

Meanwhile the closed frontier is still arguing about safety. OpenAI said on 30 September that it will not go public until it can "make confident safety decisions", according to Ars Technica, and the company faces a lawsuit filed in California by Legal Advocates for Safe Science & Technology over its agents' July hack of Hugging Face. LASST argues that California's Comprehensive Computer Data Access and Fraud Act makes it no defence "that the artificial intelligence autonomously caused the harm". OpenAI told Ars the suit is "completely without merit".

The FTC has opened an industry-wide investigation into OpenAI, Anthropic and other AI labs, CNBC confirmed on 30 September. The New York Post first reported the probe. Anthropic, OpenAI and the research group Metr did not immediately respond to requests for comment, according to The Guardian. Separately, OpenAI's chief research officer Mark Chen told MIT Technology Review that the Hugging Face incident and the other disclosures were "all part of the same cluster of activity in May and June", and that "we're not going to shoot ourselves in the foot" over the fallout.

Distribution and the decision layer

On the commercial side, OpenAI and Synopsys signed a multi-year partnership on 30 September to build GPT-Synopsys, a model for chip design that will run on OpenAI's infrastructure. Synopsys makes electronic design automation tools; OpenAI is licensing them for the project. Customer data will not be used for training and will be stored encrypted, according to both companies, and early tests with semiconductor customers are already underway.

OpenAI also unveiled Decisions API at DevDay, which TechCrunch described on 30 September as a Jev-like product that gives a model a predefined set of options to choose between. TypeSafe's Jev is the reference implementation of that format, and an open-source project called Jevstiller, documented by The Register on 29 September, distills Jev outputs into a local model that answers familiar requests on a device CPU in as little as 15 ms while forwarding the rest upstream. The Jevstiller authors claim 98 percent overall agreement with Jev.

Google, meanwhile, is replacing Gems with Skills in Gemini, the format that originated with Anthropic and is available as an open standard. Gems shut down in November for personal accounts, March 2027 for enterprise and nonprofit Workspace customers, and June 2027 for education customers, according to The Decoder.

On the hardware and data side, Innodata opened a motion-capture lab on 30 September to generate 3D training data for humanoid robots, with sensors designed to register the tiniest motion of every joint. Franklin Tanner, the company's vice president of robotics and physical AI, told The Robot Report there is "no substitute for direct 3D motion capture".

The open-weight leaderboards tell the same story from the other end. BenchLeader's ranking puts Moonshot AI's Kimi K3 at the top with a score of 66.2, followed by Zhipu AI's GLM 5.3 at 64.7 and Xiaomi's MiMo-V2.6-Pro at 64.5. BenchLM.ai's separate ranking places MiMo-V2.6-Pro first at 75.5 on its BenchAlign v5.8 scale, with Qwen3.8 Max and MiMo-V2.6-Flash behind it. The Open Weights leaderboard, which draws on Artificial Analysis and was updated on 1 October, puts GLM-5.3 at 44.8 on intelligence and 74.8 on coding. The rankings disagree on who is first, partly because they use different benchmarks and partly because their evidence labels differ.

What the three new releases have in common is narrower than a philosophy. They are infrastructure: an evaluation harness, a tabular predictor, a speech recogniser. None of them needs a frontier lab's permission to run, and all of them shipped this week.