What open weights actually get you, and what they do not

Mozilla published version 1.1 of its State of Open Source AI report on 15 September, using data current to 1 September. The headline number is a four-month gap between the best open-weight models and closed frontier offerings. The report counts 16 notable open releases. None of them ships the data recipe the Open Source Initiative requires.

That last clause matters more than the four months.

The Register's Steven J. Vaughan-Nichols put the distinction plainly on 15 September. Publishing model files to Hugging Face and letting developers run them on their own GPUs is open distribution, not open source. The Open Source Initiative, which owns the Open Source Definition, defines open weights as "the final weights and biases of a trained neural network." Those values determine how a model reads a prompt and produces an answer. They do not tell you what went into it.

What the download contains

A weight file is the output of training, not the training itself. Without the dataset or detailed documentation, an outsider cannot establish which sources were used, what copyrighted or private material was swept in, how data was filtered, which languages and communities were left thin, whether benchmark data leaked into the run, or what alignment work happened after pretraining. James Landay, director of the Stanford Institute for Human-Centered AI, told The Register: "Open weights are progress. You can download the model, run it on your own machine, keep it out of someone else's data pipeline. But you still can't see how the thing was built, what it was trained on, or why it behaves the way it does. That's not an open model. That's open distribution."

The practical upside is real, and it is worth stating without sneering. Self-hosting gives you control over data, privacy, costs, API changes and vendor lock-in. It has also produced a large ecosystem of runtimes, inference providers and fine-tuning tools. Superwhisper's s1-mini shows how small that ecosystem can get. It is a 0.6B-parameter text normalizer, released under Apache 2.0 with a naming clause: 596M unique parameters, a 462 MiB quantized build, fine-tuned from Qwen/Qwen3-0.6B, and it runs on a laptop CPU. You can pin a release with a revision tag. You still cannot see the corpus.

The definition fight

The Open Source Initiative released its Open Source AI Definition, OSAID 1.0, in October 2024, and acknowledged at the time that it would keep evolving. It requires model parameters, including weights, to be available under OSI-approved terms, but does not prescribe a legal mechanism. Luca Antiga, CTO of Lightning AI and a PyTorch contributor, argued that the treatment of weights leaves what he called a gaping hole. Licences, he said, will be less effective at determining whether OSI-licensed systems can be adopted in practice.

Bruce Perens, author of the original Open Source Definition, denounced OSAID in 2024 and later said: "It's not Open Source! … It's unfortunate that the Open Source Initiative itself is now involved in Openwashing." Bradley Kuhn of the Software Freedom Conservancy and Red Hat's Richard Fontana have called for OSAID to be repealed. The OSI acted too quickly, they wrote, and the rift damaged its reputation and influence.

Meanwhile the Linux Foundation's Mike Dolan submitted the Open Model, Data, and Weights licence, known as OpenMDW, to the OSI. It has existed since 2025 and lists contributors from Amazon, Meta, IBM, Microsoft and Nvidia. Its approach is to define separate terms for architecture, training data and weights, bringing everything a licensor supplies under one agreement. The submission has drawn objections on OSI's licence review mailing list. Stefano Maffulli, the OSI's former executive director, said he keeps getting the impression the review is tainted by ideological bias against big tech.

Open weights answer 'Can I run this?' Open source answers 'Can I trust this, improve it, and build the next thing on top of it?'

That line is Landay's, again via The Register, and it is the cleanest summary of the dispute. It also explains why the Mozilla report's benchmark work should be read with care. The four-month figure comes from METR task-horizon data, which scores models by how long a task, in human working time, they complete half the time. Mozilla's fit puts closed models at tasks taking experts 8 to 12 hours, with open models reaching that roughly four months later. Mozilla computes open capability doubling every 3.9 months against 5.5 for closed.

Price is where open weights win clearly. Mozilla found the best open model trailed the closed leader by three points on the Artificial Analysis Intelligence Index at 60% of the price, and sat two points behind Claude Fable 5 at 30%. On vals.ai's Terminal-Bench 2.1, Z.ai's GLM-5.2 scored within a point of Claude Opus 4.7 and about four points behind Opus 4.8, at less than one-fifth the cost per test. On OpenRouter, Mozilla counted eight of the top ten models by August token volume as open weights, seven of them Chinese-built. Closed providers nonetheless took 96% of model-layer revenue on OpenRouter from May to September 2025, according to the Linux Foundation.

Hardware is the caveat. Mozilla's own chart puts the best open model that fits one server at 52.6 and the best on one GPU at 40, drops of 10 and 23 points from the top, wider than four months suggests. Kimi K3's native MXFP4 checkpoint runs about 1.56TB across 96 shards, and Mozilla's serving configuration lists 64 or more accelerators. Thinking Machines' Inkling-Small, under Apache 2.0, fits one B300 at a 180GB floor.

The data stops at 1 September. Artificial Analysis has since moved to index v4.3 with a different evaluation set, and Mozilla's own chart caption reads: "the gap resets every release cycle." The label will keep outrunning the definition.