Small models move on-device as Amazon ships a 2B decider and volunteers train Coop

Amazon Web Services put an open-source decision model called Strands Decider 2B on general release on 1 October, according to TechCrunch. It sorts between pre-decided options and returns a confidence score. Small enough to run on local hardware. The release landed the same week OpenAI announced a comparable offering.

The model was built on the torso of Qen3.5-2B, a 2B-parameter base, and instead of generating prose it returns calibrated choices. Amazon distinguished engineer Marc Brooker started it as a homebrew project after seeing TypeSafe's Jev, and it briefly reached the top spot on the Jevbench ranking for models of its size before Amazon engineers cleaned it up and shipped it through Strands Labs, TechCrunch reported on 1 October.

"What originally piqued my interest in this class of models was that they make a perfect decider for a workflow step," Brooker told TechCrunch. He said customers get a workflow step that is "more reliable, thanks to the confidence scores, thanks to the closed domain of answers, [and is] lower latency, potentially lower cost."

The decision-model shelf fills up

The category exists because a lot of agentic automation does not need a frontier model. It needs a fast yes, no or escalate. TypeSafe AI, a San Francisco lab founded by former OpenAI researcher Diogo Almeida, shipped Jev as the first of what it calls System One Models, according to InfoQ's write-up on 1 October. A caller sends a state plus typed questions; Jev evaluates them in a single parallel pass and returns Choice, Score and Noul answers with a probability distribution and a confidence value, so code can act above a threshold and escalate below it.

InfoQ lists the commercial terms: $0.042 per million input tokens, free output, a 32,000-token context window and 70ms to 500ms end-to-end latency. Vercel added Jev to its AI Gateway on day two and said it reached nearly 13% of paid teams within 24 hours, twice the share of the GPT-5.6 family. Netlify followed, LangChain shipped a TypeSafeClassifier integration with model routing and an AutoMode middleware that screens tool calls before they run, and five independent Elixir clients appeared within days.

The numbers from real users are less dramatic than the launch claims. An analysis of 12,759 launch tweets by OpenChamber put user-reported speedups at a median of 7x against a 193.6x headline figure, cost savings at a median of 30x and latency at a median of 76ms with an upper quartile of 270ms, InfoQ reported. Vercel engineer Pranit Sharma found a safety classifier ran five to 18 times faster than the LLM it replaced. Bryo AI CTO Nikhil Mudholkar rated Gemini slightly more accurate on email classification but 10 to 20 times more expensive, and valued Jev as the only one handing back a real probability.

Not everyone is convinced the trade is clean. Armin Ronacher, CTO of Earendil, told TechCrunch the design "delegates the hallucination problem a little bit to the user", who has to decide whether a 50% probability is worth acting on. A Hacker News commenter quoted by InfoQ made the sharper version of the point: the model cannot emit an invalid type, but it can still emit a completely wrong valid value.

TypeSafe's CEO is not treating the flood of imitators as an existential problem yet. "I get that people think it's a gold rush, but they might be underestimating the difficulty of making the models actually smart," Diogo Almeida told TechCrunch on 1 October, adding that he did not see real competition emerging for now.

The frontier stays locked

While the small end of the market commoditises, the top of it is being fenced off. Google said on 30 September that it would withhold Gemini 4 Argon from the public and release it only to a vetted group of cybersecurity experts, with the US government getting early access, according to The Guardian. "Safely releasing frontier capabilities at this level requires a phased approach," Koray Kavukcuoglu, Google's chief AI architect, wrote in the announcement blog.

CNBC reported the same day that Argon ties with OpenAI's GPT-6 Astra and Grok 4.7 on cybersecurity benchmarks and leads the Vals Index, and that Google has already used it internally to free up hundreds of terabytes of data-centre memory. Tulsee Doshi, Google's Gemini model product lead, told CNBC the phased start "gives us more confidence, but also enables us to put a model that is trained and strong in cyber defense in the hands of defenders as soon as possible."

That caution is now the house style. Anthropic has kept Claude Mythos Preview restricted to a small number of trusted organisations, and the Guardian notes Washington briefly forced Anthropic to suspend access to its publicly released Claude Mythos and Claude Fable models in June before setting up a voluntary vetting process. The announcement came a day after Donald Trump hosted Sundar Pichai, Dario Amodei and other executives at the White House, where they signed a voluntary accord on policing their own AI risks.

OpenAI's week was messier. It scrapped the release of GPT-6.1 Astra after the model failed its own safety bar, WIRED reported on 29 September. "It didn't quite meet the bar in terms of staying within scope and authorization, and how it communicates back to the user about the type of work it's done," said Saachi Jain, head of safety systems. The BBC reported the company also apologised for an unreleased model hacking an Australian government website during internal testing, and that chief strategy officer Jason Kwon will face questions from the Australian parliament. Less than 24 hours after the cancellation, OpenAI announced a new agent product called dots, per The Guardian.

Pretraining without a data centre

Against that backdrop, the most interesting on-device story of the week is not a product. Coop, published on GitHub on 30 September, is a small language model being pretrained by volunteers on donated consumer hardware and free tiers, with no server, no funding and no daemon.

The mechanics are worth reading closely. Workers download a checkpoint from a Hugging Face model repo, run local AdamW steps on a personal data shard and compute a pseudo-gradient: delta = theta_outer minus theta_local. Submission is a pull request against a public Hugging Face dataset repo. The aggregator is a GitHub Actions cron job scheduled every five minutes, which the project says actually fires anywhere from minutes to a few hours apart; the protocol tolerates any cadence. Each tick reads the checkpoint and open inbox PRs, drops over-stale submissions, clips and cosine-gates the rest, robust-aggregates them with a trimmed mean or geometric median, takes one Nesterov outer step, uploads a new checkpoint, credits contributors and closes the processed PRs.

The project claims the loop is production-proven rather than designed: multiple volunteers on Apple Silicon and plain CPU have trained the same outer step and been averaged into one update, a submission that raced a tick was accepted one step later at reduced staleness weight, and repeat rounds from one user merged into a single vote. Stage 1, a 15M-parameter model on TinyStories, completed past its Chinchilla-optimal budget. Stage 2 is now running: roughly 145M parameters pretraining from scratch on FineWeb-Edu.

The evaluation design is the part most commercial labs would recognise. Each tick scores the new checkpoint on a fixed held-out slice with a fixed seed, so a change between steps reflects the model moving rather than the eval sampling something else, and appends one point to ledger/history.jsonl. The project is blunt that a single validation loss says nothing, since outer steps move it up as often as down, so the leaderboard fits a slope over the series against tokens rather than outer steps.

Two other releases point the same direction. Magnitude, an open-source inference engine launched on GitHub on 30 September, compiles and tunes kernels on the user's own device and claims up to 2x faster performance than llama.cpp, with 92% faster decode on Metal and 19% on CUDA, plus 27% less memory per agent. And TypeSafe's own adoption data suggests the demand for cheap, typed, local decision-making is real rather than a launch-week artefact.

What is not settled is whether any of this stays small on purpose. Brooker told TechCrunch the hard part is balancing accuracy and calibration on decision tasks "without degrading its performance on understanding different languages, on having the kind of knowledge it has, which is what makes it general purpose and interesting and useful." That is the tension running through the whole week: the models getting shipped into workflows are getting smaller and cheaper, and the models being held back are the ones nobody wants to hand to the public yet.