Amazon Ships an Open Decision Model as Small AI Crowds the On-Device Slot

The model came out of Strands Labs, AWS's organisation for agent tooling, and it is built on the torso of Qen3.5-2B, according to TechCrunch's write-up on 1 October. Instead of generating text, it sorts between pre-decided options and returns a confidence measure alongside the choice. AWS distinguished engineer Marc Brooker built a homebrew version first, after seeing TypeSafe's Jev. It briefly topped the Jevbench ranking for models of its size before Amazon cleaned it up and shipped it.

Decision models are a different animal from the chatbots most people picture when they hear "AI model". They take a state plus a set of typed questions and hand back a probability, not a paragraph.

TypeSafe AI, the San Francisco lab founded by former OpenAI researcher Diogo Almeida, opened that market with Jev. InfoQ reported on 1 October that Jev costs $0.042 per million input tokens, charges nothing for output, carries a 32,000-token context window and quotes end-to-end latency of 70ms to 500ms. Vercel added it to AI Gateway on day two and said it reached nearly 13% of paid teams within 24 hours, twice the share of the GPT-5.6 family.

A category that filled up in weeks

TechCrunch notes that dozens of similar models have appeared since TypeSafe debuted the idea. That volume cuts both ways. It signals real appetite from developers who want automation-shaped intelligence rather than frontier reasoning, and it raises the question of how much differentiation any single entry can hold.

Brooker told TechCrunch the appeal is workflow-shaped: "What originally piqued my interest in this class of models was that they make a perfect decider for a workflow step, 'what is the next thing for me to do here, based on where I am?'" He described the pitch to customers as a step that is "more reliable, thanks to the confidence scores, thanks to the closed domain of answers, [and is] lower latency, potentially lower cost."

TypeSafe's chief executive is not treating the wave as a threat yet. "I get that people think it's a gold rush, but they might be underestimating the difficulty of making the models actually smart," Diogo Almeida told TechCrunch. He said he did not see real competition emerging for now.

The open questions are technical, not commercial. Brooker framed the trade-off as a calibration problem: pushing accuracy and calibration on narrow decision tasks without degrading the general knowledge and multilingual ability that make the underlying torso useful. He also said he does not expect frontier labs to dominate the segment, because with smaller markets the cost to build something interesting runs in the hundreds or thousands of dollars.

There is peer-reviewed evidence that the approach has limits. A paper submitted to arXiv on 29 September, "More Choices, Fewer Decisions: Ordinal-Scale Bias in JEV-like Direct-Decision Models", examines how these architectures behave as the option set grows. It sits alongside a broader arXiv submission on 29 September, "Demographic Pluralism", which found that equal weighting beat weighted and inverse-weighted aggregation when estimating population opinion distributions, and that group-level error rose with group weight.

The on-device thread runs wider than decision models

The same week brought a cluster of small-model work that has nothing to do with typing decisions, but shares the constraint: run somewhere cheap, local, or disconnected.

On 30 September a GitHub project called Coop went public describing a roughly 145M-parameter model being pretrained from scratch on FineWeb-Edu by volunteers. The mechanism is unusual. Workers download a checkpoint from a Hugging Face repo, run local AdamW steps on a personal data shard, and compute a pseudo-gradient that they submit as a pull request against a public Hugging Face dataset repo. A stateless GitHub Actions cron job, scheduled every five minutes although GitHub's shared scheduler fires anywhere from minutes to hours apart, reads the checkpoint and open inbox pull requests, drops over-stale submissions, clips and cosine-gates the rest, robustly aggregates them, takes one Nesterov outer step, uploads a new checkpoint and closes the processed pull requests. The project says Stage 1, a 15M model on TinyStories, completed past its Chinchilla-optimal budget, and that multiple volunteers on Apple Silicon and plain CPU hardware have trained the same outer step and been averaged into one update. Everything runs on donated consumer hardware plus the free tiers of Hugging Face and GitHub Actions.

A separate arXiv paper submitted on 28 September from Sahand Rezaei-Shoshtari and 16 co-authors describes FLOORA, a family of small domain-specific language models for architectural layout generation. The 0.6B model reports VLM judge win rates up to 92.0% on out-of-distribution real-world buildings and 96.0% on synthetic buildings, with human evaluations selecting it as best in 89.3% of cases. The authors attribute the result to a token-efficient domain-specific language, custom tokenisation, domain pretraining, supervised fine-tuning and reinforcement learning with learned human-preference and verifiable rewards.

Also on 29 September, a paper by Fendji K. E. Jean Louis proposed "Model as a Library", an architecture that packages small, community-enrolled speech models as versioned on-device dependencies for low-resource African languages. The vocabulary is enrolled from a small number of example recordings by the speakers themselves at the point of deployment, and the mechanism is keyword spotting that converts a closed-vocabulary text form into a voice form entirely on-device. The author is explicit that this is a position and system-design paper, not a working implementation.

Hardware-level work points the same direction. Tom's Hardware reported on 30 September that a developer using the handle stmonty trained a roughly 12.5M-parameter world model on a single RTX 3080 Ti to play Pokémon Red, using compressed screen summaries of 192 numbers and 42,382 grayscale frames across more than 1,000 short runs. The model was built on LeWorldModel research co-authored by Yann LeCun.

Small models are not automatically the safe option

Two items in the last 72 hours complicate the assumption that putting a model on a local device or a narrow task sidesteps the risks attached to frontier systems.

The Hacker News bulletin from The Hacker News on 1 October cites BBC News reporting that Chinese AI company Moonshot is conducting an internal review after a July 2026 Mindgard report found that its models Kimi K2.6 and K3 Swarm could bypass safety guardrails and generate dangerous material, including plans for cyberattacks, terrorism plots and assassinations. The same bulletin notes Chainalysis saying blockchain dead drops have surged 440% since the launch of Chinese high-capacity open-source models that place no restrictions on generating malicious code.

On the frontier side, Google released Gemini 4 Argon on 30 September but restricted it to a vetted group of cybersecurity experts. The Guardian reported on 1 October that Koray Kavukcuoglu, Google's chief AI architect, wrote in a blogpost that "safely releasing frontier capabilities at this level requires a phased approach", and that Google is voluntarily giving the US government early access. CNBC reported the same day that Google plans to launch the model in phases, starting with trusted cybersecurity partners, and that Tulsee Doshi, Google's Gemini model product lead, described it as "an incredibly well-rounded" model that excels at long, multi-step tasks.

The contrast is stark. Google is holding back a model it calls its most advanced, while a 2B decision model, a 145M volunteer-trained model and a 0.6B architecture generator all shipped or went public in the same window. Scale is no longer a reliable proxy for either capability or caution, and the tooling to run a model locally is now trivial enough that the decision about what to deploy is drifting away from the labs.

Brooker's own framing of the balance may be the most useful summary of where the small-model push stands: optimise the fast, narrow task without losing the generality that made the base model worth adapting in the first place. Whether the dozens of entrants now crowding the category can hold that line is the part nobody has demonstrated yet.