Small models move on-device: Echo, Phonon-2, Strata and Nimble ship in one week
Fulcrum released Echo on 30 September, a writing model it says cost less than $5,000 to train and beats frontier models at style imitation. It was one of four small or local-first releases that landed the same day, alongside Phonon-2, Strata and Ollama's decision-model endpoint.
On 30 September, Fulcrum released Echo, a writing model it describes as the best at style imitation. The company says it cost less than $5,000 to train. It also says Echo beats frontier models on writing tasks from fiction to technical explanations, according to its launch page. The model is not open yet. Fulcrum says it plans to open-source it soon and is taking contact requests for scaled use.
The claim rests on a metric Fulcrum built itself. It computed standard stylometric features and used them to produce a style Elo. On that score it reports state of the art for both authors seen in training and authors held out. There are no accepted canonical metrics for writing quality, and the company says so on the page. Treat the ranking as its own scoreboard until someone independent reruns it.
Fulcrum's explanation for the low cost is elicitation. It argues the model already holds compressed knowledge of author style. Supervised fine-tuning, it says, mostly teaches the model to activate what is there. Its SFT stage seeds personas using human writing from the internet, wrapped in synthetic outlines generated by a frontier model. It then fine-tunes on the original post conditioned on that outline. A second stage applies token-level reinforcement learning against a voice metric. Fulcrum says this carries the persona gains into chat requests and fixes a reasoning cost it saw from off-policy human data.
One detail matters for anyone who uses AI detectors as a proxy for slop. Fulcrum writes that Pangram, a detector, judges Echo's writing human more often than frontier models. On slightly more detailed prompts, it says Echo is flagged as human most of the time. That is a deliberate design choice on Fulcrum's part, not an accident. It cuts against the common assumption that better models should be easier to spot.
Speech, in 164 MB
The same day, Fermion Research released Phonon-2, an English speech recognition model that ships in a 164 MB download. Fermion says it averages 5.21% word error across the seven English sets of the Open ASR Leaderboard. It says every open model scoring better is at least 5.8 times its size. The encoder stores each weight as one of five learned levels at about 2.1 bits, packed as base-3 digits five to a byte.
The headline comparison is against its own teacher, a 2.5 GB full-precision model. Fermion says Phonon-2 holds that teacher's accuracy set for set. It reaches 100.8% of its word accuracy on parliamentary speech and beats it on meetings, from a download 15 times smaller. On a MacBook Air it turns an hour of audio into text in about 20 seconds. On eight CPU cores the same hour takes 25 seconds. On one H100 a full day of audio takes 13 seconds in batches of 128.
Fermion is careful about its fast paths. It says each one ships only because its word error stays within noise of the exact path. That means a paired bootstrap over clips with 4,000 resamples giving a 95% interval that covers zero. The weights are CC-BY-4.0, the licence of NVIDIA's Parakeet TDT 0.6B v3, from which they derive. The model runs on Macs, Linux, Windows and NVIDIA GPUs, and the fermion command line installs from PyPI.
Both releases point at the same shift. Capability that used to need a datacentre is being compressed into files a laptop can hold. Phonon-2's 164 MB is smaller than most photo libraries. Echo's training budget is smaller than a month of one engineer's salary. Neither number tells you the model is good. Both tell you the barrier to shipping one has fallen.
A 125B model on a gaming PC, sort of
Strata, published to GitHub on 30 September, takes the opposite approach. It does not shrink the model; it streams it. The project offers a one-click install for Windows and Linux that runs Qwen3.8-Flash-Next on consumer hardware. It reads whatever does not fit in memory from the SSD. Its documentation lists a 2.00 bpw set at 257 GB, needing about 265 GB free disk, and a coder variant with half the experts removed at 191 GB.
The measured numbers are modest and the project does not hide it. On an RTX 5070 with 12 GB and 64 GB of system RAM, Strata reports 4 to 6 tokens per second of output. It reports about 110 tokens per second of prompt reading at 4K. An RTX 3060 with 12 GB and 46 GB of RAM reaches 9 to 11 tokens per second. A single Radeon AI PRO R9700 with 251 GB of RAM gets 21 to 25 tokens per second. Four of those cards reach 74 to 77.
Supported hardware is narrow and spelled out. The list covers NVIDIA RTX 30, 40 and 50 series on Windows and Linux, AMD Radeon AI PRO R9700 on Linux, AMD Ryzen AI Max Strix Halo on Windows, and Apple Silicon Macs with 64 GB or more, text only. The project is free and open source, needs no account, and serves an OpenAI or Anthropic style API on localhost. It is a real option for someone with a large SSD and patience. It is not a replacement for a hosted endpoint.
The decision-model stack consolidates
Ollama shipped version 0.35 on 30 September with support for decision models through a new /v1/systemone endpoint, following TypeSafe's Jev API. You send text as state plus a set of named questions, and a local model answers all of them in one request. Ollama positions it for ticket triage, model routing and content moderation. It reports Nimble 9B averaging 91 ms per decision on an M5 Max in its Pac-Man example.
Nimble is Bespoke Labs' 9B model, fine-tuned from Qwen3.5-9B, and it is the reference implementation for the format. Its repository documents the timeline. TypeSafe released Jev on 14 September 2026, Bespoke released Nimble-v1 as Bespoke-Nimble-9B on 18 September, Ollama added Nimble-v2 on 29 September, and Nimble-v4 and Nimble-v4-max launched on the API on 13 October. Bespoke says Nimble-v4-max scored 68.46 on the public suite of the Decision Index, with Nimble-v4 at 63.70, against Jev 1.13 at 57.96 and Perplexity Decider v1.1 at 62.25. The repository states those are Bespoke's own runs from 12 October and are not yet on the index dashboard.
Ollama's model page lists three question types: pick from a list, answer true or false, or place something on a rubric. A request can carry up to 64 questions about the same text. Choice and score questions take 2 to 26 options. Answers come back with probabilities and a confidence value that Ollama notes is not the chance the answer is right. That last caveat matters more than it looks. A confident wrong answer is still wrong, and the API hands you the number to check rather than a verdict.
The stack also has a distillation layer. Jevstiller, a GitHub project published on 30 September, sits in front of a repeated Jev classification call. It learns from Jev's own answers and their probability distributions, then answers most requests locally. On Banking77, replayed against Jev's recorded answers with default settings, its local model answered 61.7% of held-out messages at 99.40% agreement with Jev, against a 98% target. It kept Jev's accuracy at 78.8% versus 78.5% on the dataset's labels. A local answer takes about 15 ms on CPU, roughly 20 times faster than Jev's p50 of 300 ms, the project says.
What the week actually shows
Read together, these releases describe a market splitting in two. One half compresses: Phonon-2 at 164 MB, Echo at under $5,000 of training, Nimble at 9B parameters answering in under 100 ms. The other half streams. Strata moves 257 GB of weights through a gaming PC at single-digit tokens per second to keep a 125B-class model running without a server.
The compression half has the better evidence. Fermion publishes its evaluation methodology, names the licence it inherits and says which fast paths were gated on noise-level equivalence. Fulcrum publishes a metric it invented, which is weaker, but it is explicit that no canonical metric exists. Strata publishes its throughput table with the hardware attached, which is more than most inference projects do.
The missing piece is independent verification. None of these numbers has been rerun by a third party, and several come from the organisations selling the result. The Open ASR Leaderboard is a public benchmark, which helps Phonon-2. But the Decision Index scores for Nimble-v4 are Bespoke's own and the style Elo for Echo is Fulcrum's. That is normal for launch week and still worth saying out loud.
What has changed is not that small models got good. It is that the tooling to run, evaluate and distil them now fits on a laptop, and the people building it are shipping weekly. The 30 September batch is four releases in one day, from four unrelated teams, none of them a large lab. That is the story.
Sources
11- 01Echo – A SOTA writing model for style imitationEN
- 02Phonon-2: most accurate open speech recognition model in a 164 MB downloadEN
- 03Strata: Run a 125B-parameter AI model on a normal gaming PCEN
- 04Ollama now supports Jev-style decision modelsEN
- 05Ollama now supports Jev-like decision models all locally in 0.35EN
- 06Nimble: Data, Model, Recipe for an Open Jev (From Bespoke Labs)EN
- 07Jevstiller: Distill a repeated Jev classification task into a local modelEN
- 08Coop: A small language model pretrained by volunteersEN
- 09PSSA: A non-transformer language model written from scratch in RustEN
- 10AI models keep posting screenshots showing sensitive data from tech companiesEN
- 11TalkScribe – On-device dictation and local LLM rewrite for macOSEN
All figures and quotations in this text come from the sources listed below.
Content prepared by the editorial team with AI assistance.
Comments
0- No comments yet — be the first.