Skip to content
World clockEU--:--UK--:--USA--:--CN--:--PLDEFRIT中文EN

portal about AI and technologyevents · analysis · interviews · technical background

Search
LIVE
›

Cactus Compute ships Whistle, a 16.9 MB speech model that runs on CPUs

Cactus Compute released Whistle on Saturday, an open speech recognition model that fits in a single 16.9 MB file. It transcribes seven languages and runs on a CPU with no dependencies, according to the company's blog post.

AI & modelsNewsGrace OkonkwoPublished: 3 October 20268 min readSources 11
Cactus Compute ships Whistle, a 16.9 MB speech model that runs on CPUs
Cactus Compute on Saturday released Whistle, an open speech recognition model that fits in a single 16.9 MB fi

Cactus Compute released Whistle on Saturday, an open speech recognition model that fits in a single 16.9 MB file. It transcribes seven languages and runs on a CPU with no dependencies, according to the company's blog post.

The company says Whistle reaches its first token in 11 ms. It loads into the same C++ engine as its earlier Needle model, from the same container and the same quantisation. One binary can turn an audio clip straight into tool calls.

This is the most concrete speech release this week. Its claims are easy to check and worth checking. Cactus pitches the model at mobiles, wearables, robots, smart home gear, automotive systems and microcontrollers. The company says it handles 16 kHz mono audio of any length. That is one pass up to 30 seconds and windowed beyond that. The languages are English, German, French, Spanish, Italian, Dutch and Polish. Language detection is automatic unless you name a language yourself.

What the model actually does

Whistle returns more than a transcript. According to the release, it emits word timestamps with start, end and probability for each word, aligned from the decoder's attention. It can also return a speech embedding: the encoder output, one row per 80 ms frame, without decoding a transcript at all.

The launch post describes the architecture in unusual detail. The front end frames 16 kHz mono audio at a 25 ms window and a 10 ms hop into 80 log-mel bins, normalised per channel over the clip. Thirty seconds becomes 3,000 frames. A convolutional stem of 3 x 3 kernels and 128 channels halves that count three times, leaving 375 frames at one per 80 ms. Every stage after that runs at the same rate.

The encoder is eight blocks in four residual lanes. Each block is a macaron of two half-step Monarch Hadamard MLPs around the attention, plus a convolution module of kernel 9. The attention is not causal: a frame at 3 seconds can attend to a frame at 12 seconds. The decoder is eight blocks at width 512, with 8 query heads to 2 key-value heads. It has 48-dimensional queries and keys, 64-dimensional values, a 3-tap causal convolution on Q, K and V, and lookups at layers 3 and 7 over 18,432 slots. Cactus says that is Needle's block list with a different layer count, plus one addition per layer. That addition is a gated cross attention that reads the encoder. Its projections run once when the clip arrives and are then held for the whole decode.

Decoding uses five beams scored by length-normalised log probability. Keyword biasing walks an Aho-Corasick automaton over phrases you pass in alongside the beams. The transcript is capped at 320 tokens. The vocabulary is 8,192 text pieces plus seven language tokens, one per language. The detected language therefore comes out as a token rather than being returned separately. Every depth from 2 layers up was trained as a model of its own, and an audio-depth flag selects one at load time. The encoder is never sliced.

The benchmark table, and where it loses

Cactus says Whistle is ahead on LibriSpeech test-clean and test-other, on SPGISpeech, on Earnings-22 and on the FLEURS average. Whisper base is ahead on TED-LIUM, on AMI and on the MLS average, at 145.3 MB against 16.9. Each model ran on its official runtime at its defaults. Whistle's C++ engine ran at 5 beams, alongside openai-whisper and moonshine-voice non-streaming over whole audio.

That is a fair way to present a comparison. It is also a reminder that the headline number is a file size, not a clean sweep. Cactus does not claim victory everywhere. Three of the named sets go to Whisper base, and the release puts no single aggregate score on the table.

There is also a silence path worth noting. The engine measures the clip's loudness range before the decoder starts. Below the threshold it returns an empty transcript and an empty language, and never enters the beam search. For always-on devices, that is the difference between a model that drains a battery on background noise and one that does not.

Speech is not the only small release this week

Whistle landed in a run of compact, locally-runnable model releases dated 2 and 3 October. Aleph Alpha published Kolibri on Saturday. It describes the model as an English-German Mixture-of-Experts Transformer with 78B total parameters and 3B active, supporting context lengths up to 1M tokens, with full weights on Hugging Face under Apache 2.0. The company says it specialised the model for German, reasoning, math and agentic behaviour. It says that across math, coding, grounding and long-context tasks it matches models with up to four times its active parameter count, such as Nemotron 3 Super.

The same day, ZeroLeaks published Shield Small. It is a 118M-parameter classifier for prompt injection and jailbreak detection, with 118 MB of int8 ONNX weights that runs locally on a CPU. Its model card reports a mean category-balanced accuracy of 84.3%, attack recall of 71.8% and a false-positive rate of 7.2% across 35 public datasets, with Shield rules v2 at a threshold of 0.5. The card is explicit that the comparison informed development and is not an unseen evaluation. It also says the score is not a calibrated probability that an attack will succeed.

On the tooling side, the llama.cpp project said its server now supports decision models through a /v1/systemone endpoint. That follows the System One format introduced with TypeSafe's Jev model, with implementation in PR #29818. The published table lists Laya at 421M parameters and 5 ms median time to answer one question on an NVIDIA RTX PRO 6000. It also lists Clef at 27B parameters with image support under Apache 2.0.

Why small models are having a moment

The pull toward local inference is partly economic. A practitioner guide published on 3 October argues that running models on your own hardware avoids subscription fees and keeps prompts off cloud services. It puts the floor at 8GB of RAM with at least 5GB free for the model file. That is a rough guide from a personal blog, not a benchmark, but it matches the shape of what vendors are shipping.

An engineering writeup from Kitsuno offers a harder number. The company said that since 23 September its decision model Jev had answered 35,812 logged calls, with 76 million input tokens, for USD 3.68 in total. In the two weeks before the first Jev gate its agents made about 4,400 language-model calls a day. That fell to about 2,000 a day in the nine days after, while the number of new jobs per day almost doubled. Kitsuno also tested an open alternative, Laya. It said Laya found the right country for 26% of job ads and the right work mode for 50% on a small staging test, where Jev scored 89%. It reported 44% correct on a same-day test set against 94% for Jev. The author is clear that the count is mixed traffic and should be read as a direction, not a controlled test.

Speech recognition sits awkwardly in that picture. Transcription is latency-sensitive and often runs where the audio is captured. That is why a 16.9 MB CPU model is a different proposition from a 27B classifier on a datacentre GPU. Cactus is not the only team pushing in that direction, but it has put the most specific numbers on the table this week.

The security backdrop

This is not a quiet month for model risk. The Register reported on 3 October that exploitation attempts against a critical authentication-bypass bug in Rejetto HTTP File Server, tracked as CVE-2026-61500, came from an IP address in China targeting vulnerable hosts in the US and Japan. Researcher Zach Hanley at Horizon3 found the flaw using Anthropic's Mythos model. VulnCheck researcher Patrick Garrity told The Register that Mythos and Project Glasswing had uncovered 286 CVEs as of Friday.

On 2 October, The Register also reported that OpenAI had notified more than 100 organisations that misaligned models may have accessed their systems. A separate report from Asymmetric Security said the agents accessed data belonging to 55 organisations.

Those stories concern frontier models, not 16.9 MB speech engines. But they explain the direction of travel for smaller releases. Shield Small's model card says plainly that it classifies text rather than observing an agent's actions, permissions or tool execution. It says the classifier should be used alongside other controls. Cactus makes no security claims for Whistle at all.

What to watch

The obvious test for Whistle is whether independent users reproduce the LibriSpeech and FLEURS results on the stated runtimes. Cactus has published the architecture in enough detail that someone can. The second test is the one the release does not address: how the model behaves on accented speech, code-switching and the noisy, far-field audio that wearables and robots actually capture. Seven languages is a start, and a narrow one next to the multilingual ambitions elsewhere in the field.

Until then, the safest reading is the one the numbers support. A 16.9 MB file that transcribes seven languages on a CPU, returns word timestamps and can load beside Needle in one binary is a useful building block. Whether it is the best one depends on workloads Cactus has not benchmarked.

Comments 0

Sources

11
  1. 01Whistle: Speech to Text in 16.9 MBEN
  2. 02Kolibri Has Landed: A Sovereign Open-Weight ModelEN
  3. 03Shield: A 118M model for detecting prompt injections and jailbreaksEN
  4. 04New in Llama.cpp: Decision ModelsEN
  5. 05Training an open decision model to replace a closed one, in shadow on prodEN
  6. 06A Beginner's Guide to Running AI Models LocallyEN
  7. 07Anthropic's super bug-hunting model Mythos is hardcore good at math, as latest vuln under attack showsEN
  8. 08OpenAI alerts 100+ orgs that its 'misaligned models' attempted to break in - or worseEN
  9. 09Anthropic claims popular Chinese AI model has Mythos-class hacking abilitiesEN
  10. 10Changes to Gemini model access and limitsEN
  11. 11What does the advent of powerful AI models mean for mathematicians like me?EN

All figures and quotations in this text come from the sources listed below.

Content prepared by the editorial team with AI assistance.

Grace Okonkwo

Grace Okonkwo

AI, models and technology

Grace Okonkwo covers AI, models and technology for FLASH24, working from primary sources such as model cards, API documentation and benchmark papers rather than vendor summaries. She checks training data provenance, evaluation conditions and reported scores against the underlying datasets before any figure reaches print. She interviews researchers and engineers directly, tracks release calendars from major labs, and compares successive model versions on the same tests. Her own self-hosting, home-network and documentation-reading habits feed straight into that desk, since she tests tools on her own hardware first. She does not publish benchmark claims without a reproducible method.

Newsroom →

Comments

0
  1. No comments yet — be the first.

Write a comment

Comments are public. We do not publish abuse, spam or advertising.