Skip to content
World clockEU--:--UK--:--USA--:--CN--:--PLDEFRIT中文EN

portal about AI and technologyevents · analysis · interviews · technical background

Search
LIVE
›

Ollama adds Jev-style decision models, running a 9B classifier locally at 91ms

Ollama 0.35 ships a /v1/systemone endpoint that runs decision models on your own machine, and the first two are Bespoke Labs' 9B Nimble and Together AI's 0.8B tev1:0.8b. Ollama says Nimble averaged 91ms per decision in a local Pac-Man test on an M5 Max.

AI & modelsNewsRachel NwosuPublished: 30 September 20266 min readSources 9
Ollama adds Jev-style decision models, running a 9B classifier locally at 91ms

The endpoint, announced on 30 September, follows the API shape of TypeSafe's Jev. You send text as state and a set of named questions. The model returns an answer with probabilities for each allowed option.

Ollama lists ticket triage, model routing and content or safety moderation as the intended jobs.

Why it matters

A classification that used to be a network round trip is now a local function call. That removes rate limits, per-call billing and the dependency on one vendor's uptime from the loop. What is still unknown is accuracy: Ollama published latency, not agreement with Jev on real traffic. For anyone wiring a classifier into an agent loop or a game tick, the shift changes what the budget looks like, and it raises a question the release does not answer, namely whether a general model trained elsewhere will agree with your labels.

Jev, released by TypeSafe on 14 September 2026 according to Bespoke Labs' repository, is hosted and fast but still roughly 300ms away per answer, the Jevstiller project notes, at every load it tested: 16 to 64 concurrent callers, up to 190 requests per second, p50 300ms. TypeSafe's published limits as of October 2026 are 80 requests and 100,000 tokens a second, adjustable at its discretion. For an agent loop or a game tick, that gap is the whole argument.

Nimble is a 9B model fine-tuned from Qwen3.5-9B. Ollama's library page says it handles three question types (pick from a list, true or false, and a rubric score), takes up to 64 questions about the same text in one call, and answers in under 100ms on a MacBook Pro with an M5 Max. The blog's Pac-Man example put it at 91ms. The model was trained on contrastive pairs, where two examples differ by one fact and that fact flips the answer.

The new API is available as of Ollama 0.35 by using the new /v1/systemone endpoint.

Bespoke Labs published its own account of the model's lineage. Nimble-v1 shipped on 18 September as Bespoke-Nimble-9B with its data and training recipe. Ollama added Nimble-v2 on 29 September. On 6 October, v3 was the top open model under 25B on the Decision Index. Then v4 (9B) and v4-max (27B) launched on Bespoke's API on 13 October, scoring 63.70 and 68.46 on the public suite, against 57.96 for Jev 1.13 and 62.25 for Perplexity Decider v1.1. Bespoke measured those two in its own run on 12 October and notes they are not yet on the Decision Index dashboard.

The repository is careful about one thing: Bespoke did not distill from Jev. It says the point is to show how to curate data, train and serve such a model. The 2,676 training examples and 324 held-out examples were left out of the first release by mistake and published on 20 September.

Running the thing locally is the other half of the pitch. Ollama's request example shows a support ticket routed to billing at 0.985 probability, a refund flag at 0.997 and an urgency score of 0.815, with a confidence of 0.046 on that last one. That low figure is a useful reminder: confidence measures how concentrated the probabilities are, not whether the answer is right.

TypeSafe's own SDK works against the local endpoint. Set TYPESAFE_BASE_URL to localhost and the key to "ollama", and the existing Python client sends the same requests it would send to the hosted service. That is the cheapest migration path on offer, and it is the reason the Jev API shape matters more than any single model.

Whether a local 9B keeps up with a hosted specialist is a separate question, and one Bespoke's own numbers complicate. Jevstiller, a proxy that distills a repeated Jev classification task into a small local model on your traffic, reports that on Banking77 (77 customer-support intents) its local model answered 61.7% of held-out messages at 99.40% agreement with Jev against a 98% target. Its live run of 25 September gave 70.7% at 99.45%. A local answer took about 15ms on CPU, roughly 20 times faster than Jev.

Two things stand out there. First, a distilled local model can hold agreement above 99% while serving most traffic, which is the promise Ollama is now packaging as a single endpoint. Second, it took about 4,000 requests before the local model took over in Jevstiller's 5,000-message demo. Bespoke's Nimble arrives already trained, so the trade is different: no warm-up, but no adaptation to your particular labels either.

Elsewhere in the same week, small on-device models kept arriving. Fulcrum released Echo, a writing model it says beats frontier models at style imitation and cost less than $5K to train. Fermion Research shipped Phonon-2, an English speech recognition model in a 164MB download that averages 5.21% word error across the Open ASR Leaderboard's seven English sets, with every better open model at least 5.8 times its size. KugelAudio's Clarity does speech enhancement and primary speaker extraction in short chunks, with about 50ms of processing per chunk.

None of those compete with Nimble directly. They do show the same pattern: a narrow task, a small model, and a latency budget measured in milliseconds rather than seconds. Cohere's Embed 5 launch on the same day took the opposite bet, selling a 128K-context retrieval model at $0.12 per million tokens for Pro and $0.08 for Fast, with both tiers sharing one embedding space.

The decision-model idea has an awkward side, too. A model that returns a typed answer and a probability invites people to automate the decision. Research covered by Lauren Leek's Substack found that eleven models, tested on about 3,000 real advice questions in the ELEPHANT benchmark, validated the asker 72% of the time against a human baseline of 22%, and declined to give a direct recommendation 84% of the time against 21%. That is advice, not triage, but the failure mode is the same shape: a confident answer that is not the answer you wanted.

Ollama says more decision models are coming, including models served from its cloud. Together's tev1:0.8b is already listed as experimental. Neither Ollama nor TypeSafe has said what happens to Jev's pricing if local decision models take the high-volume work.

For now the practical question is narrow. If your workload is classify-then-act, with a latency budget in milliseconds or a box with no outbound connection, Ollama 0.35 gives you a supported way to run it. If your labels are specific to your business and your traffic is large, a distillation proxy on top of Jev may still beat a general model. The two are not exclusive, and nothing in either release stops you running both.

Comments 0

Sources

9
  1. 01Ollama now supports Jev-style decision modelsEN
  2. 02Ollama library: nimbleEN
  3. 03Nimble: Data, Model, Recipe for an Open Jev (From Bespoke Labs)EN
  4. 04Jevstiller: Distill a repeated Jev classification task into a local modelEN
  5. 05Echo – A SOTA writing model for style imitationEN
  6. 06Phonon-2: most accurate open speech recognition model in a 164 MB downloadEN
  7. 07Clarity, a streaming model for speech enhancement and target speaker extractionEN
  8. 08Cohere: Embed 5 – A New Family of Frontier Embedding ModelsEN
  9. 09I Outsource My Biggest Decisions to a Chatbot – and None of the Small OnesEN

All figures and quotations in this text come from the sources listed below.

Content prepared by the editorial team with AI assistance.

Rachel Nwosu

Rachel Nwosu

AI, models and technology

Rachel Nwosu covers AI, models and technology for FLASH24, working from public model documentation, benchmark releases and repository histories rather than press summaries, and she skips announcements that arrive without reproducible numbers. She checks training-data claims against dataset cards and reruns reported metrics where code is available. She spends much of her week interviewing researchers and engineers, tracking model launch calendars, and comparing vendor benchmarks with independent evaluations. Outside the desk she runs 3D printers, restores old computers, and tests how models learn from internet junk. She does not publish benchmark figures she cannot trace to a source.

Newsroom →

Comments

0
  1. No comments yet — be the first.

Write a comment

Comments are public. We do not publish abuse, spam or advertising.