Jeff-Code puts a 0.8B model in the loop to speed up a 27B coder by 47%

On 5 October the project's GitHub page went up with a narrow claim. Jeff-Code runs a 0.8B decision model, named Jeff, inside the agent loop of Pi, the coding agent by Mario Zechner and contributors. Jeff takes two quick decisions around every Qwen turn, each about 0.2 seconds. Should it run the next step itself, and does Qwen need to think hard on this turn.

Across 1,242 paired tasks, pass rate came out at 62.4% for Jeff-Code against 62.8% for Qwen 3.8-27B alone. The paired difference is minus 0.2 points, with a 95% interval of minus 2.6 to plus 2.1. Average time per task fell to 0.68x, interval 0.64 to 0.72, and the median was 0.70x. That is where the 47% figure comes from: 32% less time, on average.

This is the shape of on-device AI that actually ships in 2026. The small model does not write the code, and nobody claims it does. It decides what to skip.

The same pattern shows up in Liquid AI's d1, a decision model that returns probabilities instead of generating tokens, and in Reika, a coding agent CLI built around 8B to 35B models at Q2 to Q4 on a 16 to 32k window. For users, the practical question stops being whether a small model is smart enough and becomes whether it is cheap enough to leave running. Neither Jeff-Code nor Liquid has said how these systems behave on hardware slower than a workstation GPU.

What the harness can and cannot fix

Reika's documentation makes the case with measurements rather than adjectives. The project reports that one mid-context rewrite re-processed 8,453 tokens of prompt, which cost 7.9 minutes of prefill, where an append in the same session cost 25 tokens and 3.5 seconds. It also found that prompt wording was the weakest lever it had. On a vague task at Q2, a model converges or spirals about 50/50, and the harness cannot move that rate. It can only change what the failing half costs.

The test results are unflattering by design. The page says 14B to 35B models handled multi-file tasks most reliably, 8B to 9B models managed simple, well-scoped ones, and anything below 7B rarely got far. The headline demo runs Qwen3.6 35B at Unsloth UD-IQ2_M through llama.cpp, sped up three times, and the model's first test fails before it fixes it from the error. Reika is pre-1.0, so commands and behaviour can still change between releases.

Jeff-Code's numbers show the same honesty about where the gains stop. Total wall-clock time for all tasks combined drops less than per-task time, to 0.86x with an interval of 0.80 to 0.93, because in about 5% of tasks Jeff-Code runs more than 30 minutes longer than Qwen alone. The per-benchmark split is uneven too: 0.63x on SWE-bench Verified and 0.64x on Terminal-Bench Pro, but no clear speed-up on Terminal-Bench 2.0 at 0.96x or SkillsBench at 0.91x.

Then the part that matters to anyone tempted to copy the trick. Running with thinking off on every turn is faster still, and costs 7.6 points, interval minus 10.6 to minus 4.5, with a 13.5 point drop on Terminal-Bench 2.0. A less cautious Jeff, with its threshold raised from 0.6 to 0.7, loses 4.5 points. In the evaluation as run, about three quarters of Qwen's turns ran with thinking off. The small model is not adding intelligence. It is deciding when intelligence is required.

Decision models move from text to images

Liquid AI took the same idea in a different direction on 5 October, announcing d1, which it calls its first decision model and the first to handle images as well as text. A d1 call reads its input in one forward pass and returns probabilities per answer without generating tokens; a text decision takes 200 to 300 milliseconds. The company says d1 matches or beats GPT-6.1 Sol on four of six test applications and costs 19x to 200x less than both that model and Claude Opus 5.5, and answers faster on every task.

The most interesting part is that the model was never trained for this. Thanks to its excellent generalizability, it understands the task from a short description.

That line comes from Liquid's own post, about a visual inspection demo built on the public VisA dataset of circuit boards, candles, cashews and chewing gum, where d1 sorts good and defective parts with 85% to 97% accuracy. The company's own framing is the interesting bit: a small specialised model generalising to a task it never saw, from a sentence of description, instead of a prompt and a context window.

One of Liquid's demos lands directly on the coding-agent problem. Called Context Compaction, it has d1 read each tool output in an agent session and keep, trim or drop it, removing 52% of the tokens while keeping every output the task needs. Liquid says it adapted four of the six text applications from open-source projects. The benchmark table is the company's own, and d1's rival Jev appears only as a name: Liquid says a week-earlier experimental release made d1 the first model to rival Jev on text decisions, but gives no Jev figures.

Sean Goedecke's 5 October post explains why a model like Jev or d1 is easy to replace once it works. Because you prompt it for a specific task, you can log its inputs and outputs and train your own classifier on them. The hand-built model will be cheaper and faster, since it carries no knowledge of unrelated tasks. Goedecke's point is not that small decision models are weak. It is that they are a stepping stone, and a cheap one to validate a feature before anyone hires ML expertise.

The small-model stack fills in around them

Other releases in the same 24 hours show how far the tooling around small local models has moved. Interfaze 1.1 Lite, published on Hugging Face on 5 October, is a mixture-of-architectures model that runs on one 80 GB GPU with no external services. A vision-language core routes work to specialists for documents, speech, object detection and safety checks, and it holds a 131k-token context. Its own table has it at 73.2 on MMMU-Pro against 71.1 for Interfaze and 40.4 for GPT-5.4-Mini, and at 83.8 on olmOCR-Bench against 85.7 for Interfaze, so the small model loses to its own larger predecessor on one document benchmark and wins on multimodal reasoning.

Metagente, posted on GitHub on 5 October, takes the opposite bet: a single Rust binary and a short declarative agent file instead of a Python framework. It speaks MCP and A2A natively, and its pitch is that each agent declares which folders, environment variables and other agents it may use. That permission model is the part worth watching for anyone running agents on their own machine, where the failure mode is not a bad answer but a bad file operation.

The academic side is not settled either. A UC Riverside study, published on 23 September, found that confidence and correctness in large language models arise from different internal features. The team studied Meta's Llama-3.1-8B and Google's Gemma-2-9B with sparse autoencoders and found that suppressing features tied purely to uncertainty sharply reduced accuracy, while suppressing most features tied to incorrect answers did not.

Het Patel, a UCR computer science doctoral student and lead author, framed the finding against the field's default: "The main assumption in the field is that when the model is confident, it is likely to be correct, and when the model is unsure, it is more likely to be incorrect." The useful part is that the researchers changed behaviour by turning internal features down rather than retraining, which is the same economics Jeff-Code and d1 are chasing.

What the numbers do not settle

The case for small models on-device rests on three claims that the 5 October releases support unevenly. On cost, Liquid's 19x to 200x gap and Jeff-Code's 0.68x per-task time are both self-reported, run against baselines the companies chose. On capability, Reika is explicit that its harness makes a small model less frustrating rather than smarter, and Jeff-Code's pass rate is flat, not better. On trust, the UCR work says a model's own confidence is a poor guide, which matters more when the model is deciding what to skip than when a person is reading its answer.

Reflection's Beam, announced on 5 October, shows the counterweight. Its 501B sparse model, with 23 billion active parameters, reaches 80.1 on Terminal Bench v2.1, where Kimi K3 scores 88.3 and DeepSeek V4.1 Flash 90.6. Reflection says Beam matches GLM-5.2 on advanced reasoning while using 3 to 4 times less inference compute. The weights, technical report and model card are due later this month, and the model is still in red-teaming.

So the frontier is not standing still, and the small-model story is not a race it is winning. It is a different bet: that most agent work is routing, reading and deciding, and that a 0.8B model taking those steps while a bigger one thinks is worth more than a bigger one doing everything. The evaluation behind Jeff-Code supports that bet on speed and leaves it unproven on quality. Nothing in it shows the pairing beating Qwen alone. It shows it keeping up while spending less.

What nobody has published is the number buyers would want most: the cost of running the pair on hardware a normal team owns. Jeff-Code ran on the same Qwen server in paired blocks. Liquid quotes per-decision cost against two hosted models. Reika's figures are prefill times on a local llama.cpp setup, which at least puts the harness overhead in view. Until those numbers are comparable, the honest summary of 5 October is that small models have become cheap enough to sit inside the loop, and the arguments about whether they belong there are only starting.