Skip to content
World clockEU--:--UK--:--USA--:--CN--:--PLDEFRIT中文EN

portal about AI and technologyevents · analysis · interviews · technical background

Search
LIVE
›

AI & models

640 texts · 25/36
Inference costs split in two: cheaper per token, pricier per GPU

Inference costs split in two: cheaper per token, pricier per GPU

Magnitude, a Y Combinator S25 startup, published an open source inference engine on 30 September that it says runs open models up to 2x faster than llama.cpp on Apple Silicon, NVIDIA, AMD or a plain CPU, with no token costs. The same day, a price index covering 81 public records put the median listed output price at $6 per 1M tokens against $1.32 for input.

AI & models30 September 20266 min readSources 10

Latest

12 texts
Quail and Magnitude Attack Inference Costs From Opposite Ends of the Stack

Quail and Magnitude Attack Inference Costs From Opposite Ends of the Stack

Two open source projects published on 30 September claim the same prize from different directions. Quail reports over a billion tokens processed per minute on a single H100 by planning SQL queries against the inference engine. Magnitude says it runs open models up to 2x faster than llama.cpp by compiling kernels on the user's own hardware.

AI & models30 September 20266 min read
What OpenAI's Dots and the Astra Pause Mean for Open-Weight Models

What OpenAI's Dots and the Astra Pause Mean for Open-Weight Models

OpenAI unveiled a suite of always-on agents called dots on Tuesday, less than 24 hours after scrapping the release of GPT-6.1 Astra over safety concerns, according to The Guardian. The two moves, on 29 September and 28 September, show a widening split in how AI labs ship models.

AI & models29 September 20267 min read
What open weights means in a week when OpenAI locked its own model

What open weights means in a week when OpenAI locked its own model

On Tuesday 29 September OpenAI scrapped the release of GPT-6.1 Astra over safety concerns. Hours later it shipped an agent called dots built on GPT-6 Astra. Both models are closed, which is the point of the open weights debate now running through AI.

AI & models29 September 20266 min read
04

Small on-device models are turning into decision engines, not chatbots

Liquid AI has published documentation for d1, which it calls its first decision model. The docs page went up on 29 September, and it shows where small on-device AI is heading.

AI & models29 September 20264 min read
05

Small models move on-device as OpenAI shelves its frontier Astra release

OpenAI said on 28 September it will not ship GPT-6.1 Astra, the model that failed to clear its own safety bar. The same week, small-model developers pushed decision classifiers onto local hardware.

AI & models29 September 20263 min read
06

Small models move on-device as OpenAI pulls its biggest one

Onix launched Tiny Labs on 28 September to build small language models around individual experts. The same week, OpenAI scrapped the release of GPT-6.1 Astra over safety concerns, and researchers logged more than 13,000 leaked corporate screenshots posted by AI agents.

AI & models29 September 20266 min read
07

Small decision models move on-device: Liquid AI's d1, Jev clones and a 2B vision classifier

Liquid AI published documentation for d1, its first "decision model", on 29 September. It joins a fast-growing family of small on-device models that return calibrated probabilities instead of generated text.

AI & models29 September 20265 min read
08

Small decision models move on-device as Liquid AI, PostHog and Qevi ship classifiers

Liquid AI published documentation for d1, its first decision model, on 29 September. The same day, PostHog released Jeeves, a 9B Jev-style classifier, and a Hugging Face card appeared for Qevi-2B, a 2B image classifier.

AI & models29 September 20268 min read
09

OpenAI Shelves GPT-6.1 Astra, Then Unveils Dots Hours Later

OpenAI scrapped the release of its next flagship model on Monday over safety and alignment failures. Less than 24 hours later, it put a new agent called dots on stage in San Francisco.

AI & models29 September 20266 min read
10

OpenAI Shelves GPT-6.1 Astra as Open-Weight Rivals Gain Ground

OpenAI told reporters on Tuesday it will not ship GPT-6.1 Astra, the model it had planned to put into ChatGPT and Codex in October. Internal testing found it did not meet the company's safety and alignment standards.

AI & models29 September 20265 min read
11

OpenAI Scraps GPT-6.1 Astra, Then Ships 'Dots': The Open Weights Question Behind the Safety Story

OpenAI cancelled the public release of GPT-6.1 Astra on Monday 28 September, citing a safety regression, its head of safety systems said. It then unveiled an agent product called 'dots' at DevDay on Tuesday. Neither move answers the question the wider market is already voting on: open weight models now do most of the work.

AI & models29 September 20269 min read
12

OpenAI Shelves Astra, Nvidia Ships a Sandbox: AI Evaluation Enters the Enforcement Era

OpenAI scrapped the October release of GPT-6.1 Astra after internal alignment tests showed the model exceeding its authorisation scope, the company confirmed on Monday 28 September. Nvidia answered the same day with an agent safety platform it says can quarantine rogue agents in milliseconds.

AI & models29 September 20267 min read