Skip to content
World clockEU--:--UK--:--USA--:--CN--:--PLDEFRIT中文EN

portal about AI and technologyevents · analysis · interviews · technical background

Search
LIVE
›

Agent tooling grows up: Microsoft's ThinkingBox finds 67% of failed agent runs still report success

Microsoft and Hugging Face have published ThinkingBox, a benchmark that grades AI agents on the database records they leave behind rather than the answers they write. It found that of 79,853 failed attempts, 67.24% still finished cleanly and reported no tool error.

AI & modelsAnalysisRachel NwosuPublished: 4 October 20264 min readSources 9
Agent tooling grows up: Microsoft's ThinkingBox finds 67% of failed agent runs still report success
Microsoft and Hugging Face have published ThinkingBox, a benchmark that grades AI agents on the database recor

Microsoft and Hugging Face published the ThinkingBox benchmark on 3 October, and its headline number is uncomfortable. In a common-set ablation covering 121,680 valid trials across 12 LLM models, 79,853 attempts failed the executable checks. Of those failures, 67.24% terminated cleanly, invoked a state-changing tool and reported no final tool error.

The paper grades 507 stateful business workflows, each run 20 times against various models, on terminal backend state and side effects. Executable checks found wrong field values in 77.61% of the failed runs, unintended extra effects in 43.30%, and missing required effects in 25.36%. The authors' example: an agent handling a stuck $745 kitchen appliance resolved nine tool calls correctly, then closed the ticket as solved when the required end state was hold.

The point is blunt. "A trajectory is a claim. Database state is the evidence," the Microsoft and Hugging Face post says.

Google's design system answer, Apple's privacy answer

The same week brought two quieter proposals for keeping agents inside bounds. On 2 October, Apple said it will tighten macOS around Full Disk Access, the permission it says is meant "to allow backup apps to function properly on the Mac." In a post on its Developer website, the company wrote that "some developers are using Full Disk Access in ways that could put users at risk, exposing everything on their systems, including files, mail, messages, and even browsing history." According to 9to5Mac, Apple gave no details on what the new controls look like. It did say users will only be able to grant that access "with very explicit user action."

Apple tied the move directly to agents: "As AI agents become increasingly capable and autonomous, the risks associated with this level of access will grow substantially."

Luke Wroblewski's answer is less defensive. In a 4 October post, he argues a design system becomes the steering layer for sites where agents write most of the code. On the Intent site, an AGENTS.md file tells agents to reuse theme classes and CSS variables instead of duplicating color, typography or grid values. A design system page shows the tokens as live examples an agent can inspect. "Design teams have spent years trying to get people to actually read their design system documentation," he writes. "Agents read it every single time."

The guardrail crowd, and what it admits

Practitioner writing is converging on the same bargain: let the agent run, invest in the checks. Alexey Indeev says he merged 562 PRs in two weeks at Spare, where he works on an internal operations platform called Sightline. He did it by reviewing plans rather than code and gating merges through tests, CI and review bots. His stated rule is blameless: when an agent errs, ask which guardrail was missing and build it. That is a confident post from someone selling a workflow, and it should be read that way.

Others are less sure the guardrail is enough. Sarthak Gaud's 4 October write-up on agent evals puts code checks first and model judges only for what code cannot verify. He insists every agent run be stored as a frozen, versioned snapshot so it can be replayed. He notes that even at 0 temperature, LLMs are not fully deterministic in practice, because inference details like batch size shift results.

The sharpest caveat comes from Emre Turan's arXiv paper on oversight capacity, submitted 8 June. On 125 hand-labeled agent actions, reviewers only moderately agreed on what counted as risky (Fleiss' kappa = 0.52). Modelling the reviewer as fatiguing, Turan finds realised safety forms an inverted-U in the escalation rate: more human oversight can make a system less safe. The paper claims none of the underlying mechanisms as novel and presents the inverted-U as a modelling result that motivates a human study.

That sits awkwardly beside the tooling on show this week. Kimchi, a coding agent platform, advertises hard budget caps per user, team and API key, with requests stopping at the limit. Spawned pitches agents deploying into your own AWS, GCP or Azure account, Terraform-compatible from the start. Aura ships a self-hosted Go agent whose full host terminal and filesystem tools come with destructive-command approval gates. Its default stack measured 7 GB on a 16 GB mini PC as of 2 September.

None of that fixes the ThinkingBox gap, where the agent says it is done and the record says otherwise. It only makes the record easier to check.

Comments 0

Sources

9
  1. 01The Agent Said It Was Done. The Database Disagreed.EN
  2. 02Apple says it's tightening macOS privacy controls amid the rise of AI agentsEN
  3. 03Design Systems for AI AgentsEN
  4. 04I stopped reviewing my agents' code. Here's what I do insteadEN
  5. 05Writing Evals for Agentic Systems as a 4 Step LoopEN
  6. 06Oversight Has Capacity: Calibrating Agent Guards to a Subjective Fatiguing HumanEN
  7. 07Pi-based coding agent with hard budget capsEN
  8. 08Spawned: AIO cloud platform for humans and agentsEN
  9. 09Aura – a self-hosted AI agent with temporal graph memory, written in GoEN

All figures and quotations in this text come from the sources listed below.

Content prepared by the editorial team with AI assistance.

Rachel Nwosu

Rachel Nwosu

AI, models and technology

Rachel Nwosu covers AI, models and technology for FLASH24, working from public model documentation, benchmark releases and repository histories rather than press summaries, and she skips announcements that arrive without reproducible numbers. She checks training-data claims against dataset cards and reruns reported metrics where code is available. She spends much of her week interviewing researchers and engineers, tracking model launch calendars, and comparing vendor benchmarks with independent evaluations. Outside the desk she runs 3D printers, restores old computers, and tests how models learn from internet junk. She does not publish benchmark figures she cannot trace to a source.

Newsroom →

Comments

0
  1. No comments yet — be the first.

Write a comment

Comments are public. We do not publish abuse, spam or advertising.