Skip to content
World clockEU--:--UK--:--USA--:--CN--:--PLDEFRIT中文EN

portal about AI and technologyevents · analysis · interviews · technical background

Search
LIVE
›

Microsoft and Hugging Face put agent reliability at 35% of enterprise tasks

A benchmark run by Microsoft and Hugging Face across 507 stateful business workflows found that agents failed executable checks in 79,853 of 121,680 valid trials. Of those failures, 67.24% still ended with a clean reply and no tool error.

AI & modelsAnalysisRachel NwosuPublished: 4 October 20266 min readSources 8
Microsoft and Hugging Face put agent reliability at 35% of enterprise tasks

Microsoft and Hugging Face published ThinkingBox on 3 October. It grades AI agents on the records they leave in a database, not on the sentences they write back. The headline number is blunt. Across a common-set ablation of 121,680 valid trials run on 12 LLM models, 79,853 attempts failed the executable checks. The work is out through Hugging Face, and the paper ships with it.

What changes for anyone buying or running agents is the unit of proof. A polished final answer and a well-formed tool call are proxies, the authors argue, and weak ones. An agent can sound correct while writing the wrong ticket status, skipping a required side effect or creating an extra one. For enterprise buyers, that means acceptance tests built on transcripts will pass systems that quietly corrupt records. Neither Microsoft nor Hugging Face has said how the benchmark will feed into product gating. The 507 workflows are a fixed set, not a census of production tasks.

A clean reply is not a clean end state

The benchmark's opening example is a $745 kitchen appliance stuck fifteen days past its delivery estimate at a Nashville distribution center. The agent makes nine tool calls. It reads the refund policy correctly and works out that the customer does not qualify for late-delivery compensation. Then it closes the ticket as resolved. The carrier exception is still open. The required end state was on hold, pending resolution. The database is what disagrees.

The failure breakdown is where it gets uncomfortable for teams that already run agents. Of the 79,853 failed attempts, 67.24% still terminated cleanly, invoked a state-changing tool and reported no final tool error. Executable checks found wrong field values in 77.61% of those cases, unintended extra effects in 43.30% and missing required effects in 25.36%. Those findings overlap, and the paper says so. Every task runs 20 times from an identical clean backend. An agent that handles a refund once and botches the next four is not a refund agent.

A trajectory is a claim. Database state is the evidence. Repetition is the trust test.

The authors also credit outside contributors, including Tommy Guy of Enderis AI, previously at Microsoft, and Sergio Paniego from Hugging Face, alongside former interns Zhuochun Li, Ali Keramati and Youngmin Ko. The paper is available with the blog. The example task is runnable as sandbox_external_retail_group1.py:test_case_ST003_006, where the failing check is a single field.

Apple moves on the permission agents ask for

Separately, Apple said on 2 October that it will add controls to macOS around Full Disk Access, the permission it says exists so backup apps can work. In a post on its Developer website, Apple wrote that "some developers are using Full Disk Access in ways that could put users at risk, exposing everything on their systems, including files, mail, messages, and even browsing history, without users' full knowledge and understanding." 9to5Mac reported the change the same day.

Apple gave no detail on what the new controls look like or when they ship. It did tie the move to agents directly: "As AI agents become increasingly capable and autonomous, the risks associated with this level of access will grow substantially." 9to5Mac names Meta Muse and OpenAI's Dots as always-on agents that request broad access to personal data. For enterprise Mac fleets, the practical question is whether the extra prompt breaks scripted setup. Apple has not said.

Agent identity is drawing attention at the other end of the stack too. AgentMail's AgentID pitch, published on 4 October, is that handing an agent your own login means every app records its actions under your name. The only way to cut it off is to change your password. Its alternative gives the agent its own inbox address as a stable identifier, with the human recorded as owner. The post is vendor material and the security claims are the company's own. The shape of the problem is real enough that Replit has shipped per-agent identity tracking, according to a 27 October headline in the trade press.

Oversight has a cost, and a limit

The human approval gate that most teams rely on has its own arithmetic. A paper by Emre Turan, submitted to arXiv on 8 June, labelled 125 adversarially weighted agent actions. It found reviewers only moderately agree on which are risky, with a Fleiss' kappa of 0.52. There is no single correct label to train against. Worse, when the reviewer is modelled as tiring as escalation volume rises, realised safety traces an inverted U: escalating more can make the system less safe. The safety-optimal guard escalates below full escalation, which is also the setting a load-aware policy can use to resist a flooding attack. The paper's own framing is that oversight is a resource-allocation problem. It says the inverted U and the flooding result are modelling outcomes that motivate a human study.

Sarthak Gaud's write-up on evals, published on 4 October, makes a compatible point from the build side. Non-deterministic systems cannot be graded against one fixed expected output. So he puts plain code checks first, such as whether the database ended in the right state or the right tool was called. He brings in a judge only for what code cannot check. He also warns that agent runs must be frozen and versioned, tools and MCP responses included, or old runs cannot be replayed. That is the same lesson as ThinkingBox from a different direction: the record is the test.

Drift arrives without a deploy

There is a quieter failure mode that no single run captures. On 3 October, sdarchitect.blog published the fourth part of a series on AI risk, asking why an agent's behaviour changes when nobody changed the agent. No new prompt, no code change, no swapped model, and the release pipeline is clean. The post's answer is that an agent is a system of models, harnesses, tools, data, other agents and humans. Any of those moving alters the output. It cites NIST guidance on post-deployment monitoring of generative AI systems and Microsoft's lifecycle guidance treating foundation models as versioned dependencies.

The author's five dimensions of agent drift are interconnected. That is what makes diagnosis harder than reading a diff. The commercial implication is that continuous evaluation is not a nice extra for teams shipping agents into production. It is the only way to notice that the thing you approved last month is no longer the thing running now.

None of this has stopped the tooling rush. Kimchi, launched on 4 October, sells a governed coding agent with hard budget caps per user, team and API key, plus per-model spend tracking and an enforcement layer that sits before a request leaves the environment. Spawned, also out on 4 October, offers a control surface for deploying infrastructure into a customer's own AWS, GCP or Azure account, with resources versioned and Terraform-compatible. Both are addressing a real gap: agents that act need budgets and boundaries that humans can audit.

The pattern across the week is that the tooling layer is maturing faster than the evidence layer. Benchmarks like ThinkingBox put a number on how often agents leave the wrong record behind. Apple is tightening the permission that lets them reach the disk. Identity vendors want agents to stop borrowing human logins. What remains unknown is whether enterprises will pay for that evidence, or keep shipping on the strength of a clean final answer.

Comments 0

Sources

8
  1. 01The Agent Said It Was Done. The Database DisagreedEN
  2. 02Apple says it's tightening macOS privacy controls amid the rise of AI agentsEN
  3. 03Give Your AI Agent Its Own IdentityEN
  4. 04Oversight Has Capacity: Calibrating Agent Guards to a Subjective Fatiguing HumanEN
  5. 05Writing Evals for Agentic Systems as a 4 Step LoopEN
  6. 06AI Risk - A user's guide (Part IV): Understanding Agent DriftEN
  7. 07Pi-based coding agent with hard budget capsEN
  8. 08Spawned: AIO cloud platform for humans and agentsEN

All figures and quotations in this text come from the sources listed below.

Content prepared by the editorial team with AI assistance.

Rachel Nwosu

Rachel Nwosu

AI, models and technology

Rachel Nwosu covers AI, models and technology for FLASH24, working from public model documentation, benchmark releases and repository histories rather than press summaries, and she skips announcements that arrive without reproducible numbers. She checks training-data claims against dataset cards and reruns reported metrics where code is available. She spends much of her week interviewing researchers and engineers, tracking model launch calendars, and comparing vendor benchmarks with independent evaluations. Outside the desk she runs 3D printers, restores old computers, and tests how models learn from internet junk. She does not publish benchmark figures she cannot trace to a source.

Newsroom →

Comments

0
  1. No comments yet — be the first.

Write a comment

Comments are public. We do not publish abuse, spam or advertising.