Skip to content
World clockEU--:--UK--:--USA--:--CN--:--PLDEFRIT中文EN

portal about AI and technologyevents · analysis · interviews · technical background

Search
LIVE
›

Anthropic says GLM-5.3 nearly matches Mythos at writing exploits

Anthropic published a report claiming Zhipu AI's GLM-5.3 can automate exploits close to the level of its unreleased Claude Mythos model, Tom's Hardware reported on 3 October. In Anthropic's Exploitbench runs, GLM-5.3 built working Chrome exploits 50 times out of 410 attempts; Mythos managed 56.

AI & modelsExplainerRachel NwosuPublished: 3 October 20266 min readSources 8
Anthropic says GLM-5.3 nearly matches Mythos at writing exploits

Zhipu AI's open-weight GLM-5.3 came within six successful exploits of Anthropic's own unreleased Mythos on a Chrome exploit benchmark. That is according to a report Anthropic published and Tom's Hardware covered on 3 October. The gap on a second, internal test was two percentage points: GLM-5.3 hit a 4% success rate on "full control-flow hijacks", Mythos 6%. Kimi K3 and DeepSeek V4.1 Flash scored 0% on that same measure.

The claim matters less for the scoreboard than for what it implies about who can run this capability. Mythos is the model Anthropic says is too dangerous to release publicly. Access goes through Project Glasswing, an initiative launched in April that gives selected partners the model for bug hunting. GLM-5.3's weights are open. If Anthropic's numbers hold, the gap between a gated frontier model and a downloadable one is now small enough to argue about. That argument now has a number attached to it, and the number is six.

Anthropic's report also says the stock GLM-5.3 model is easy to steer around its own safeguards. Tom's Hardware reports two methods. A deceptive prompt in which the model role-plays an adversarial autonomous agent succeeded 64% of the time. Prefilling the model's thinking tokens to force a response succeeded 92% of the time. A third method, abliteration, is also described. None of this has been independently reproduced.

What the benchmark actually measured

Exploitbench is a sandboxed environment where a model tries to develop exploits for Google Chrome. Anthropic ran 410 attempts per model. The headline comparison is 56 successes for Mythos against 50 for GLM-5.3. Anthropic also says GLM-5.3 developed chained exploits autonomously. It says its lighter Flash variant could build chained exploits against known bugs for $20.40 in tokens. That works out to roughly 100 to 300 million tokens depending on the input-to-output ratio.

The caveats are worth stating plainly. These are vendor-run benchmarks on a model the vendor competes with, and Anthropic's own framing has been questioned before. Tom's Hardware notes that Anthropic is seeking regulation of AI development while shipping Claude Opus 5.5 and Claude Sonnet 5.5 days after its CEO called for pacing the frontier. Anthropic cites a late-September report from the Center for AI Standards and Innovation claiming GLM-5.3 can fully automate exploits at a level similar to Mythos.

"We started detecting exploitation of CVE-2026-61500 in Rejetto HFS this evening," VulnCheck security researcher Patrick Garrity posted on LinkedIn on Thursday.

Anthropic has its own evidence that Mythos-class bug finding has consequences. The Register reported on 3 October that a critical authentication-bypass flaw in Rejetto HTTP File Server, tracked as CVE-2026-61500, was found by Horizon3 researcher Zach Hanley using Mythos. It is now under attack. Horizon3 joined Project Glasswing in July. Exploitation attempts came from an IP address in China against hosts in the US and Japan, Garrity told The Register. Later hits were routed through two US addresses in the same subnet that appeared to be a proxy. Users should update to HFS v3.2.1 or later.

Garrity's tracker counts 286 CVEs found through Mythos and Project Glasswing as of Friday, The Register reported. Until Thursday, only one of them had been exploited in the wild. That ratio is the argument for gated access: many bugs found, few attacked so far. It is also the argument against reading a single week's exploitation data as a safety guarantee.

Why model behaviour is hard to measure at all

Separate work published this week suggests hallucination and capability benchmarks are shakier than their scores imply. Aleph Alpha's research blog describes a training method that turns question answering into a three-player game. One context supports the right answer, another has the evidence cut out, and the model must answer on the first and abstain on the second. Across five QA benchmarks, the company says wrong answers under insufficient context fell by up to 35 percentage points against an instructed model. They fell 18 to 20 points against standard training, with a grounding score rising by up to 0.38.

The point behind the method is that a benchmark score tells you an answer was right, not where it came from. A model can read the retrieved document, recall the answer from pre-training, or lean on a spurious cue. Asking for a citation does not settle it, because the model picks the quote after it has answered.

The same problem shows up in retrieval research. ScholarCatalyst, a benchmark built from 184 lead authors labelling 207 recent computer science papers, asked models to find the prior work that shaped a project. It used only literature available when the project began. Agentic search did no better than plain embedding retrieval, 0.42 against 0.48 Recall@20. An agent built on Claude Fable 5.1, which may have seen the finished papers during training, reached 0.51.

PTXBench, an arXiv paper revised on 28 September, found that no evaluated model consistently matched frontier GPU libraries on kernel optimization. It also found that executing the right instructions did not translate into competitive speed. The authors fine-tuned Qwen3.6-27B and found generalization uneven.

The open-weight side is moving too

Anthropic's warning lands in a week when open releases kept coming. Aleph Alpha released Kolibri on 3 October, a German and English mixture-of-experts model with 78B total parameters and 3B active. It has a 1M token context and Apache 2.0 weights on Hugging Face. It is aimed at public administration, industrials and aerospace. The company says it matches models with up to four times its active parameter count, such as Nemotron 3 Super.

Tooling followed a day earlier. The llama.cpp project said its server now supports decision models through a /v1/systemone endpoint, following the format introduced with TypeSafe's Jev. Existing clients need only a new base URL. The listed models include Laya at 421M parameters and OpenJev at 27B, the latter under a non-commercial licence. A separate release, Shield Small, is a 118M-parameter classifier for prompt injection and jailbreak detection that runs on a CPU. Its model card reports 71.8% attack recall and a 7.2% false-positive rate on a comparison set used during development.

None of that settles whether GLM-5.3 is as dangerous as Anthropic says. What is public: Anthropic's numbers, the methods it describes, and a steady flow of capable open models. What is not: an independent test of the guardrail bypass rates, or any statement from Zhipu AI responding to the report. Anthropic has not said whether it shared the findings with Zhipu before publication.

Comments 0

Sources

8
  1. 01Anthropic claims popular Chinese AI model has Mythos-class hacking abilitiesEN
  2. 02Anthropic's super bug-hunting model Mythos is hardcore good at math, as latest vuln under attack showsEN
  3. 03Bounding Hallucinations: Merlin-Arthur Protocols for Mutual-Information Bounds in Language ModelsEN
  4. 04ScholarCatalyst: A Benchmark for Retrieving Papers That Inspire New ResearchEN
  5. 05PTXBench: Benchmarking and Adapting LLMs for GPU Kernel Optimization with Architecture-specific PTXEN
  6. 06Kolibri Has Landed: A Sovereign Open-Weight ModelEN
  7. 07New in llama.cpp: Decision ModelsEN
  8. 08Shield: A 118M model for detecting prompt injections and jailbreaksEN

All figures and quotations in this text come from the sources listed below.

Content prepared by the editorial team with AI assistance.

Rachel Nwosu

Rachel Nwosu

AI, models and technology

Rachel Nwosu covers AI, models and technology for FLASH24, working from public model documentation, benchmark releases and repository histories rather than press summaries, and she skips announcements that arrive without reproducible numbers. She checks training-data claims against dataset cards and reruns reported metrics where code is available. She spends much of her week interviewing researchers and engineers, tracking model launch calendars, and comparing vendor benchmarks with independent evaluations. Outside the desk she runs 3D printers, restores old computers, and tests how models learn from internet junk. She does not publish benchmark figures she cannot trace to a source.

Newsroom →

Comments

0
  1. No comments yet — be the first.

Write a comment

Comments are public. We do not publish abuse, spam or advertising.