Skip to content
World clockEU--:--UK--:--USA--:--CN--:--PLDEFRIT中文EN

portal about AI and technologyevents · analysis · interviews · technical background

Search
LIVE
›

AI memory plugins patch the wrong problem, developers argue as new tools arrive

A proxy that compresses Claude Code and Codex sessions used 29% fewer input tokens than the built-in compaction step, its developers say. It solved 61 of 89 Terminal-Bench 2.1 tasks.

AI & modelsExplainerRachel NwosuPublished: 3 October 20264 min readSources 9
AI memory plugins patch the wrong problem, developers argue as new tools arrive
A proxy that compresses Claude Code and Codex sessions used 29% fewer input tokens than the built-in compactio

Gobstopper, a compaction proxy for Claude Code and Codex, published benchmark results on 27 and 28 September. It solved 61 of 89 tasks with the proxy against 60 without it, on 84.3M input tokens against 118.6M.

The project's own site puts the run at a 45,000-token threshold rather than the 128,000 default, one trial per arm, with GLM 5.3 Flash. The same page concedes the two-task gap is within single-trial noise. It also says 21 of the 89 tail-0 trials may have run an earlier build. That honesty is rarer than the speedup claim.

Tools like this rest on one premise: context windows are not the bottleneck. Memory is. A growing group of practitioners argues the industry has spent two years solving the wrong half of that problem. That is why a benchmark page with a caveat paragraph now reads as more interesting than a launch post.

What the memory plugins actually do

Kevin Liao, writing on 3 October, describes the standard architecture of an agent memory plugin. Conversations get chopped into snippets, the snippets go into a vector database, and on every prompt the five most similar ones are attached. If the agent seems confused, it is given a search tool to dig for more.

Liao lists five failures of that design. Similarity search cannot tell which snippet is correct or current. Snippets lose the context around them. The past is treated as truth even as the codebase changes. Agents cannot search for what they do not know they lack. And the store of 10,000 embeddings is unauditable. His conclusion is blunt. People write things down instead of rewatching three-year-old meetings, he argues, and agents should work the same way.

The solution is NOT to give an agent a search tool that it uses to search over the past 10 million tokens worth of conversations.

"The entire memory plugin ecosystem is solving the wrong problem," he writes. That is an opinion piece, not a measurement. But it lands next to harder evidence that context handling is a systems problem, not a retrieval problem.

The hardware side of the same argument

A Systematization of Knowledge paper, arXiv:2609.30854, by Tejinder Singh of Dell Technologies, makes the case that long-context decode is bound by memory bandwidth rather than arithmetic. As summarised on jaehun.me on 3 October, the paper derives a crossover point where per-step KV cache bytes overtake model weight bytes. That threshold falls hyperbolically as batch size rises.

The numbers are concrete. Llama-3-70B in BF16 needs 140 GB of weights, more than the 80 GB of HBM on an H100. Its KV cache adds 42 GB for a single 128k-token sequence and 328 GB at one million tokens, more than four H100s hold in total.

The paper also notes that the same 2-bit quantization yields 1.25x at batch 1 and 4.82x at batch 32. The regime changed, not the algorithm. Gobstopper's 29% token cut and the paper's crossover curves are the same story told at different layers of the stack.

Postgres and the cost of getting memory wrong

Memory safety is the older, more literal version of this debate. ClickHouse engineer Philip Dubé wrote on 30 September about bringing C++ into Postgres extensions. There Postgres allocates through palloc and MemoryContext arenas and handles failure with setjmp/longjmp. C++ destructors do not survive that jump. An uncaught exception can force the postmaster to restart everything in case shared memory was corrupted.

ClickHouse's answer for pg_clickhouse was to remove C++ entirely, replacing clickhouse-cpp with a new C library. pg_re2 keeps C++ scoped to one wrapper file. pg_chdb moved libchdb into a separate helper executable.

The same pattern shows up in image decoding. The Halide team's wpd, a Rust WebP decoder with hand-written SIMD, was built in response to CVE-2023-4863. CISA confirmed that flaw was exploited in the wild, and it affected Chrome, Firefox, Signal and Microsoft Teams. The team cites Chromium's own figure that around 70% of its high-severity security bugs come from memory safety issues.

Smaller items, same direction

Elsewhere the week's releases were quieter. Kagi said on 3 October that it is ending development of Orion for Linux and Windows and open-sourcing both. The Linux beta receives no Kagi updates after 2 October 2026, and the planned late-2026 Windows launch is cancelled.

A developer published Warply, an unofficial one-button Cloudflare WARP client for Windows 10 and 11 that stores its WireGuard key with Windows DPAPI. Microsoft's Old New Thing blog ran a 1 October post on hot-patching under Windows on Itanium. LiberNT, a ReactOS fork aimed at ARM64 and Windows 11, was unreachable on 3 October because its Anubis proof-of-work challenge would not load.

None of these settle whether agents need memory or documentation. But the burden of proof has shifted. Anyone shipping a memory layer now has to show it beats a well-kept file, and the first serious benchmarks are only just arriving.

Comments 0

Sources

9
  1. 01Gobstopper: context compaction proxy for Claude Code and CodexEN
  2. 02Agents don't need memory, they need documentationEN
  3. 03The KV Cache Is the New Memory WallEN
  4. 04Memory safety for Postgres extensions in C/C++EN
  5. 05Memory-Safe WebP DecodingEN
  6. 06An Update on Orion for Linux and WindowsEN
  7. 07Warply: unofficial one-button Cloudflare WARP app for WindowsEN
  8. 08Windows on Itanium also provided for hot-patchingEN
  9. 09LiberNT: fork of ReactOS focused on ARM64 and Windows 11EN

All figures and quotations in this text come from the sources listed below.

Content prepared by the editorial team with AI assistance.

Rachel Nwosu

Rachel Nwosu

AI, models and technology

Rachel Nwosu covers AI, models and technology for FLASH24, working from public model documentation, benchmark releases and repository histories rather than press summaries, and she skips announcements that arrive without reproducible numbers. She checks training-data claims against dataset cards and reruns reported metrics where code is available. She spends much of her week interviewing researchers and engineers, tracking model launch calendars, and comparing vendor benchmarks with independent evaluations. Outside the desk she runs 3D printers, restores old computers, and tests how models learn from internet junk. She does not publish benchmark figures she cannot trace to a source.

Newsroom →

Comments

0
  1. No comments yet — be the first.

Write a comment

Comments are public. We do not publish abuse, spam or advertising.