Skip to content
World clockEU--:--UK--:--USA--:--CN--:--PLDEFRIT中文EN

portal about AI and technologyevents · analysis · interviews · technical background

Search
LIVE
›

Trillium Labs Wants High-Stakes AI Research in the Open

Nathan Lambert and Tom Zick launched Trillium Labs on 3 October. The nonprofit will publish details of experiments on recursive self-improvement and AI agents so outside scientists can replicate them, WIRED reported.

AI & modelsExplainerRachel NwosuPublished: 3 October 20265 min readSources 9
Trillium Labs Wants High-Stakes AI Research in the Open
Nathan Lambert and Tom Zick launched Trillium Labs on 3 October, a nonprofit that will publish the details of

Nathan Lambert and Tom Zick launched Trillium Labs on 3 October. The nonprofit plans to work on recursive self-improvement and AI agents. It will publish the details of those experiments so outside scientists can replicate them, according to WIRED.

The launch lands in the middle of an argument about how much of the lab should be glass. On one side, frontier developers keep their strongest models behind an app or an API. On the other, a growing group wants the training runs, data and evaluations readable from the outside. Lambert, a former Ai2 and Hugging Face researcher, told WIRED that closed development is "taking us a step backwards". He said the scientific method is the better tool for reducing harm.

Zick previously worked at Harvard University. There she helped Charles Schwab devise policies around responsible AI. She said the nonprofit will initially focus on post-training, the fine-tuning done after a large model is built. The second area is RSI, where AI contributes to the research that produces new models.

Why the timing matters

The stakes are not abstract. WIRED notes that frontier models can automate the discovery of software vulnerabilities and probe and hack into systems. Recent high-profile hacking sprees have sharpened scrutiny of who gets access. Trillium's answer is disclosure rather than lockdown: publish enough of the method that other researchers can check the claims.

That is not the only open camp. WIRED points to Xiaomi, which recently published live details of a major training run involving one of its models. It also points to Stanford researchers pretraining a model called Marin in the open. Lambert also founded American Truly Open Models, an initiative aimed at pushing US companies to release more open models. The two met as UC Berkeley graduate students during the pandemic.

The lab's bet is that a shared understanding of risk beats a small trusted circle. Publishing experiment details on agents and self-improvement may make the risks smaller, or just more visible. Nobody in the article answers that question.

Meta's six papers

Meanwhile, Meta's AI research arm published a different model of openness on 3 October. It released six papers from a collaboration with mathematicians. They used Muse Spark 1.1 and 1.2 in Thinking Mode through the regular meta.ai chat interface, with no custom research scaffold. Five of the papers present answers to previously open research questions, according to the Meta AI Research blog. The company says the papers mark which passages were primarily drafted by researchers and which by AI. They also credit the earlier work they build on.

After completing our work, we learned that other teams outside Meta had independently announced solutions to some of the same problems using different approaches.

That sentence, from the blog, is the useful part. For the Gaussian ellipsoid fitting problem, Meta names three concurrent works: Misiakiewicz and Wen, de la Cerda, Potechin, Tulsiani and Xu, and Koehler and Sohn. The blog says the approaches differ and were developed independently. Meta's stated aim was not to mass-produce papers. It was to help researchers develop insights others can build on.

The retrieval problem underneath

If AI is going to help with research, it first has to find the right prior work. A benchmark posted on 3 October, ScholarCatalyst, measures exactly that. Its authors had 184 lead authors of 207 recent computer science papers label which candidates did or could have advanced their projects, each with a rationale. They then set a retrieval task restricted to literature available when each project began.

The results are modest. Agentic search scored 0.42 Recall@20 against 0.48 for plain embedding retrieval, even though the agent called the same retriever as a tool. An agent built on Claude Fable 5.1 reached only 0.51 R@20. That model may have seen the completed papers during training. The corpus held 191,000 papers.

That gap matters for claims about automated discovery. The authors argue that picking the paper which opens a half-formed idea is still a human skill. Current systems do not have it.

The workbench view

Away from the research frontier, the same pattern shows up at desk scale. A developer writing on 3 October described spending about four hours to scan and edit a 36-frame roll of 35mm film. He then handed the job to an agent. The first attempt had Claude drive the scanner over SANE with inverted X and Y scanbed directions and nearly broke the gear mechanism. The author kept VueScan for capture and automated the processing after it. Dust detection and colour correction are now handled by code and models. A blank leading frame serves as a reference for weighting colour across the roll.

None of this is laboratory automation in the sense the headlines use. It is the same trade, though: a person decides what is worth doing, and software takes the repetitive middle. The Academy of Natural Sciences story reads in reverse. Drexel University announced on Tuesday that the museum will close on 30 September. It has been open since 1828 and holds more than 19 million specimens. Its last day of regular hours is Sunday, 27 September. Officials cited attendance that never recovered from the pandemic. The collections stay available for research.

Trillium Labs is the newest test of the open approach, and the least proven. Its first publications will show whether transparency about agents and self-improvement produces scrutiny or just more material to argue over.

Comments 0

Sources

9
  1. 01AI Experts Want to Do High-Stakes Research Out in the OpenEN
  2. 02Solving Open Research Problems TogetherEN
  3. 03ScholarCatalyst: A Benchmark for Retrieving Papers That Inspire New ResearchEN
  4. 04Automating my 35mm film scanning pipelineEN
  5. 05Academy of Natural Sciences is shutting down its museum after nearly 200 yearsEN
  6. 06Some topics are off limits inside popular Chinese-made free AI, researchers findEN
  7. 07EventMaxxer – Automatically apply to events to get you in the right roomEN
  8. 08Swiss scientist soils underpants, wins silly science prizeEN
  9. 09PL research is dead, the age of PL exploration is just beginningEN

All figures and quotations in this text come from the sources listed below.

Content prepared by the editorial team with AI assistance.

Rachel Nwosu

Rachel Nwosu

AI, models and technology

Rachel Nwosu covers AI, models and technology for FLASH24, working from public model documentation, benchmark releases and repository histories rather than press summaries, and she skips announcements that arrive without reproducible numbers. She checks training-data claims against dataset cards and reruns reported metrics where code is available. She spends much of her week interviewing researchers and engineers, tracking model launch calendars, and comparing vendor benchmarks with independent evaluations. Outside the desk she runs 3D printers, restores old computers, and tests how models learn from internet junk. She does not publish benchmark figures she cannot trace to a source.

Newsroom →

Comments

0
  1. No comments yet — be the first.

Write a comment

Comments are public. We do not publish abuse, spam or advertising.