Surge测试发现:提高思考强度胜过给编程智能体加装外挂框架
Surge表示,把Claude Code的思考强度调到xhigh后,其sudo L7得分提高了11.3分,超过它测试过的任何外挂框架;Superpowers增加2.4分,/goal加/code-review增加3.3分。

Surge在其sudo L7基准的软件工程任务上测试了多套编程智能体配置,底层模型统一固定为Claude Code中的Opus 5.5。该公司称,在原版Claude Code中把思考强度设为xhigh,得分比medium强度高出11.3分。这一提升幅度超过实验中任何一种单一框架配置带来的增益。
Surge表示,各框架的对比测试都在medium思考强度下进行。Superpowers是一款技能插件,要求在写代码前强制走计划与测试流程,它带来2.4分提升,成本上升10%。Claude Code内置的/goal和/code-review插件带来3.3分提升,成本是原版的1.8倍。Compound Engineering是一套包含计划、执行和审查的工作流,配有专门的审查智能体,它的成本是原版的八倍,却没有产生统计上明确的改进。
Surge把最有效的框架增益描述为2到3分。/goal与/code-review这一组合达到3.3分,是所有框架配置中的最好成绩。对需要拼装审查智能体、技能插件和规划循环的团队来说,成本差距很重要,这类配置的token开销常常是原版Claude Code的好几倍。Surge的结果指向一个更便宜的杠杆:先提高思考强度,再考虑加编排。
多花的思考换来了什么
Surge称,在xhigh强度下,模型常常为了让答案更可靠而做更多工作。与medium强度相比,xhigh在sudo L7评分表的“验证与周全性”一项上提高14.9分,“坚持性”提高9.8分,“思考伙伴”提高7.0分。
Surge举了一个视频剪辑应用的例子。一段把两个片段拼接起来的代码路径,在第一个片段缺失时会崩溃,用户要求智能体把这个片段改成可选。在medium强度下,智能体修好了直接原因并报告成功。在xhigh强度下,智能体继续追查,发现第一个修复会把执行引向一条同样假定第一个片段存在的回退路径,应用仍然会崩溃。Surge称,在这个任务上,多花29美分决定了修复是坏的还是真能用的。
智能体更多,判断力并未更好
Surge开篇的例子说明了编排的极限。一套配置额外加载了37项技能,启动了17个智能体:五个审查计划的可行性、范围与安全,五个寻找简化实现的办法,七个审查产出的代码。没有一个注意到确认邮件在用户注册时就发出,而那时用户还没付款。照用户的指令执行,会把Zoom通行码发给未付款的注册者。
该基准的设计目标是考察工程判断力,而不只是正确性。Surge称,sudo L7的任务都在真实代码库中进行,其中许多代码库由运营公司授权,提示词往往写得不够明确。专家编写的评分标准从八个维度评估功能正确性,以及验证、实际判断和沟通。测试套件通过,并不一定说明智能体干得好。
Surge的建议
Surge给出的实际建议是先试更简单的杠杆。该公司称,如果团队正在花一下午搭建复杂的智能体配置,应该先问问自己有没有试过提高思考强度。
资料来源
25- 01We Tested Coding Agent Harnesses. Thinking Harder Beat All of ThemEN
- 02New open-source benchmark reveals reasoning as the biggest bottleneck for AI in cloud operationsEN
- 03Which coding-agent CLI is best for headless workflow automation?EN
- 04iOS Simulator MCP benchmark: eight tools for AI agentsEN
- 05Does a code knowledge graph help coding agents? 360 trials found no clear gainEN
- 06Open models can do 90% of agentic work at a twentieth of the cost of a frontier LLM: GartnerEN
- 07Stop teaching leadership. Start letting people practise itEN
- 08Anthropic's text watermark is easy to bypass on several older Claude modelsEN
- 09Pangram’s new models spot AI fakes in images and video, it saysEN
- 10GPT-6 Astra sees the matrix and reprograms other AI models directly through their weightsEN
- 11Xpeng unveils GroundingPI pre-trained model, outperforming GPT-6 Astra in perception benchmarksEN
- 12Nvidia is happy to take your money whether you're using open or closed models, says Jensen HuangEN
- 13New Research Tests Whether AI Models Know Their Own ThoughtsEN
- 14Raising the quality of insights in agentic analyticsEN
- 15Leading European Financial Services Organizations Embrace Open Models to Build Proprietary IntelligenceEN
- 16Google Launches LiteRT Tensor API for Building, Customizing, and Optimizing ML Models for On-DeviceEN
- 17Was I wrong about open models & cyber risks?EN
- 18Cities Worldwide Build and Scale AI Services With NVIDIA Physical and Agentic AIEN
- 19Article: Systemic Approach to Memory, Knowledge, and Context in Agentic AI ArchitecturesEN
- 20Free AI models suddenly look much worse: leaderboard update delivers a massive rankings hitEN
- 21Launch HN: Momentic (YC W24) – Agents that catch bugs by using your appEN
- 22Artificial Analysis v5 and the benchmarker's dilemmaEN
- 23NVIDIA Cosmos Opens New Tools to Train, Evaluate and Deploy World ModelsEN
- 24Hiding Capabilities in AI ModelsEN
- 25NVIDIA Open Models Power a New Wave of European Public Sector ProjectsEN
本文所有数字与引语均来自下列来源,未添加来源之外的内容。
本内容由编辑部在人工智能辅助下制作完成。
评论
0- 还没有评论——来说两句吧。