Surge finds higher thinking effort beats coding-agent harness add-ons

Surge tested several coding-agent setups on software engineering tasks from its sudo L7 benchmark, holding the underlying model fixed at Opus 5.5 in Claude Code. The company said running vanilla Claude Code at xhigh thinking effort improved its score by 11.3 points over medium effort. That was larger than the gain from any individual harness configuration in the experiment.

Surge said the harness comparisons ran at medium thinking effort. Superpowers, a skills plugin that forces plan-and-test discipline before coding, added 2.4 points while raising cost by 10%. Claude Code's built-in /goal and /code-review plugins added 3.3 points at 1.8 times the cost. Compound Engineering, a plan, work and review workflow with specialized reviewer agents, cost eight times as much as vanilla and produced no statistically clear improvement.

Surge described the most effective harness additions as worth 2 to 3 points. The /goal and /code-review pair reached 3.3 points, the top harness result. The cost gap matters for teams that assemble reviewer agents, skills plugins and planning loops, often at several times the token cost of vanilla Claude Code. Surge's result points to a cheaper lever first: raise thinking effort before adding orchestration.

What extra thinking bought

At xhigh effort, the model often did more work to make its answer trustworthy, Surge said. Compared with medium effort, xhigh improved the sudo L7 rubric by 14.9 points on Verification and Thoroughness, 9.8 points on Persistence and 7.0 points on Thought Partnership.

Surge gave an example from a video-editing application. A code path that spliced two clips together would crash if the first clip was missing, and the user asked the agent to make that clip optional. At medium effort, the agent fixed the immediate cause and reported success. At xhigh effort, the agent kept investigating and found that the first fix would send execution down a fallback path that also assumed the first clip existed, so the application would still crash. On that task, an extra 29 cents made the difference between a broken fix and a working solution, according to Surge.

More agents did not mean better judgment

Surge's opening example showed the limit of orchestration. One harness with 37 additional skills spawned 17 agents: five reviewed the plan for feasibility, scope and security; five looked for ways to simplify the implementation; and seven reviewed the resulting code. None noticed that the confirmation email went out when someone registered, before they had paid. Following the user's instructions would have given unpaid registrants the Zoom passcode.

The benchmark is designed to test engineering judgment, not only correctness. Surge said sudo L7 tasks take place in real codebases, many licensed from operating companies, with prompts that are often underspecified. Expert-written rubrics evaluate functional correctness alongside verification, practical judgment and communication across eight dimensions. A passing test suite does not necessarily mean the agent did a good job.

What Surge recommends

Surge's practical advice is to try the simpler lever first. If a team is spending an afternoon engineering a complicated agent setup, it should ask whether it has tried higher thinking effort, the company said.