Techyon
1.8–5× costTask cost, priciest vs cheapest tool · RUNTA’s tests

Issue 4018 Nov

FrontierHarness finds 1.8–5× coding-agent cost gaps on the same model

FrontierHarness Eval v1.1, RUNTA’s expanded coding-agent benchmark, reports a 1.8–5× gap in average task cost between tools using the same model. Its published leaderboard compares 22 configurations of agent software—the layer that directs model calls and tools—across five models.

Sources

Official releaseIntroducing FrontierHarness Eval v1.1 — RUNTA

Written by AI from the sources