Benchmarks

Opus 5 ties Sol, Terra matches Fable at a quarter the price

Sol, Terra, Opus 5 and Fable 5 on the same agent: a dead heat at the top, the same score at a quarter of the cost, and effort settings that can hurt.

On this page
  1. The lineup: prices, windows and effort dials
  2. What does more effort actually buy?
  3. Who wins when the agent is identical?
  4. Would a maintainer actually merge it?
  5. The specialist boards: comprehension, tests, refactoring
  6. Terminals, VMs and the hard tail
  7. How should you read a 2026 leaderboard?
  8. Which model gets which job?

No single model wins software development in July 2026. On Datacurve’s DeepSWE, the one major board that runs every model through the same agent, Claude Opus 5 posts 74% and GPT-5.6 Sol 73%, with overlapping error bars [1]. GPT-5.6 Terra matches Claude Fable 5’s 70% at under a quarter of the cost [1]. What separates the four is profile, not rank.

I run the Claude models and Sol against real repositories every week; Terra I know only through the boards below. So this is the comparison I wanted someone to write for me: the two strongest ChatGPT models against the two strongest Claude models, benchmark by benchmark, including the part launch threads skip, what each number is allowed to mean.

The lineup: prices, windows and effort dials

The four models split cleanly by role. OpenAI routes the plain gpt-5.6 alias to Sol, “the model for flagship capability”, and positions Terra as the model that “balances intelligence and cost” [2] [3]. Anthropic’s docs pitch Opus 5 at complex agentic coding and enterprise work, and Fable 5, the top of the range, at long-running agents [4].

The list prices tell the same story. Sol costs $5 per million input tokens and $30 for output; Terra exactly half, $2.50 and $15 [2]. Opus 5 sits at $5 and $25, Fable 5 at $10 and $50 [5]. Anthropic’s premium keeps going up from there: the pricing page lists a fast mode for Opus 5, in research preview, at exactly Fable’s rates [5].

Spec SolTerraOpus 5Fable 5
Input, $/MTok 5.00 2.50 (Best value in this row) 5.00 10.00
Output, $/MTok 30 15 (Best value in this row) 25 50
Cache read, $/MTok 0.50 0.25 (Best value in this row) 0.50 1.00
Context window 1.05M 1.05M 1M 1M
Effort levels 6, none to max 6, none to max 5, low to max 5, low to max
Figure 1. List prices and specs for the four models. OpenAI and Anthropic developer docs, July 2026.

The table’s one-liner: Terra is the only discount in the room, and the context windows are close enough to identical that they stop being a differentiator [2] [4].

The effort dials deserve more attention than the prices. GPT-5.6 takes six values, none through max, and OpenAI’s guide calls medium the balanced starting point [3]. The Claude models take five, low through max, default to high, and Anthropic tells you to step Opus 5 up to xhigh for demanding coding and agentic work, and to reserve max for tasks that justify unconstrained token spending [6]. Nothing guarantees that OpenAI’s high and Anthropic’s high buy comparable compute. Effort names are per-vendor knobs, not units.

What does more effort actually buy?

Ten to fifteen points of measured intelligence, for eight to sixteen times the tokens. Artificial Analysis scores Opus 5 from 51 at low effort to 61 at max, Sol from 49 to 59, and Terra from 40 to 55 on its Intelligence Index [9] [10] [11].

That index, v4.1, blends nine evaluations, from agentic terminal work and real-world task completion to GPQA Diamond and long-context reasoning, weighted 34% agents, 24% coding, 24% scientific reasoning and 18% general [12]. It is the best public picture of the effort curve, because the same suite ran at every setting.

  • Opus 5
  • Sol
  • Terra
Intelligence Index by effort level for Opus 5, Sol and Terra Opus 5 leads at every setting, Sol tracks two points behind, Terra trails but climbs fastest from 40 to 55. 30 40 50 60 70 low medium high xhigh max
Show the data as a table
Effort Opus 5SolTerra
low 514940
medium 565446
high 595649
xhigh 605852
max 615955
Figure 2. Intelligence Index v4.1 by reasoning effort. Artificial Analysis, pulled July 30, 2026.

Two things stand out in figure 2. The curves barely cross, so the pecking order survives every setting. And the top is flat: Opus 5’s printed step from 60 to 61 is 0.62 unrounded index points, bought by raising the suite’s output bill from 76 million to 100 million tokens [9].

The token bills are the real curve. Opus 5 spent 12M output tokens on the suite at low and 100M at max; Sol went from 6.6M to 70M; Terra from 5.9M to 96M [9] [10] [11]. Terra’s last step is the expensive one: xhigh to max nearly triples its tokens for three index points [11].

Fable 5 has no public effort sweep. Artificial Analysis lists exactly one configuration, max, at 60, one rounded point under Opus 5, generated with 87M tokens [13]. Both Anthropic flagships ran with server-side fallback to Opus 4.8 for the prompts their safety layer stops [13] [14]. AA’s launch verdict on Opus 5 doubles as this section’s summary: Fable-level intelligence at $2.03 per index task against Fable’s $2.75 [14].

Who wins when the agent is identical?

Nobody, and that is the finding. DeepSWE runs all 18 models through the same mini-swe-agent, and at max effort Opus 5 solves 74% ± 4 of tasks while Sol solves 73% ± 3 [1]. The intervals overlap heavily. Fable 5 and Terra land level at 70% [1].

DeepSWE is the comparison I trust most because it removes the harness variable. Its 113 tasks span 91 repositories and five languages, are written from scratch so no model saw a solution during training, and are graded by hand-written verifiers that test behavior rather than implementation details [1]. Prompts run about half the length of SWE-bench Pro’s while solutions need 5.5 times more code [1].

Measurement Opus 5SolFable 5Terra
Tasks solved, % 74 73 70 70
Cost per task, $ 11.84 8.39 21.63 4.95 (Best value in this row)
Output tokens, K 118 60 (Best value in this row) 119 72
Agent steps 99 61 (Best value in this row) 88 76
Figure 3. DeepSWE v1.1 at max effort, same mini-swe-agent for every model. Datacurve, updated July 25, 2026.

The solved row in figure 3 is deliberately unmarked: 74 ± 4 against 73 ± 3 is a tie, and Datacurve itself warns that top configurations “often overlap on confidence intervals” [1]. The other rows separate cleanly. Sol reached the same outcome with half the output tokens, in 61 steps against Opus 5’s 99, at $8.39 per task against $11.84 [1]. Opus investigates longer. Sol ships sooner.

Terra is the row that reprices the market: the same 70% as Fable 5 at $4.95 per attempted task against $21.63 [1]. Divide cost per attempt by the solve rate, the optimistic floor I used on the Kimi K3 numbers, and a solved task costs $7.07 on Terra and $30.90 on Fable.

tasks solved at max effort
70%
level with Fable 5 max
measured, per attempted task
$4.95
Fable 5 max: $21.63
derived, per solved task
$7.07
cheapest of the four at max
Figure 4. What Terra's DeepSWE row means in money. Measured costs from Datacurve; the per-solved figure is my arithmetic.

Max is rarely the rational setting on this board anyway. Opus 5 at high effort holds 73% ± 2 at $6.08, matching Sol max for less money; Sol at xhigh holds 71% ± 1 at $4.70, the tightest interval near the top [1]. Fable never beats its cheaper self here: xhigh scores the same 70% as max at $13.41 instead of $21.63 [1]. Datacurve’s own cost-versus-score frontier tags the efficient configurations as Opus 5 at max, Fable 5 at high and Sol at medium [1].

Would a maintainer actually merge it?

Cognition’s FrontierCode asks exactly that, and it reorders the field. Tasks are written by maintainers of the repositories under test, runs that consult solution-bearing sources score zero, and grading covers correctness, test quality, scope and codebase conventions [15]. The best configurations: Fable 5 xhigh at 53.5, Opus 5 medium at 53.4, Sol max at 47.5, Terra max at 41.3 [15].

Read that list again. Opus 5’s best patch quality comes from its second-lowest effort setting.

  • Fable 5
  • Opus 5
  • Sol
FrontierCode score by effort level for Fable 5, Opus 5 and Sol Sol rises with every effort step to 47.5, Fable 5 peaks at xhigh with 53.5, Opus 5 peaks at medium with 53.4 and drops to 43.6 at xhigh. 30 40 50 60 low medium high xhigh max
Show the data as a table
Effort Fable 5Opus 5Sol
low 48.041.935.4
medium 49.853.439.9
high 52.748.045.1
xhigh 53.543.646.8
max 51.648.047.5
Figure 5. FrontierCode 1.1 main set by reasoning effort. Cognition leaderboard data, July 2026.

Figure 5 is the most useful chart I found all week. Sol climbs with every step of the dial, 35.4 to 47.5, so more effort reliably buys Sol a better patch [15]. Fable peaks at xhigh and gives 1.9 points back at max [15]. Opus 5 peaks at medium with 53.4, falls to 43.6 at xhigh, and only partially recovers at max [15]. Anthropic’s system card prints the same curve and states the anchor itself: best main-set score at medium [16].

The mechanism is visible in the grading. FrontierCode scores scope discipline [15] and checks for prohibited implementation patterns [16], and higher effort makes a model explore more, touch more, expand more. On an open research bench that is the point. Inside a scoped pull request it reads as scope creep. It is the benchmark version of what I wrote when Opus 5 ran my week: a great employee whose manager keeps assigning themselves extra work.

Two footnotes before anyone crowns Fable. The 0.1-point gap over Opus 5 medium spans five runs per configuration and identical 58.9% pass rates, so it is noise; the costs are not, at $13.09 per rollout for Fable xhigh against $4.30 for Opus medium [15]. And even Sol’s best sits under Fable’s worst: 47.5 against 48.0 at low [15]. On merge-quality grading the Claude stack leads outright, with the standing caveat that Claude ran in Claude Code and GPT-5.6 in Codex [15].

The specialist boards: comprehension, tests, refactoring

Scale’s SWE Atlas family measures what a solve rate hides, and it is where the four profiles separate hardest.

Codebase QnA hands the agent a real repository in a container and 124 questions that require running the software and tracing execution, weighted toward architecture (35%) and root-cause analysis (30%) [17]. Scale Labs scores Sol at xhigh in Codex at 46.0 ± 5.0, Fable 5 at 39.0 ± 5.0 in Claude Code, and notes both models refused benign questions that triggered security filters, with refusals scored as zero [17]. Neither Opus 5 nor Terra has a row. The whole board is topped by the previous generation: Opus 4.8 at 57.26 [17].

Run by a different lab, the same 124 questions flip. Inside Artificial Analysis’ Coding Agent Index, Opus 5 xhigh posts the best SWE-Atlas-QnA component score at 54.8, ahead of Fable’s 48.9, with Sol max at 43.3 [18]. Same task set, different runs, opposite ordering. Hold that thought for the next section.

Test Writing is the clever one: 90 tasks where the generated tests must pass against the real implementation, then fail once the relevant code is mutated away, so vacuous tests score nothing [19]. Fable 5 leads at 55.6 ± 5.8 against Sol’s 45.9 ± 6.0, though the board’s interval-based ranking still lists both at rank 1 [19].

  • Fable 5
  • Sol
SWE Atlas Codebase QnA and Test Writing scores Sol leads codebase comprehension 46.0 to 39.0, Fable 5 leads test writing 55.6 to 45.9. Codebase QnA 39.0 46.0 Test writing 55.6 45.9
Show the data as a table
Benchmark Fable 5Sol
Codebase QnA 39.046.0
Test writing 55.645.9
Figure 6. SWE Atlas scores, Fable 5 (Claude Code, xhigh) against Sol (Codex, xhigh). Scale Labs, July 28, 2026.

The two boards in figure 6 point in opposite directions, and that is the profile in miniature: on Scale’s run Sol reads an unfamiliar system a little better, while Fable tests it meaningfully better.

Refactoring is Fable’s clearest win: 54.76 ± 6.76, top of a board whose 70 tasks cover decomposition, interface evolution, extraction and relocation, and whose graders check for regressions, broken call sites, circular dependencies and leftover dead code [20]. No Opus 5, Sol or Terra row exists; the best GPT configuration listed is the previous-generation GPT-5.5 at 44.79 [20]. A missing row is not a zero. But the standing result belongs to Fable.

Terminals, VMs and the hard tail

Terminal-Bench 2.1 measures whether a model can drive a shell to a verified end state, and the top is a photo finish. On Artificial Analysis’ independent run, Sol at xhigh leads with 89.5%, Opus 5 max posts 89.1%, and three configurations, Sol max, Terra max and Opus 5 xhigh, land on an identical 88.0% [21]. With 89 tasks and three repeats [12], a single flipped trial moves a score by about 0.4 points. Fable 5 trails at 84.6% [21].

Terminal-Bench v2.1 scores, best configuration per model Sol xhigh leads at 89.5 percent, Opus 5 max at 89.1, Terra max at 88.0, Fable 5 max at 84.6. Sol xhigh 89.5% Opus 5 max 89.1% Terra max 88.0% Fable 5 max 84.6%
Show the data as a table
Configuration Value
Sol xhigh 89.5%
Opus 5 max 89.1%
Terra max 88.0%
Fable 5 max 84.6%
Figure 7. Terminal-Bench v2.1, best configuration per model. Artificial Analysis, pulled July 30, 2026.

Figure 7’s takeaway: the terminal crown is a rounding error shared by three models, and Fable is the only one visibly behind. Note that Terra hangs with the flagships here, at $2.50 per million input tokens [2] [21]. OpenAI’s own launch table tells a similar story with different digits, Sol 88.8%, Terra 87.4%, Fable 5 83.1% [7], and the disagreement between self-reported and independent runs is itself about the size of the gaps being argued over.

FrontierBench v0.1 is the successor, built by the same team: 74 harder tasks leaning into computational biology, physics simulation, CAD, formal proofs and GPU performance work [16]. Anthropic ran it on mini-swe-agent and reports Opus 5 at 44.4% at xhigh effort, its best, with max within noise at 43% and high reaching 39% on 19% fewer output tokens; Sol lands at 37.5% and Fable 5 at 33.7%, both at max [16]. On genuinely open problems, Opus 5 leads the four outright.

Two adjacent boards round out the agentic tail. On OSWorld 2.0, where the model drives a live Ubuntu VM with mouse and keyboard for up to 500 actions per task, Opus 5 scores 70.6%, Fable 5 66.1% and Sol 62.6%, Sol’s figure taken from OpenAI’s own release post [16]. On Zapier’s AutomationBench, a simulated company with 47 apps and layered business policies, Opus 5 max scores 26.0% and Opus 5 medium 24% at $0.89 per task, against Sol’s 18.1% and Fable’s 17.4% [16]. The messier and more tool-bound the environment, the further Opus pulls ahead. And medium keeps embarrassing max.

How should you read a 2026 leaderboard?

As a measurement of one model wearing one agent at one effort setting under one grader. Change any of those and the number moves, sometimes by more than the distance between first and fourth place. Five habits keep the reading honest.

The habitThe exhibit in this comparison
Ask which harness ranSWE-bench Pro reads Fable 80.0, Opus 5 79.2, Sol 64.6, each vendor’s own agent [16]; DeepSWE’s shared agent compresses the same models into four points [1]
Respect the error bars74 ± 4 against 73 ± 3 [1]; three exact ties at 88.0% [21]; a 0.1-point lead across five runs [15]
Notice saturationOpus 5 scores 96.0% on SWE-bench Verified [16]; a board with no headroom ranks nothing
Check the versionOpenAI’s launch page cites Coding Agent Index v1.1 with Sol at 80 [7]; the live v1.3 scores Sol max at 67 [18]
Missing is not zeroOpus 5 has no row on any of Scale’s three boards [17] [19] [20]; Terra barely exists outside DeepSWE and FrontierCode

The version row deserves one more sentence, because it is the sneakiest. A v1.1 score of 80 and a v1.3 score of 67 describe the same model family on the same-named index and are not comparable at all; DeepSWE similarly rescored GPT-5.5 xhigh from 70% to 67% between its v1 and v1.1 sets [1]. A benchmark name without a version number is a vibe.

Even primary sources wobble. The Opus 5 system card’s summary table tags Sol’s FrontierBench run as Codex while its methods text says mini-swe-agent on the same infrastructure, and the card does not resolve the contradiction [16]. Its SWE-bench Multimodal win, 59.4 against Fable’s 54.1, comes from an internal harness modified from the public one [16]. Scale grants newer mini-swe-agent rows 500 steps where earlier runs got 250, and observes that frontier models simply perform better on their native scaffolds [17]. None of this is fraud. It is what measuring a moving system looks like, and it is why the Kimi K3 and GLM-5.2 hype check reached the same conclusion from different data: capability claims survive scrutiny, comparability claims mostly do not.

Which model gets which job?

Terra for routine volume, Sol for known goals, Opus 5 for unknown problems, Fable 5 for changes that must merge clean. That is what the profiles above add up to, and the effort setting matters as much as the model name.

The workMy pickEffort
Routine endpoints, mappings, small tests, configTerramedium or high
A scoped pull request with clear acceptance criteriaOpus 5medium
Implementing a known plan, fixing a reproducible bugSolhigh
Root-causing an unfamiliar failureOpus 5high or xhigh
Test strategy for a feature that mattersFable 5xhigh
Repo-wide refactoring and interface migrationsFable 5xhigh
Long terminal and debugging sessionsSol or Opus 5high or xhigh
Open-ended engineering with no known solutionOpus 5xhigh

None of these boards ran my stack, and none ran yours, so the table is an inference from profiles, not a measurement. The effort rules are measured, though. Sol is the only model of the four whose merge quality rose with every step of the dial [15]. Fable tops out at xhigh on the boards it leads [15] [19]. Opus 5 wants the dial matched to the job: medium for a scoped change [15], xhigh when the problem is genuinely open [16]. Both vendors’ own docs agree with their curves: OpenAI calls medium the balanced start [3], Anthropic defaults to high and frames max as the exception that must justify its token bill [6].

Three results would redraw this table: an Opus 5 row on Scale’s boards, a public effort sweep for Fable 5, and any of these four measured in the other vendor’s harness. Until then, buy the profile, not the rank. The most expensive model in this comparison is not the best at most jobs, the cheapest one is embarrassingly close on half of them, and the single most valuable setting change costs nothing at all: turning the effort dial down.

Sources

  1. DeepSWE leaderboardDatacurve · 2026-07-25
  2. API modelsOpenAI Developers
  3. Latest model guideOpenAI Developers
  4. Models overviewClaude Platform Docs
  5. PricingClaude Platform Docs
  6. EffortClaude Platform Docs
  7. GPT-5.6OpenAI
  8. Codex modelsChatGPT Docs
  9. Claude Opus 5 analysisArtificial Analysis
  10. GPT-5.6 Sol analysisArtificial Analysis
  11. GPT-5.6 Terra analysisArtificial Analysis
  12. Intelligence benchmarking methodologyArtificial Analysis
  13. Claude Fable 5 analysisArtificial Analysis
  14. Opus 5: Fable 5 level intelligence at a lower cost per taskArtificial Analysis · 2026-07-24
  15. FrontierCode leaderboardCognition · 2026-07-07
  16. Claude Opus 5 system cardAnthropic on alphaXiv · 2026-07-24
  17. SWE Atlas: Codebase QnAScale Labs · 2026-07-28
  18. Coding Agent IndexArtificial Analysis
  19. SWE Atlas: Test WritingScale Labs · 2026-07-28
  20. SWE Atlas: RefactoringScale Labs
  21. Terminal-Bench v2.1 evaluationArtificial Analysis