Shortlistas of
As of 5 November 2026, no option is proven ahead. 3 options of 57 cannot be separated with the evidence in hand: Claude Opus 5.5 (Max), Gemini 4 Carbon (High) and Claude Sonnet 5.5 (Max).
- Options
- 57
- Measured
- 57
- Shortlist
- 3
- Plotted
- 53
- Priced
- 53
The job
Run an AI agent on real software tasks and judge it by how much the agent's own work was kept and corrected, against what each task costs.
Ranking
57 options measured, shortlisted rows shaded
| # | Option | Score 0 to 100 | Cost per task (median) | Interval | Price page |
|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 (Max)anthropicFrontierShortlist | 90.1 | $6.82 | ci9585.8 to 94.3 | |
| 2 | Gemini 4 Carbon (High)googleFrontierShortlist | 83.1 | $1.73 | ci9581.5 to 84.7 | |
| 3 | Claude Sonnet 5.5 (Max)anthropicShortlist | 79.2 | $3.94 | ci9573.7 to 84.8 | |
| 4 | GPT-6 Astra (Max)openai | 77.4 | $6.09 | ci9573.3 to 81.4 | |
| 5 | Claude Fable 5.1 (Max)anthropic | 76.3 | $6.95 | ci9572.5 to 80.0 | |
| 6 | GPT-6.1 Sol (Max)openaiFrontier | 74.5 | $0.955 | ci9572.3 to 76.7 | |
| 7 | Gemini 4 Argon (High)google | 71.2 | $0.999 | ci9569.5 to 72.9 | |
| 8 | GPT-6.1 Terra (Max)openaiFrontier | 70.4 | $0.788 | ci9568.0 to 72.7 | |
| 9 | Muse Wonder Full (Max)meta | 69.3 | Not priced | ci9567.3 to 71.2 | |
| 10 | Claude Fable 5 (High)anthropic | 65.6 | $3.92 | ci9563.3 to 67.8 | |
| 11 | Claude Opus 5 (Max)anthropic | 65.2 | $5.62 | ci9562.7 to 67.7 | |
| 12 | GPT-6 Sol (Max)openai | 63.9 | $1.74 | ci9560.7 to 67.1 | |
| 13 | Kimi K3.1 Pro (Max)moonshot | 59.4 | $1.47 | ci9556.8 to 62.0 | |
| 14 | GPT-5.6 Sol (xHigh)openai | 59.1 | $1.73 | ci9557.0 to 61.2 | |
| 15 | Claude Sonnet 5 (High)anthropic | 55.7 | $1.11 | ci9553.3 to 58.1 | |
| 16 | Grok 4.7 (xHigh)xai | 55.3 | $2.61 | ci9552.4 to 58.2 | |
| 17 | Gemini 3.8 Flash (High)googleFrontier | 55.1 | $0.758 | ci9553.5 to 56.8 | |
| 18 | Muse Wonder (Max)meta | 55.0 | $1.43 | ci9552.3 to 57.8 | |
| 19 | Muse Spark 1.3 (Max)metaFrontier | 54.9 | $0.449 | ci9553.2 to 56.5 | |
| 20 | MiMo-V2.6-ProxiaomiFrontier | 54.9 | $0.156 | ci9553.1 to 56.7 | |
| 21 | DeepSeek-V4.1-Flash-20261017 (Max)deepseek | 54.3 | $0.177 | ci9553.1 to 55.4 | |
| 22 | GPT-6 Luna (Max)openaiFrontier | 53.8 | $0.0914 | ci9553.0 to 54.7 | |
| 23 | GLM-5.3 (Max)zai | 52.3 | $1.14 | ci9550.7 to 53.9 | |
| 24 | Hy4 previewtencent | 51.9 | $0.444 | ci9550.5 to 53.3 | |
| 25 | Kimi K3 (Max)moonshot | 51.7 | $1.12 | ci9550.6 to 52.9 | |
| 26 | GLM-5.2 (Max)zai | 51.5 | $1.09 | ci9550.0 to 53.0 | |
| 27 | Claude Haiku 5.5 (High)anthropic | 51.3 | $0.0964 | ci9549.3 to 53.4 | |
| 28 | MiMo-V2.6-FlashxiaomiFrontier | 50.9 | $0.0713 | ci9548.8 to 53.0 | |
| 29 | Qwen3.8-Maxalibaba | 50.8 | $1.35 | ci9549.5 to 52.1 | |
| 30 | Grok 4.6 (xHigh)xai | 50.6 | $1.69 | ci9548.4 to 52.9 | |
| 31 | DeepSeek-V4-Pro (High, 20260813)deepseek | 49.4 | Not priced | ci9546.0 to 52.8 | |
| 32 | GPT-5.6 Terra (xHigh)openai | 47.0 | $0.606 | ci9545.4 to 48.5 | |
| 33 | Gemini 3.7 Flash (High)google | 46.7 | $0.589 | ci9545.0 to 48.5 | |
| 34 | GPT-6 Luna (Medium)openaiFrontier | 46.1 | $0.0270 | ci9545.0 to 47.3 | |
| 35 | GLM-5.3-Flashzai | 45.0 | $0.160 | ci9543.6 to 46.4 | |
| 36 | Step 5 Previewstepfun | 44.9 | $0.631 | ci9542.1 to 47.7 | |
| 37 | MiniMax-M3.1-Flashminimax | 43.6 | $0.242 | ci9540.4 to 46.7 | |
| 38 | Qwen3.8-27Balibaba | 41.4 | $0.430 | ci9539.7 to 43.1 | |
| 39 | Qwen3.8-Flash-Nextalibaba | 41.3 | $0.221 | ci9539.9 to 42.8 | |
| 40 | Gemini 3.1 Pro Previewgoogle | 40.8 | $0.633 | ci9538.0 to 43.5 | |
| 41 | Beam 1.0 501B A23Breflection | 40.6 | Not priced | ci9535.5 to 45.7 | |
| 42 | GPT-6 Luna (Low)openaiFrontier | 38.3 | $0.0105 | ci9536.8 to 39.7 | |
| 43 | Mistral Large 4mistral | 37.2 | $0.446 | ci9533.0 to 41.4 | |
| 44 | Qwen3.7-Maxalibaba | 36.7 | $0.353 | ci9534.3 to 39.1 | |
| 45 | Qwen3.7-Plusalibaba | 35.3 | $0.141 | ci9532.0 to 38.5 | |
| 46 | Hy3tencent | 34.9 | $0.0733 | ci9532.1 to 37.8 | |
| 47 | Muse Spark 1.1meta | 34.6 | $0.372 | ci9533.3 to 36.0 | |
| 48 | Muse Spark 1.2 (xHigh)meta | 34.5 | $0.439 | ci9532.8 to 36.2 | |
| 49 | MiMo-V2.5-Proxiaomi | 32.0 | $0.0768 | ci9529.9 to 34.0 | |
| 50 | Spark-X2.6synlan | 30.0 | $0.0978 | ci9527.0 to 33.0 | |
| 51 | MiniMax-M3minimax | 29.2 | $0.235 | ci9526.8 to 31.5 | |
| 52 | Mistral Medium 3.5mistral | 23.6 | $0.470 | ci9520.2 to 27.0 | |
| 53 | Nemotron 3 Ultranvidia | 18.8 | $0.150 | ci9512.4 to 25.2 | |
| 54 | NVIDIA-Nemotron-3.5-Super-VL-120B-A12B (NVFP4)nvidia | 18.2 | Not priced | ci9516.2 to 20.1 | |
| 55 | Inklingthinky | 17.8 | $0.332 | ci9514.9 to 20.8 | |
| 56 | Inkling-Smallthinky | 16.2 | $0.188 | ci9512.5 to 19.9 | |
| 57 | Solar Pro 4upstage | 8.1 | $0.0536 | ci953.9 to 12.3 |
Order is by score, highest first. Intervals overlap, so neighbouring ranks do not differ with certainty. Cost is in USD per task (median); bars start at zero. Interval: ci95, built from 95% confidence intervals given by the source.
Price against performance
Score against median cost per task, log scale
- Frontier: no other option is both cheaper and scores higher
- Other option
- Ring: on the shortlist, not separable with certainty
- Line: score interval, ci95 (built from 95% confidence intervals given by the source)
4 options not plotted: a coordinate is unknown
- Beam 1.0 501B A23B: Cost unknown
- DeepSeek-V4-Pro (High, 20260813): Cost unknown
- Muse Wonder Full (Max): Cost unknown
- NVIDIA-Nemotron-3.5-Super-VL-120B-A12B (NVFP4): Cost unknown
Cost for this job
Date read and source for all 57 options
Median cost per task, in USD per task (median), cheapest first within each group.
Workload and assumptions
One task. Cost is the median cost per task that Arena reports for the model.
| Option | Cost | Price read | Source | Notes |
|---|---|---|---|---|
| Cost 53 | ||||
GPT-6 Luna (Low)openai | $0.0105 | 2 days old | ||
GPT-6 Luna (Medium)openai | $0.0270 | 2 days old | ||
Solar Pro 4upstage | $0.0536 | 2 days old | ||
MiMo-V2.6-Flashxiaomi | $0.0713 | 2 days old | ||
Hy3tencent | $0.0733 | 2 days old | ||
MiMo-V2.5-Proxiaomi | $0.0768 | 2 days old | ||
GPT-6 Luna (Max)openai | $0.0914 | 2 days old | ||
Claude Haiku 5.5 (High)anthropic | $0.0964 | 2 days old | ||
Spark-X2.6synlan | $0.0978 | 2 days old | ||
Qwen3.7-Plusalibaba | $0.141 | 2 days old | ||
Nemotron 3 Ultranvidia | $0.150 | 2 days old | ||
MiMo-V2.6-Proxiaomi | $0.156 | 2 days old | ||
GLM-5.3-Flashzai | $0.160 | 2 days old | ||
DeepSeek-V4.1-Flash-20261017 (Max)deepseek | $0.177 | 2 days old | ||
Inkling-Smallthinky | $0.188 | 2 days old | ||
Qwen3.8-Flash-Nextalibaba | $0.221 | 2 days old | ||
MiniMax-M3minimax | $0.235 | 2 days old | ||
MiniMax-M3.1-Flashminimax | $0.242 | 2 days old | ||
Inklingthinky | $0.332 | 2 days old | ||
Qwen3.7-Maxalibaba | $0.353 | 2 days old | ||
Muse Spark 1.1meta | $0.372 | 2 days old | ||
Qwen3.8-27Balibaba | $0.430 | 2 days old | ||
Muse Spark 1.2 (xHigh)meta | $0.439 | 2 days old | ||
Hy4 previewtencent | $0.444 | 2 days old | ||
Mistral Large 4mistral | $0.446 | 2 days old | ||
Muse Spark 1.3 (Max)meta | $0.449 | 2 days old | ||
Mistral Medium 3.5mistral | $0.470 | 2 days old | ||
Gemini 3.7 Flash (High)google | $0.589 | 2 days old | ||
GPT-5.6 Terra (xHigh)openai | $0.606 | 2 days old | ||
Step 5 Previewstepfun | $0.631 | 2 days old | ||
Gemini 3.1 Pro Previewgoogle | $0.633 | 2 days old | ||
Gemini 3.8 Flash (High)google | $0.758 | 2 days old | ||
GPT-6.1 Terra (Max)openai | $0.788 | 2 days old | ||
GPT-6.1 Sol (Max)openai | $0.955 | 2 days old | ||
Gemini 4 Argon (High)google | $0.999 | 2 days old | ||
GLM-5.2 (Max)zai | $1.09 | 2 days old | ||
Claude Sonnet 5 (High)anthropic | $1.11 | 2 days old | ||
Kimi K3 (Max)moonshot | $1.12 | 2 days old | ||
GLM-5.3 (Max)zai | $1.14 | 2 days old | ||
Qwen3.8-Maxalibaba | $1.35 | 2 days old | ||
Muse Wonder (Max)meta | $1.43 | 2 days old | ||
Kimi K3.1 Pro (Max)moonshot | $1.47 | 2 days old | ||
Grok 4.6 (xHigh)xai | $1.69 | 2 days old | ||
GPT-5.6 Sol (xHigh)openai | $1.73 | 2 days old | ||
Gemini 4 Carbon (High)google | $1.73 | 2 days old | ||
GPT-6 Sol (Max)openai | $1.74 | 2 days old | ||
Grok 4.7 (xHigh)xai | $2.61 | 2 days old | ||
Claude Fable 5 (High)anthropic | $3.92 | 2 days old | ||
Claude Sonnet 5.5 (Max)anthropic | $3.94 | 2 days old | ||
Claude Opus 5 (Max)anthropic | $5.62 | 2 days old | ||
GPT-6 Astra (Max)openai | $6.09 | 2 days old | ||
Claude Opus 5.5 (Max)anthropic | $6.82 | 2 days old | ||
Claude Fable 5.1 (Max)anthropic | $6.95 | 2 days old | ||
| No cost available 4 | ||||
Beam 1.0 501B A23Breflection | Unknown | Not read | No source | 1 note
|
DeepSeek-V4-Pro (High, 20260813)deepseek | Unknown | Not read | No source | 1 note
|
Muse Wonder Full (Max)meta | Unknown | Not read | No source | 1 note
|
NVIDIA-Nemotron-3.5-Super-VL-120B-A12B (NVFP4)nvidia | Unknown | Not read | No source | 1 note
|
What evidence would settle it
The three conditions and what is missing
- Claude Opus 5.5 (Max): another independent origin or repeats (interval 8.4 points wide), metric Agent Arena score (net improvement over a uniform baseline).
- Claude Sonnet 5.5 (Max): another independent origin or repeats (interval 11.1 points wide), metric Agent Arena score (net improvement over a uniform baseline).
- Gemini 4 Carbon (High): another independent origin or repeats (interval 3.2 points wide), metric Agent Arena score (net improvement over a uniform baseline).
The three conditions for naming a best option
- The leader's lower bound must exceed every other option's upper bound by at least 3 points. Not met: the closest gap is 1.0 points (Claude Opus 5.5 (Max) against Claude Sonnet 5.5 (Max)).
- Every critical metric must come from two independent origins, or from one auditable test of our own. Not met: 1 origin and no auditable own test for Claude Opus 5.5 (Max).
- The price must be verified within 30 days. Met: read 2 days before this date.
Sources and limits
1 source, the method notes and this board as JSON
Scores on this board come from Arena Leaderboard Dataset (lmarena-ai/leaderboard-dataset). We rescale them to a 0 to 100 scale against fixed anchors (Agent Arena score (net improvement over a uniform baseline): -0.25 counts as 0 and 0.25 counts as 100), so a new option never moves another option's score. The licence and our changes are stated in the data credit at the foot of this page.
- Arena Leaderboard Dataset (lmarena-ai/leaderboard-dataset) free to use with credit; checked
Notes on this board. Only one origin (Arena) is used, so no board entry can be named Best until a second independent origin or an own test exists. Arena's IPS score comes from agent sessions of its own users; it is not a test of this board's job on your code.
Scores are on a 0 to 100 scale against editorial anchors that are not yet validated. Prices are for the workload above and change without notice. Evidence older than 60 days is left out, and prices older than 30 days are flagged.
This board as JSON, with every input id, anchor and weight.