As of 5 November 2026, no option is proven ahead. 6 options of 19 cannot be separated with the evidence in hand: gpt-5.6-sol-xhigh, qwen3.8-max, claude-opus-4-6-search, gpt-5.5-search, claude-fable-5.1-high and 1 more.
- Options
- 19
- Measured
- 19
- Shortlist
- 6
- Plotted
- 16
- Priced
- 16
The job
Answer questions with live web search through a vendor's own API, and judge the model by how often people prefer its answers in blind comparisons on Arena, against what 1,000 searches cost.
Ranking
19 options measured, shortlisted rows shaded
| # | Option | Score 0 to 100 | Cost per 1,000 queries | Interval | Price page |
|---|---|---|---|---|---|
| 1 | gpt-5.6-sol-xhighopenaiFrontierShortlist | 84.5 | $30.00 | ci9581.9 to 87.0 | |
| 2 | qwen3.8-maxalibabaShortlist | 84.4 | Not priced | ci9579.2 to 89.7 | |
| 3 | claude-opus-4-6-searchanthropicShortlist | 80.5 | $35.00 | ci9578.7 to 82.4 | |
| 4 | gpt-5.5-searchopenaiShortlist | 78.6 | $37.00 | ci9576.7 to 80.6 | |
| 5 | claude-fable-5.1-highanthropicShortlist | 78.3 | $60.00 | ci9575.5 to 81.2 | |
| 6 | claude-opus-5-highanthropicShortlist | 76.6 | $35.00 | ci9573.9 to 79.4 | |
| 7 | gpt-6-astra-maxopenai | 73.4 | $60.00 | ci9570.3 to 76.6 | |
| 8 | gpt-6.1-sol-maxopenaiFrontier | 73.2 | $20.00 | ci9569.7 to 76.8 | |
| 9 | claude-opus-4-7anthropic | 72.1 | $35.00 | ci9570.1 to 74.0 | |
| 10 | claude-fable-5-highanthropic | 71.7 | $60.00 | ci9569.1 to 74.3 | |
| 11 | ernie-5.1baidu | 70.7 | Not priced | ci9566.9 to 74.5 | |
| 12 | claude-sonnet-4-6-searchanthropic | 68.2 | $25.00 | ci9566.4 to 70.0 | |
| 13 | grok-4.5xai | 66.4 | Not priced | ci9564.0 to 68.7 | |
| 14 | claude-opus-5.5-mediumanthropic | 65.1 | $30.00 | ci9562.3 to 67.9 | |
| 15 | gemini-3.1-pro-groundinggoogle | 63.5 | $24.80 | ci9561.7 to 65.4 | |
| 16 | gpt-5.1-searchopenaiFrontier | 59.9 | $17.75 | ci9557.8 to 62.0 | |
| 17 | gemini-3-flash-groundinggoogleFrontier | 58.3 | $16.70 | ci9556.6 to 60.0 | |
| 18 | claude-sonnet-5-searchanthropic | 56.4 | $20.00 | ci9553.9 to 58.8 | |
| 19 | gpt-6-luna-maxopenaiFrontier | 43.8 | $10.50 | ci9540.9 to 46.7 |
Order is by score, highest first. Intervals overlap, so neighbouring ranks do not differ with certainty. Cost is in USD per 1,000 queries; bars start at zero. Interval: ci95, built from 95% confidence intervals given by the source.
Price against performance
Score against cost of 1,000 queries, log scale
- Frontier: no other option is both cheaper and scores higher
- Other option
- Ring: on the shortlist, not separable with certainty
- Line: score interval, ci95 (built from 95% confidence intervals given by the source)
3 options not plotted: a coordinate is unknown
- ernie-5.1: Cost unknown
- grok-4.5: Cost unknown
- qwen3.8-max: Cost unknown
Cost for this job
Date read and source for all 19 options
Cost of 1,000 queries, in USD per 1,000 queries, cheapest first within each group.
Workload and assumptions
Cost of 1,000 queries. Each query makes one search, sends 3,000 input tokens (the question and the retrieved content) and returns 400 output tokens. The cost is the search fee per 1,000 plus the tokens at the model's list price. Free monthly search allowances, caching and the extra searches an agent may run are not counted. The search fees of xAI and the other vendors without a read price are unknown, so those models have unknown cost.
| Option | Cost | Price read | Source | Notes |
|---|---|---|---|---|
| Cost 16 | ||||
gpt-6-luna-maxopenai | $10.50 | 0 days old | ||
gemini-3-flash-groundinggoogle | $16.70 | 0 days old | ||
gpt-5.1-searchopenai | $17.75 | 0 days old | ||
claude-sonnet-5-searchanthropic | $20.00 | 0 days old | ||
gpt-6.1-sol-maxopenai | $20.00 | 0 days old | ||
gemini-3.1-pro-groundinggoogle | $24.80 | 0 days old | ||
claude-sonnet-4-6-searchanthropic | $25.00 | 0 days old | ||
claude-opus-5.5-mediumanthropic | $30.00 | 0 days old | ||
gpt-5.6-sol-xhighopenai | $30.00 | 0 days old | ||
claude-opus-4-6-searchanthropic | $35.00 | 0 days old | ||
claude-opus-4-7anthropic | $35.00 | 0 days old | ||
claude-opus-5-highanthropic | $35.00 | 0 days old | ||
gpt-5.5-searchopenai | $37.00 | 0 days old | ||
claude-fable-5.1-highanthropic | $60.00 | 0 days old | ||
claude-fable-5-highanthropic | $60.00 | 0 days old | ||
gpt-6-astra-maxopenai | $60.00 | 0 days old | ||
| No cost available 3 | ||||
ernie-5.1baidu | Unknown | Not read | No source | 1 note
|
grok-4.5xai | Unknown | Not read | No source | 1 note
|
qwen3.8-maxalibaba | Unknown | Not read | No source | 1 note
|
What evidence would settle it
The three conditions and what is missing
- claude-fable-5.1-high: another independent origin or repeats (interval 5.6 points wide), metric Arena search rating (Elo-style, overall category).
- claude-opus-4-6-search: another independent origin or repeats (interval 3.6 points wide), metric Arena search rating (Elo-style, overall category).
- claude-opus-5-high: another independent origin or repeats (interval 5.5 points wide), metric Arena search rating (Elo-style, overall category).
- gpt-5.5-search: another independent origin or repeats (interval 3.9 points wide), metric Arena search rating (Elo-style, overall category).
- gpt-5.6-sol-xhigh: another independent origin or repeats (interval 5.0 points wide), metric Arena search rating (Elo-style, overall category).
- qwen3.8-max: another independent origin or repeats (interval 10.5 points wide), metric Arena search rating (Elo-style, overall category).
The three conditions for naming a best option
- The leader's lower bound must exceed every other option's upper bound by at least 3 points. Not met: the closest gap is -7.8 points (gpt-5.6-sol-xhigh against qwen3.8-max).
- Every critical metric must come from two independent origins, or from one auditable test of our own. Not met: 1 origin and no auditable own test for gpt-5.6-sol-xhigh.
- The price must be verified within 30 days. Met: read 0 days before this date.
Sources and limits
4 sources, the method notes and this board as JSON
Scores on this board come from Arena Leaderboard Dataset (lmarena-ai/leaderboard-dataset). We rescale them to a 0 to 100 scale against fixed anchors (Arena search rating (Elo-style, overall category): 1050 counts as 0 and 1300 counts as 100), so a new option never moves another option's score. The licence and our changes are stated in the data credit at the foot of this page.
- Arena Leaderboard Dataset (lmarena-ai/leaderboard-dataset) free to use with credit; checked
- Claude API pricing prices only, as facts with a link; checked
- Gemini API pricing prices only, as facts with a link; checked
- OpenAI API pricing prices only, as facts with a link; checked
Notes on this board. Only one origin (Arena) measures quality, so no board entry can be named Best until a second independent origin or an own test exists. Arena's rating comes from blind votes by its own users on their own prompts; it is not a test of your job. Prices are read from each vendor's own API price page and are list prices for the stated workload; a model whose price could not be read on the vendor's page is listed with unknown cost and is not plotted. Selection: the 15 models with the highest Arena rank in the overall category, plus every model ranked 16 to 42 whose price was read on vendor pages and whose output price is at most $10 per million tokens.
Scores are on a 0 to 100 scale against editorial anchors that are not yet validated. Prices are for the workload above and change without notice. Evidence older than 60 days is left out, and prices older than 30 days are flagged.
This board as JSON, with every input id, anchor and weight.