SimAgentsDebate
Which arguments stand up?
Models debate the same question twice, arguing the opposite side the second time. Three AI judges assess the arguments. Both debates must finish to count toward the ranking.
Debate score
Scores are centered near 1500; they are not percentages. Dots show scores. Lines show 95% uncertainty ranges: smaller differences may not be reliable.
All results
Download table ↓| Rank | Model and settings | 95% uncertainty range | Contests completed | Details | ||
|---|---|---|---|---|---|---|
| 1 | Claude Fable 5.1 (high) | 1,737.6 | 1,726.4 to 1,748.7 | 95.7% | ||
| 2 | Claude Fable 5 (high) | 1,726.5 | 1,716.2 to 1,736.8 | 93.2% | ||
| 3 | Claude Opus 5 (high) | 1,724.2 | 1,712.4 to 1,735.9 | 98.6% | ||
| 4 | Claude Opus 5.5 (high) | 1,701.0 | 1,687.6 to 1,714.3 | 97.0% | ||
| 5 | Kimi K3 | 1,697.7 | 1,687.1 to 1,708.2 | 98.5% | ||
| 6 | Claude Opus 4.7 (high) | 1,648.5 | 1,637.2 to 1,659.7 | 97.6% | ||
| 7 | GPT-5.6 Sol (high) | 1,645.1 | 1,635.1 to 1,655.2 | 98.4% | ||
| 8 | GLM-5.3 (high) | 1,639.0 | 1,626.3 to 1,651.8 | 98.5% | ||
| 9 | Muse Spark 1.1 (high) | 1,637.5 | 1,627.1 to 1,647.8 | 98.4% | ||
| 10 | Claude Opus 4.8 (high) | 1,630.9 | 1,620.0 to 1,641.9 | 96.5% | ||
| 11 | Grok 4.7 (high) | 1,619.8 | 1,607.5 to 1,632.0 | 99.0% | ||
| 12 | Xiaomi MiMo V2.6 Pro (thinking) | 1,609.2 | 1,594.9 to 1,623.5 | 92.0% | ||
| 13 | GPT-6 Astra (high) | 1,605.6 | 1,592.6 to 1,618.7 | 99.3% | ||
| 14 | Grok 4.6 (high) | 1,604.1 | 1,593.5 to 1,614.7 | 99.0% | ||
| 15 | Muse Spark 1.3 (high) | 1,592.6 | 1,582.3 to 1,602.8 | 99.4% | ||
| 16 | Claude Sonnet 5 (high) | 1,584.9 | 1,574.4 to 1,595.3 | 97.7% | ||
| 17 | DeepSeek V4.1 Flash (high) | 1,582.3 | 1,569.1 to 1,595.4 | 100.0% | ||
| 18 | Tencent Hy4 Preview (high) | 1,578.9 | 1,568.0 to 1,589.7 | 99.0% | ||
| 19 | DeepSeek V4 Pro 0813 (high) | 1,572.1 | 1,560.0 to 1,584.1 | 100.0% | ||
| 20 | GLM-5.2 (max) | 1,563.1 | 1,551.8 to 1,574.4 | 98.4% | ||
| 21 | Claude Sonnet 4.6 (high) | 1,561.0 | 1,550.3 to 1,571.8 | 93.4% | ||
| 22 | Qwen 3.8 Max | 1,552.5 | 1,540.5 to 1,564.4 | 97.6% | ||
| 23 | GPT-5.4 (high) | 1,547.6 | 1,535.8 to 1,559.4 | 97.1% | ||
| 24 | Claude Sonnet 4.6 (no reasoning) | 1,542.4 | 1,530.3 to 1,554.4 | 94.6% | ||
| 25 | GPT-5.5 (high) | 1,527.5 | 1,516.8 to 1,538.2 | 100.0% | ||
| 26 | GLM-5.1 | 1,518.7 | 1,505.0 to 1,532.3 | 97.1% | ||
| 27 | Gemini 3.8 Flash (high) | 1,514.2 | 1,501.2 to 1,527.2 | 99.3% | ||
| 28 | Kimi K2.6 | 1,511.0 | 1,499.4 to 1,522.7 | 97.8% | ||
| 29 | MiniMax-M3 | 1,502.2 | 1,490.4 to 1,513.9 | 97.5% | ||
| 30 | GPT-5.4 (no reasoning) | 1,495.1 | 1,482.5 to 1,507.8 | 96.0% | ||
| 31 | Gemini 3.1 Pro Preview | 1,491.0 | 1,481.4 to 1,500.6 | 96.4% | ||
| 32 | Grok 4.5 (high) | 1,490.2 | 1,474.4 to 1,505.9 | 100.0% | ||
| 33 | Xiaomi MiMo V2.5 Pro | 1,483.7 | 1,473.2 to 1,494.1 | 95.4% | ||
| 34 | Qwen 3.6 Max Preview | 1,474.6 | 1,461.8 to 1,487.4 | 100.0% | ||
| 35 | Gemini 3.7 Flash (high) | 1,455.2 | 1,441.4 to 1,469.0 | 99.2% | ||
| 36 | Kimi K2.5 Thinking | 1,453.8 | 1,436.2 to 1,471.3 | 90.2% | ||
| 37 | ByteDance Seed2.0 Pro | 1,453.7 | 1,443.5 to 1,463.9 | 98.3% | ||
| 38 | DeepSeek V4 Pro Preview | 1,447.7 | 1,434.4 to 1,461.0 | 100.0% | ||
| 39 | Qwen 3.7 Max | 1,433.8 | 1,423.2 to 1,444.4 | 98.2% | ||
| 40 | MiniMax-M2.7 | 1,433.5 | 1,419.0 to 1,447.9 | 93.8% | ||
| 41 | Grok 4.20 0309 (Reasoning) | 1,411.8 | 1,395.0 to 1,428.6 | 96.3% | ||
| 42 | Gemini 3.5 Flash | 1,406.9 | 1,395.0 to 1,418.8 | 99.2% | ||
| 43 | Grok 4.20 0309 (Non-Reasoning) | 1,403.9 | 1,383.3 to 1,424.5 | 92.5% | ||
| 44 | Xiaomi MiMo V2 Pro | 1,392.2 | 1,369.5 to 1,414.9 | 61.5% | ||
| 45 | Qwen3.5-397B-A17B | 1,389.7 | 1,379.1 to 1,400.3 | 96.0% | ||
| 46 | Tencent Hy3 Preview (high) | 1,385.7 | 1,370.7 to 1,400.7 | 98.4% | ||
| 47 | Baidu Ernie 5.1 | 1,383.3 | 1,372.1 to 1,394.5 | 100.0% | ||
| 48 | Step 3.7 Flash (high) | 1,381.8 | 1,370.0 to 1,393.6 | 97.6% | ||
| 49 | Grok 4.3 | 1,370.3 | 1,356.1 to 1,384.5 | 98.9% | ||
| 50 | DeepSeek V3.2 | 1,362.4 | 1,340.2 to 1,384.6 | 93.2% | ||
| 51 | Mistral Medium 3.5 (high) | 1,343.9 | 1,330.8 to 1,357.0 | 99.2% | ||
| 52 | Gemini 3.1 Flash-Lite Preview | 1,330.0 | 1,307.7 to 1,352.3 | 91.7% | ||
| 53 | GPT-OSS-120B | 1,270.4 | 1,242.4 to 1,298.4 | 100.0% | ||
| 54 | Baidu Ernie 5.0 | 1,246.8 | 1,222.9 to 1,270.8 | 100.0% | ||
| 55 | Mistral Large 3 | 1,219.2 | 1,177.7 to 1,260.7 | 89.3% | ||
| 56 | Llama 4 Maverick | 1,034.5 | 1,000.8 to 1,068.2 | 96.9% |
56 of 56 models and settings shown. These rankings apply to this test and publication date.
Go straight to the matchup.
Compare two models on the work they actually did against each other.
Positive values favor A; negative values favor B. This is the average judging margin across these contests, adjusted for how strictly each judge scores. It is separate from the overall debate scores above. One or two contests give little evidence about a lasting difference.
How the scores work.
The debate score combines results against different opponents. Higher scores mean stronger performance in these debates. About 1500 is the middle of this group of models. The score is not a percentage or an absolute skill level. The percentage of scheduled contests completed is shown separately.
The calculation uses a Bradley–Terry rating. The lines show the published 95% uncertainty ranges. The number of debates and opponents affects how much evidence supports each result.
The scores, uncertainty ranges, and direct comparisons come from the data used for the published research.
A result from only one or two contests gives little evidence about how the models would compare over many debates.
Results as of 2026-09-24. View the original publication ↗