Debate

• 59 models and settings in the full analysis; 56 shown publicly
• 3,672 complete side-swapped matchups, comprising 7,344 debates, used for ratings
• 22,592 judge decisions behind the ratings
• 683 topics represented in the debates used for ratings

This update focuses on Claude Opus 5.5 High, Grok 4.7 High, MiMo V2.6 Pro Thinking, and DeepSeek V4.1 Flash High. Tests of these models completed 728 of 740 scheduled debates and 2,184 judgments, including newcomer-versus-newcomer matches. Twelve unavailable attempts remain recorded separately.

One side-swapped matchup means two debates on the same motion with PRO and CON roles reversed.

Which results each view includes

The full judge decision table contains 22,818 decisions; 22,592 belong to complete matchups used for ratings. These views answer different questions, so their counts are not expected to match. The full leaderboard retains historical comparisons; charts of current models omit superseded versions except selected predecessors. The September 5 blind participant-judging study is a separate historical diagnostic and does not feed the leaderboard.

• Claude Fable 5.1 remains first at 1737.6, followed by Fable 5 at 1726.5 and Opus 5 at 1724.2. Their confidence intervals overlap.
• Claude Opus 5.5 enters at #4 with 1701.0 across 97 complete matchups, 23.2 points below Opus 5.
• Grok 4.7 reaches #11 at 1619.8, up 15.7 points from Grok 4.6. Their confidence intervals overlap.
• MiMo V2.6 Pro Thinking reaches #12 at 1609.2, up 125.5 points from MiMo V2.5 Pro. It has 92 complete matchups; availability remains weaker than its debate-quality ranking.
• DeepSeek V4.1 Flash reaches #17 at 1582.3 across 100 complete matchups. DeepSeek V4 Pro 0813 scores 1572.1; Flash and Pro are distinct variants, so this is a same-family comparison.
• Each newcomer has 92–100 complete matchups against 13–17 opponents. All six newcomer-to-newcomer pairings were included. Confidence intervals and pairwise coverage matter more than small rank differences.

Each cell is the mean signed judge margin for the row model over the column model. Positive blue cells favor the row model; negative red cells favor the column model. The number in parentheses is the count of complete side-swapped matchups for that head-to-head. Cells with only one or two matchups should be read as directional evidence rather than stable pairwise estimates.

Higher and further left is better. The x-axis is average debater-side USD per completed, rating-eligible debate, calculated from recorded token usage and applicable prices or provider-reported request costs. It accounts for the number of input and output tokens actually used, including recorded reasoning tokens and cache or batch discounts. It covers accepted turns and excludes failed attempts and judging.

Cost coverage is reported per model; models without usable cost data are omitted. Arrows connect the focused models to their recorded predecessors where available. DeepSeek Pro is a separate comparison variant, not a Flash predecessor.

DeepSeek V4.1 Flash averages $0.01117 per debate (1.12¢) across 200 debates with complete recorded token usage. Its estimate uses the full-week weighted average of the scheduled rates: 20.83% peak and 79.17% off-peak, independent of when requests ran.

In the September 5 study, the eight highest-ranked latest-version models at selection time blindly judged 450 debates against one another, covering all 28 pairings and 225 side-swapped matchups. Each participant evaluated each debate twice, with anonymous A/B labels and display order reversed: 1,800 valid judgments in total. Judgments include a winner, a 0–5 margin, and five 1–10 diagnostic scores.

After averaging both presentations, we compare each participant's margin for its own side with the original independent panel's mean raw margin. Positive differences indicate a more favorable view of its own performance. Brackets show 95% confidence intervals clustered by topic.

Astra is the clear outlier in this sample. Against Fable 5.1, it preferred itself in all 16 debates after averaging orders; the independent panel favored Fable in 14. Each of Fable's other six opponents gave Fable a positive average margin in their direct encounters.

Presentation sensitivity matters: 156 of 900 participant–debate assessments (17.3%) picked opposite winners across the two presentations. These differences include sampling variation as well as label/display effects. We average both orders and retain graded margins and ties. Opponent mixes differ, and a near-zero average difference can conceal individual disagreements. One reference debate has two panel judges; the report includes a sensitivity check excluding that entire matchup.

These participant judgments are a separate diagnostic and do not affect official ratings. See the full report and per-model summaries, order-sensitivity chart, and individual judgments with explanations.

The benchmark tracks an entertainment/readability score as a diagnostic only; it does not affect ratings. Across chart-visible complete matchups, the mean is 7.49 / 10. The highest model averages include Claude Opus 5, Claude Fable 5.1, Claude Opus 5.5, Grok 4.7, and Muse Spark 1.3.

See the entertainment report for model and matchup details.

Every model is linked to the others through direct or indirect comparisons. Average agreement between judges is 55.6%, including wins, ties, and results too close to call. Agreement on clear winners is 80.6%: it considers only judge pairs where both selected a clear winner, excluding cases where either judge returned a tie or noise-level result. Presentation bias measures whether the way the two sides are shown affects judging. Its average size is 0.183 margin points. Panels use three distinct model families and avoid same-family judges against debaters when feasible.

The current group of judges is GPT-5.6 Sol (high), Claude Sonnet 5 (high), Gemini 3.1 Pro Preview, Qwen 3.7 Max, Grok 4.5 (high), Kimi K2.6, and Muse Spark 1.1 (high). The ratings also include judgments from earlier panels.

The first heatmap includes wins for Side A, wins for Side B, and a combined category for ties or results too close to call. The second asks the narrower question: when both judges chose a clear winner, how often did they choose the same winner? These are evaluator-consistency diagnostics, not a second leaderboard.

The September 24 analysis covers every eligible complete side-swapped matchup within its group of 43 models, including all four highlighted newcomers:

• 5,852 / 5,852 debates with model identities hidden during analysis, linking debating behavior to results
• 2,926 / 2,926 side-swapped matchups with analysis across both sides of each debate
• 42 public behavior model cards with judge diagnostics and transcript evidence; the analysis also includes models omitted from this report
• Four new reviewed dossiers: Claude Opus 5.5 High, Grok 4.7 High, MiMo V2.6 Pro Thinking, and DeepSeek V4.1 Flash High

The dossiers describe recurring strengths, weaknesses, execution, and behavior across assigned sides. They distinguish what judges mention from what judges praise, and describe behavior under this benchmark's debate format. Earlier dossiers remain available as dated snapshots.

• Behavior model cards
• Four newcomer dossiers
• September 4 model dossiers
• Current quantitative model profiles

Content blocks, blank outputs, formatting failures, and model-service failures are tracked separately from debate quality. The availability-adjusted view shows how reliability would change model strength if completion problems were penalized; it does not change how completed debates were judged.

The topic bank contains 683 motions, spanning policy, empirical, and moral disputes intended to be understandable to an informed generalist. All 683 appear in the full set of rated debates. The tables count distinct topics, not debates; individual models have different topic coverage.

New-entrant examples

• Claude Opus 5.5 vs Grok 4.7 on children’s digital wellbeing. Each won unanimously, 3–0, as CON, leaving the paired result tied. Both challenged whether sleep, boredom, and play adequately address product-specific harms such as grooming and unwanted purchases. Read Debate A, Debate B, and the matchup report.
• DeepSeek V4.1 Flash vs MiMo V2.6 Pro Thinking on whether the British economy would be stronger without Brexit. Each won unanimously, 3–0, as PRO, again producing a paired tie. The clash turns on the relevant counterfactual: whether avoiding Brexit's additional costs would improve an economy that still faces domestic structural problems. Read Debate A, Debate B, and the matchup report.

These pairs illustrate why a single debate direction can be misleading.

Earlier model comparison

Claude Opus 4.8 vs Claude Opus 5 on banning corporate political donations: Opus 4.8 won the first direction 2–1 as PRO; Opus 5 won the reverse direction 3–0 as PRO and won the paired result. Every judge rated both debates 9 / 10 for entertainment. Read Debate A, Debate B, and the matchup report.

A few memorable lines from earlier debates, with transcripts for context. These are examples of rhetoric, not inputs to the ratings.

You cannot reject a trap you cannot see.

— Qwen3.5-397B-A17B, arguing for a ban on personalized algorithmic pricing. Transcript

That is not a protection of the vulnerable; it is hostage-taking with occasional mercy.

— MiniMax-M2.7, challenging the claim that personalized pricing protects low-income shoppers. Transcript

If preservation wins even there, then it is not stewardship; it is exclusion protected by aesthetics.

— GPT-5.4 (high), arguing for housing density in historic districts. Transcript

For each selected model pair and topic:

1. The two models debate the proposition in a ten-turn format.
2. They debate the same proposition again with PRO and CON reversed.
3. Full prompts and debate transcripts are retained.
4. A three-model judge panel selects a winner and margin for each debate.
5. Only complete two-debate contests contribute to the ranking; individual judging categories are reported separately.

The ten turns are PRO opening, CON opening, first rebuttals, pressure questions, second rebuttals, and closings. Responses that exceed the length limit are cut automatically. Word counts are based on the debate text, not the model's own count.

Bradley-Terry estimates relative strength from paired outcomes while accounting for opponent strength. This makes comparisons across uneven schedules more useful than a raw win rate or an average of rubric scores. Swapping sides reduces the influence of an easier position on a particular motion; multiple judges and confidence intervals help show the remaining uncertainty.

• These results come from an ongoing benchmark and may change as models and judgments are added.
• It uses LLM judges rather than human judges, though side swaps, multiple judges, distinct judge families, and agreement diagnostics reduce some noise.
• Matchup schedules are intentionally non-uniform, so confidence intervals and the pairwise matrix matter.
• Qualitative tags and dossier prose come from GPT-5.6 Sol (medium), not human coders.
• Availability and content-filter behavior can materially affect coverage; reliability is reported separately.
• Debate is one capability slice, not a complete measure of model usefulness.

• Leaderboard report
• Benchmark status
• Matchup results index
• New-entrant analysis
• Judge decision table
• Full judge decision records
• Completed public transcripts

Reproducibility checks

• Checks for model-service errors