# LLM chess benchmark: the public leaderboards compared

> Every public LLM chess benchmark we could verify, with its method, date and top ratings. October 2026 data, with a source on every row.

- URL: https://thechesscoach.netlify.app/data/llm-chess-benchmark/
- Updated: 2026-10-26
- Author: Ignacio Soler

An LLM chess benchmark makes language models play chess, against each other, against engines or against a random mover, and turns the results into a rating. As of October 26, 2026, we could verify four public ones with their methods and numbers. Gemini models lead three of them, the best models from OpenAI and Anthropic sit in the 1500s to 1600s on Dubesor's and Saplin's boards, and no model comes close to a real chess engine.

The ratings can't be swapped between boards. Each one has its own opponents, its own rules for illegal moves and its own zero point. So this page puts the method next to every number.

## The four LLM chess leaderboards at a glance

Checked on October 26, 2026. "Last data" is the date the source itself shows.

| Leaderboard | Who runs it | How ratings are made | Opponents | Illegal moves | Last data |
|---|---|---|---|---|---|
| [AI Chess Leaderboard](https://dubesor.de/chess/chess-leaderboard) | Dubesor, independent developer | Elo updated after each game; starting value from Stockfish 17.1 accuracy | Other AI models; maia5-1500 and a random mover as references | Tracked as a "Legal %" column | Oct 26, 2026 (357 models, 5,163 matches) |
| [LLM Chess Leaderboard](https://maxim-saplin.github.io/llm_chess/) | Maxim Saplin; paper with co-authors ([arXiv 2512.01992](https://arxiv.org/abs/2512.01992)) | Maximum-likelihood Elo with a 95% interval | A random player and the Komodo Dragon engine at levels rated on Chess.com; the model always plays Black | Loss after 3 mistakes in one move, or 10 turns of dialogue | Data file of Oct 20, 2026 ([GitHub](https://github.com/maxim-saplin/llm_chess)) |
| [LLM Chess Benchmark](https://chessbenchllm.onrender.com/) | Open-source project ([GitHub](https://github.com/lightnesscaster/chess-llm-benchmark)) | Glicko-2, calibrated to an estimated Lichess Classical rating | Fixed-rating anchors (Maia networks, the Eubos engine, a random mover set to 400) and other models | One retry, then the game is forfeited | No date on the page; checked Oct 26, 2026 |
| [Game Arena paper](https://arxiv.org/abs/2609.31473) | Google DeepMind and Kaggle, 62 authors | Bradley–Terry Elo; the weakest model is set to 0 | Frontier models from 5 labs, 40 games per pair | Loss after four failed attempts | Submitted Sept 25, 2026; models chosen as of Feb 2026 |

Kaggle also runs a [live chess leaderboard](https://www.kaggle.com/benchmarks/kaggle/chess-text) for Game Arena. Its numbers load in the browser and we couldn't read them reliably, so we left them out.

## Top ratings on each board

Every row names its source. "±" and ranges are each board's own uncertainty. We only copied numbers we could see on the source on October 26, 2026.

| Board | Model | Rating | Games | Source |
|---|---|---|---|---|
| Dubesor | Gemini 3.1 Pro Preview | 1860 ±85 | 81 | [dubesor.de, Oct 26](https://dubesor.de/chess/chess-leaderboard) |
| Dubesor | GPT-6 Astra | 1637 ±86 | 45 | [dubesor.de, Oct 26](https://dubesor.de/chess/chess-leaderboard) |
| Dubesor | Claude Opus 5.5 | 1571 ±98 | 35 | [dubesor.de, Oct 26](https://dubesor.de/chess/chess-leaderboard) |
| Dubesor | maia5-1500 (human-like network, not an LLM) | 1554 ±90 | 46 | [dubesor.de, Oct 26](https://dubesor.de/chess/chess-leaderboard) |
| Dubesor | DeepSeek V4 Flash (0731) | 1438 ±99 | 44 | [dubesor.de, Oct 26](https://dubesor.de/chess/chess-leaderboard) |
| Dubesor | Grok 4.20 Reasoning | 1362 ±89 | 47 | [dubesor.de, Oct 26](https://dubesor.de/chess/chess-leaderboard) |
| Dubesor | Random mover (baseline) | 419 | 200 | [dubesor.de, Oct 26](https://dubesor.de/chess/chess-leaderboard) |
| Saplin | GPT-6 Astra (high) | 1614 ±110 | 67 | [Saplin, data of Oct 20](https://maxim-saplin.github.io/llm_chess/) |
| Saplin | GPT-6.1 Sol (xhigh) | 1612 ±162 | 30 | [Saplin, data of Oct 20](https://maxim-saplin.github.io/llm_chess/) |
| Saplin | Gemini 4 Argon (high) | 1609 ±131 | 40 | [Saplin, data of Oct 20](https://maxim-saplin.github.io/llm_chess/) |
| Saplin | Claude Opus 5.5 (high) | 1575 ±113 | 64 | [Saplin, data of Oct 20](https://maxim-saplin.github.io/llm_chess/) |
| LLM Chess Benchmark | Gemini 3.1 Pro Preview (high) | 2144 (1963–2325) | 18 | [chessbenchllm, Oct 26](https://chessbenchllm.onrender.com/) |
| LLM Chess Benchmark | GPT-6 Astra (high) | 1975 (1870–2081) | 50 | [chessbenchllm, Oct 26](https://chessbenchllm.onrender.com/) |
| LLM Chess Benchmark | maia-1900 (anchor, not an LLM) | 1816 | 256 | [chessbenchllm, Oct 26](https://chessbenchllm.onrender.com/) |
| LLM Chess Benchmark | Claude Opus 5.5 (high) | 1197 (517–1876) | 2 | [chessbenchllm, Oct 26](https://chessbenchllm.onrender.com/) |
| Game Arena paper | Gemini 3 Pro Preview | 1325 | 40 per pair | [arXiv 2609.31473](https://arxiv.org/abs/2609.31473) |
| Game Arena paper | o3 | 1009 | 40 per pair | [arXiv 2609.31473](https://arxiv.org/abs/2609.31473) |
| Game Arena paper | Grok 4 | 773 | 40 per pair | [arXiv 2609.31473](https://arxiv.org/abs/2609.31473) |
| Game Arena paper | Claude Opus 4.5 | 236 | 40 per pair | [arXiv 2609.31473](https://arxiv.org/abs/2609.31473) |

Look at the game counts before the ratings. Claude Opus 5.5 has two games on the LLM Chess Benchmark, so its 95% range there runs from 517 to 1876: it tells you almost nothing yet. The Game Arena paper tested an older generation of models, which is why its numbers look so different.

## How each LLM chess benchmark works

**Dubesor's AI Chess Leaderboard** is the biggest pool. Each model gets a starting rating from the accuracy of its first ten games, as judged by Stockfish 17.1, and then normal Elo updates after every AI-vs-AI game. It also publishes Stockfish accuracy and a blunder rate for every model, plus a separate puzzle rating against official Lichess puzzle ratings. On that puzzle table, Gemini 4 Argon (high) leads at 2193 ±85, GPT-6 Astra scores 1921 ±71 and Claude Opus 5.5 1631 ±72. Dubesor's page warns that Elo here is relative to its own pool.

**Saplin's LLM Chess Leaderboard** asks models to play through a short dialogue: they can ask for the board, ask for the legal moves, or submit a move. Breaking the format three times in one move, or talking for ten turns without moving, loses the game. That makes it partly a test of following instructions. The [paper behind it](https://arxiv.org/abs/2512.01992) (December 2025) found that many leading models struggled to finish games or win consistently, even against a random opponent, and that reasoning models clearly beat non-reasoning ones.

**The LLM Chess Benchmark** tries hardest to speak the language of human ratings. Models face anchors with fixed Lichess Classical ratings, and the results are mapped onto that scale with Glicko-2, a rating system that also tracks how uncertain each rating is. There's no rating floor, so weak models can go below zero. Its [methodology page](https://chessbenchllm.onrender.com/methodology) adds a caveat we agree with: the author expects models would do worse against humans, who are better at exploiting systematic weaknesses.

**The Game Arena paper** is the most controlled. Every pair of models plays 40 games, 20 with each colour, through public APIs with default settings and no tools. Ratings are relative, so the weakest model is set to 0. The authors used Stockfish to track who was winning during each game and found that the models start out balanced and drift apart in the middlegame.

## What these benchmarks measure: playing, not explaining

All four boards score the same thing: can the model win a game and keep its moves legal. None of them checks whether the model explains a position correctly, which is what most people actually ask a chatbot to do after a game.

Those are different skills. A separate research benchmark, [ChessQA](https://arxiv.org/abs/2510.23948) (October 2025), tested understanding instead of play, across five areas from basic rules to tactics, judging positions and chess concepts. Its authors report persistent weaknesses in every area for the models they tested. So a model with a decent playing rating can still give a confident, wrong explanation.

That gap is the honest takeaway from every LLM chess leaderboard. Even on the board that maps to Lichess Classical ratings, where the best model is estimated at 2144, the models still blunder, and nothing inside the model checks its claims. For Claude in particular, see [What is Claude's chess Elo?](/answers/claude-chess-elo/) and [Claude vs ChatGPT at chess](/answers/claude-vs-chatgpt-chess/). For what happens when a model meets a real engine, see [ChatGPT vs Stockfish](/answers/chatgpt-vs-stockfish/).

## Why Chess Coach uses Stockfish for the facts

These leaderboards are the reason Chess Coach doesn't let the language model decide what's true about a position. Chess Coach is a connector for Claude and ChatGPT. It reviews your finished Chess.com and Lichess games with Stockfish on our own server, and every evaluation, best move and line comes from the engine. Your assistant's job is the part language models are good at: turning those facts into plain words for your level, on an interactive board with the key moments marked. That way, the assistant's own chess rating doesn't limit the quality of your review. The free plan includes one in-depth review a day. [Connect Chess Coach](/connect/), read the [docs](/docs.html), or see how chess tools plug into Claude in our [Claude chess skill guide](/guides/claude-code-chess-skill/).

## FAQ

### What is the best LLM chess benchmark?

There isn't a single standard. Dubesor's AI Chess Leaderboard has the most models and games, Saplin's LLM Chess Leaderboard is described in a paper on arXiv, the LLM Chess Benchmark maps results to a Lichess Classical scale, and the Game Arena paper by Google DeepMind and Kaggle plays 40 games between every pair of models. Each answers a slightly different question.

### Which LLM tops the chess leaderboards in October 2026?

Gemini models lead most boards. On Dubesor's leaderboard (October 26, 2026) Gemini 3.1 Pro Preview is first at 1860 ±85. On Saplin's (data of October 20, 2026) GPT-6 Astra is first at 1614 ±110, with Gemini 4 Argon and Claude Opus 5.5 inside its margin of error.

### Can I compare LLM chess Elo with my Chess.com rating?

Not directly. Each leaderboard rates models inside its own pool and rules. The LLM Chess Benchmark tries to map to Lichess Classical using engine anchors, and Dubesor gives a rough bot comparison, but both say these are approximations.

### Do LLM chess benchmarks measure how well a model explains chess?

No. The four leaderboards here measure playing: winning games and making legal moves. Explaining a position well is a different skill, tested by separate research such as the ChessQA benchmark.

## Related

- [Claude chess Elo: what the LLM leaderboards say (2026)](https://thechesscoach.netlify.app/answers/claude-chess-elo.md)
- [Claude vs ChatGPT chess: which AI plays better? (2026)](https://thechesscoach.netlify.app/answers/claude-vs-chatgpt-chess.md)
- [Can Claude play chess? What it can and can't do (2026)](https://thechesscoach.netlify.app/answers/can-claude-play-chess.md)
- [ChatGPT vs Stockfish: who wins at chess, and why](https://thechesscoach.netlify.app/answers/chatgpt-vs-stockfish.md)
- [Claude chess skill: skills vs MCP for chess (2026)](https://thechesscoach.netlify.app/guides/claude-code-chess-skill.md)