Why does ChatGPT play illegal moves in chess?
ChatGPT plays illegal moves in chess because it is a language model: it writes the move that sounds most likely after the moves so far, and nothing inside it checks that the move is allowed. A chess program generates moves from the rules, so it can't play an impossible one. Even in Kaggle's Game Arena benchmark (September 2026), where models got the exact position every turn, illegal attempts kept appearing, mostly late in games.
Four reasons ChatGPT plays illegal moves
| Reason | What it looks like on the board |
|---|---|
| It predicts text, not moves | It plays a move that is common in chess writing but impossible in this position |
| It keeps no board in memory | A piece moves through another piece, or a captured piece comes back |
| Nothing checks the rules | It ignores a check, or "castles" when castling isn't allowed |
| Chat training made it worse | Chat versions break the rules more often than older text-completion models |
1. It predicts text. To a language model, a move like "bishop to c4" is a short piece of text that often follows certain other text. If the pattern fits, the move gets written, whether or not the bishop can actually reach that square.
2. There's no board, only a transcript. An engine stores the exact position and updates it after every move. ChatGPT has to rebuild the position from the conversation each time it answers. The longer the game, the more there is to keep straight, which is why the Game Arena researchers found errors concentrated in the later stages of games. Early on, models lean on openings they have memorized.
3. Nothing enforces the rules. In a benchmark, a separate program acts as referee. Kaggle's tells the model the move is invalid, asks it to try again and, after four failed attempts, scores the game as a loss. In the August 2025 exhibition, Kimi k2 lost all four of its games against OpenAI's o3 that way (Chess.com, 5 August 2025). In a normal ChatGPT conversation there's no referee at all: you are it.
4. Chat tuning hurt. In Mathieu Acher's 2023 tests, the chat model gpt-3.5-turbo made an illegal move in 93% of its games against Stockfish, the chat-tuned GPT-4 in 32%, and the older text-completion model gpt-3.5-turbo-instruct in 16%. Dynomight saw the same direction in November 2024: instruction tuning made models play worse in every comparison.
Example 1: ignoring a check
Check means your king is under attack, and the rules say you must deal with it on your very next move. The Game Arena paper names this as a frequent failure: the model is in check and proposes a move that doesn't solve it.
Here Black is in check from White's bishop after a normal opening (1.e4 e5 2.Nf3 d6 3.Bb5+).
Developing the knight to f6 is one of the most common moves in chess openings, so it's exactly the kind of move a text predictor likes. Here it is illegal: it does nothing about the check. Stockfish's choices are the pawn to c6, the knight to d7 or the bishop to d7. All three block the check and keep the game about equal.
Example 2: a bishop that walks through its own pawn
This one is an illustration rather than a move from a specific game. After a quiet opening (1.d4 d5 2.Nf3 Nf6), White can't play bishop to c4: the bishop on f1 would have to pass through e2, and White's own pawn is still standing there.
In openings where White pushed the king's pawn first, bishop to c4 is a very common developing move. A model that pattern-matches on "the bishop comes out to c4 around move three" can write it here, even though the board says no. That's what "losing track of the board" means in practice.
Why can't ChatGPT play chess like an engine?
Because it does none of the three things an engine does. An engine lists every legal move, searches ahead through the replies, and scores the positions it reaches. ChatGPT does one thing: it continues the text. Sometimes that produces good chess, because good moves are common in the games it learned from. When the position is new, it has nothing to fall back on. We compare the two approaches in detail in ChatGPT vs Stockfish.
ChatGPT lost at chess to an Atari 2600
The most famous example came in June 2025. Engineer Robert Caruso set ChatGPT against Video Chess, a 1979 game for the Atari 2600, running in an emulator on its beginner setting. The console has a 1.19 MHz processor and 128 bytes of memory, according to The Register (9 June 2025).
ChatGPT, using GPT-4o per Time Extension, confused rooks with bishops, missed pawn forks and kept losing track of where its pieces were. It blamed the Atari's piece icons, then did no better in standard notation. After about 90 minutes of Caruso correcting it, it conceded.
The sequels followed the script. Microsoft Copilot lost two pawns, a knight and a bishop while the Atari was down a single pawn (1 July 2025). Google's Gemini first said it would win, then backed out once it heard what happened to the others (14 July 2025).
The lesson isn't that the Atari is smart. It's that a tiny program built on the rules always knows the position and only ever plays legal moves. A chatbot has neither guarantee.
How to get chess help without illegal moves
- Give the full position every turn, not just the last move.
- Check every suggestion on a real board before believing it.
- For analysis, let an engine do the chess and the assistant do the talking.
That last point is what Chess Coach does inside Claude. It runs Stockfish on our server to review your finished Chess.com and Lichess games, and its position check tells Claude when a move is illegal and why, so Claude doesn't have to guess. It works in Claude, not in ChatGPT. See how to connect it or can Claude play chess for how Claude compares on its own.
Frequently asked questions
Why can't ChatGPT play chess properly?
Because it is a language model, not a chess program. It writes the move that looks most likely after the text so far, keeps no checked board in memory and doesn't search ahead, so it can produce moves that are impossible or simply bad.
Does ChatGPT still make illegal chess moves in 2026?
Yes, less often than in 2023 but not never. In Kaggle's Game Arena benchmark, published in September 2026, models still needed retries after illegal moves, and the errors were concentrated in the later part of games, especially for weaker models.
Did ChatGPT really lose at chess to an Atari 2600?
Yes. In June 2025 engineer Robert Caruso had ChatGPT play Video Chess, a 1979 Atari 2600 game, on its beginner level in an emulator. ChatGPT lost track of the pieces and eventually conceded.
How do I stop ChatGPT from making illegal chess moves?
You can't fully stop it, but you can catch them: give it the full position every turn and check each move on a real board. For analysis, use a tool that runs a chess engine and checks legality for it.