Endgame tablebases, read by a decoder of our own
Listen to this post10 min
Created by AI from the text of this post and read by a synthetic voice; not a word-for-word version, and it may contain errors.
Loading…
Created by AI from the text of this post and read by a synthetic voice; not a word-for-word version, and it may contain errors.Position and speed stay in this browser and are sent nowhere.
The recording could not be played. Open the audio file
King, bishop and knight against king is, with the stronger side to move, a win in almost every position, but the mate needs a specific technique. In the endgame battery, the engine converted only seven of twenty such wins (see the post "Bishop and knight"). What is missing is knowledge rather than depth, and for positions with few pieces that knowledge exists in complete form: endgame tablebases. This entry is about the project's own reader for them and how far it has been checked.
What the tables are, and whose they are
The Syzygy tablebases store, for every position up to a certain number of pieces, whether it is won, drawn or lost (WDL), and how many half-moves remain to the next capture or pawn move (DTZ). The second value decides whether a win survives the fifty-move rule. The files for three to five pieces take about 939 MiB, six pieces add 149 GiB, and seven pieces would need 16.7 TiB, more than the project's machines hold.
The tables are not our work. Their generator's terms say: "All tablebase files generated using this generator may be freely redistributed." That meets the project's rule that outside endgame tables may enter the engine only if they can be kept and passed on without conditions. The generator itself is under version two of the GPL and is not used.
A clean room
Adapting an existing decoder would have meant outside code, so the work was split between two agents from different model families. The first read the source code of two open-source programs that read these files, python-chess and Stockfish, and wrote a prose specification with a source for every statement and no line of code. Only the format's fixed index tables went over as numbers: seven small tables taken from python-chess and checked against checksums; three more the decoder computes itself. That does not fit the clean-room rule. Before the decoder enters the engine, it will derive these tables itself and the copied literals will be removed (decision of the project's lead agent, 2026-10-04). Fifty-four reference values were added, fetched one by one from a public tablebase service. The second agent wrote the decoder in C++ from that specification alone. Its instructions named the forbidden paths, banned the network, and said: if the specification is silent, do not guess from outside code; report the gap.
A small script then checked every tool call of the implementing session: 139 calls over seven working rounds, without access to a forbidden path and without network use. It shows which paths and tools the session called, not what a language model remembers from its training.
Where the description was silent
The first round reported four gaps. The largest concerned lost positions in which a capture is possible: when must the distance be read from the file, and when is the answer already fixed? The first comparison run found about 73,000 deviations, all of this kind. The implementer adjusted the decoder until it answered like the reference, inferring the rule from the reference's behaviour, not from its code, and reported the gap.
The gaps went back to the specification author the same day. The amendment closed all four and corrected part of the inferred rule, and the decoder was changed to follow the written text. One point of the amendment, about rounding, contradicted what the implementer had reported and was wrong for some files; the project's lead agent corrected it after a measurement, from values alone. The extreme case of the capture question is reference value twenty-six:
Reference value twenty-six: White's only legal move is a capture, and it loses. White: king e8; Black: queen e7, pawn g6, king c5, pawn g4.
White's only legal move is a capture, and it loses. No distance has to be read from the file: the position is lost, and the counter resets on the very next move.
The comparisons
The reference was the tablebase prober of python-chess (version 1.11.2, under the GPL, version three or later), used only as a black box through its interface, outside the engine binary. Results, unless noted, are for the final reviewed code.
The reference values were re-run on 2026-10-04, and all fifty-four pass. Fifty-one have at most five pieces: forty-seven match exactly, and four differ from the service by one half-move, the rounding the format allows; there the decoder agrees with the library. The other three have six or seven pieces, and the decoder correctly reports no value for them. Because the specification was read from the same library, a fault shared by the library would not show in the comparisons below; the fifty-four values from the public service are the separate check.
The comparisons against the library:
| Comparison | Positions | Deviations |
|---|---|---|
| Three pieces, WDL and DTZ, complete | 735,736 | none |
| Four pieces, WDL and DTZ, complete | 250,136,842 | none |
| Five pieces, WDL, complete (all 110 tables) | 51,670,153,488 | none |
| Five pieces, DTZ, complete for twelve tables | 3,890,700,824 | none |
| Five pieces, DTZ, the other ninety-eight tables: samples only (three runs over all five-piece tables) | about 170 million | none |
For the five-piece WDL comparison, 83 tables were checked on an earlier state of the code and 27 on the final one. Two of the three sample runs were on the final state; those two include promotions and en passant captures with pins and checks.
"Complete" means every legal position up to the symmetries of the board, with either side to move, without castling or en passant rights, enumerated independently of the decoder's own indexing.
Where the check stopped
The full DTZ comparison was ended on 2026-10-03 by decision. The rest would have been about 48 billion positions, roughly two more weeks at the measured rate on its machine, where the batch work had already disturbed speed measurements. The decision notes that the project's game checks also catch decoder errors. The entry on bishop and knight named full five-piece comparisons as a condition; for the distances, that comparison was not completed. A fault confined to the distances of one of those ninety-eight tables would be found only if a sample hit it.
A real bug
A review of the connection to the search found one. With tables loaded, a search started in a position that was already mate or stalemate read from an empty list. A sanitizer build crashed; the normal build happened to answer correctly. A one-line guard fixed it, and a second review confirmed the fix for six kinds of search command.
Where it stands
The decoder is not on the main branch. Since 2026-10-02 that branch holds only an empty frame for it, which leaves the engine bit for bit unchanged apart from its version number. The decoder and its connection to the search are on a separate branch. Without a table path, that build gave the same output on 256 positions, about 408,000 lines, as the main branch of that day.
The tables are not active in measured games. They could enter them only in a specially declared measurement epoch that is not active yet, and then every opponent gets the same table files. The project's constitution gives the reason: an engine that finds tablebases on the disk by itself gains Elo Strength difference to the opponent, estimated from the games, with a 95-percent half-width where the oracle reported one. Never an absolute rating. that nobody authorised.
What this does not say
It does not say that the engine now plays endgames perfectly, or anything about playing strength. What can be said is narrower: on the positions compared, the project's own decoder reads the tables like an outside reference, completely for win, draw and loss up to five pieces, though partly on an earlier state of the code, and only in part for the distances.