Own data, file by file: where the training data of the 15 networks came from
Listen to this post15 min
Created by AI from the text of this post and read by the synthetic voice “Sulafat”; not a word-for-word version, and it may contain errors.
Loading…
Created by AI from the text of this post and read by the synthetic voice “Sulafat”; not a word-for-word version, and it may contain errors.Position and speed stay in this browser and are sent nowhere.
The recording could not be played. Open the audio file
The entry “15 networks in 40 days: the lineage of the evaluation” listed every network the oracle adopted onto the main branch up to 2026-10-18 and named its gaps, among them teachers that were not documented. At the operator's request, an internal forensic review then checked, read-only, where the training data of all 15 networks came from. The result: all 15 networks were trained on the project's own data. This is documented for every data file and supported by replayed games. 2 tolerance cases under the project's constitution and 6 limits belong to the result; they are named below. The earlier entry keeps its text and now points to this one; this entry, also written at the operator's request, supplements and corrects it.
Who generated the data
Every data file comes from games that an engine of the project played and evaluated against itself. The chart shows, for each training data set, which main-branch network was inside the generating engines, counted in files. All filled cells lie in columns of earlier networks or of the hand-written evaluation that preceded the first network. No data set contains files from an engine with its own or a later network, and none from outside the project. The matrix counts original files by the evaluation source in the generating engine: the hand-written evaluation or one of 12 earlier main networks. The additional replay file of campaign 1 is not shown; for the old set, 96 denotes its source files from the pilot campaign.
| Run | Data set | Data rows | Start weights |
|---|---|---|---|
| T-0072 | pilot campaign, 96 files, 96,000 games | 6,946,527 after deduplication | seed |
| T-0076 | old set (the positions of T-0072) and 9 files of campaign 1 | 20,875,984 | seed |
| T-0085, T-0099, T-0100 | campaign 1: 146 files and 1 replay file | 226,723,955 | seed |
| T-0119 | v2 prefix, 317 files | 493,721,393 | seed |
| T-0120, T-0124 | v4, 1,563 files | 2,501,801,023 | seed |
| T-0127, T-0131 | v4 and Δ5 with 384 files | 3,131,080,541 | seed |
| T-0135 | v4, Δ5 and Δ6 with 1,082 files | 4,934,592,622 | seed |
| T-0144 | v7j, 2,464 files | 4,100,869,336 | seed |
| T-0168 | v8j, 3,348 files | 5,575,059,348 | seed, via an own preliminary run |
| T-0180 | positions from the v8 delta, relabelled | 151,999,515 | T-0168 |
| T-0189 | v10 positions, relabelled | 148,834,647 | intermediate network |
| intermediate network | v10, 8,197 files after the test-set filter | 13,591,787,474 | seed |
Data rows are data set sizes including the validation share. The intermediate network is the project's own network from which T-0189 started; it did not lie on the main branch itself. No main network has a foreign weight ancestor.
What was checked
- Files: all 10,224 original files of the data sets from T-0085 to T-0189, 10,223 shards with 301.6 GB and 1 replay file, are byte-identical with their manifest hashes, as are the 96 files of the pilot campaign. Each file is counted once, even if several networks use it.
- Production records: for every file there is a completed match record naming the engine binary, the generator version, the seed and the game range. There are 501 campaigns; none mixes binaries. All 31 generating engines are own engines: 30 farm binaries and the hand-written evaluation.
- Replays: from each of the 10,224 files one randomly drawn game was replayed with the binary and generator of its record: 10,224 of 10,224 byte-identical, with 1,196,488 data rows from all 30 farm binaries. The same holds for 96 of 96 files of the pilot campaign.
- Sample: the stratified sample fixed in advance under the constitution gave 505 of 505 games byte-identical, with 58,418 data rows in 11 strata. An independent checker with its own seed got 22 of 22. With no mismatch, the one-sided 95 per cent upper bound for the mismatch probability per draw, under the respective sampling scheme, is 5.8 per cent for a stratum of 50 draws and 45 per cent for the replay file with 5 draws.
- Byte-identical means the same rows byte for byte at fixed seed and fixed node count. The replays show that the games can be reproduced with the recorded binary and generator; that they were produced this way at the time is documented by the match records.
- Openings: all generators play 8 random legal half-moves from the starting position, driven by a seed, and discard what their own search sees outside ±300 centipawns. No generator reads an opening book.
- Endgame tablebases: no generator and no labeller had a tablebase option, and the word “syzygy” occurs in none of the 31 generator binaries. Tablebases could not enter any label.
- Lichess: Lichess data were used only in research networks. None of them reached the main branch, none is a start network of a main network, and none generated data for one.
Additions to the earlier entry
- Teachers are now documented for every network. The first generation learned from the hand-written evaluation at 5,000 nodes. The self-play data of all later networks came from the hand-written evaluation or from engines with earlier main networks; the playing engine versions of v7j, v8j and v10 are 8, 13 and 23 binaries. No network learned from self-play of its own engine.
- T-0180 and T-0189 are deliberate fine-tunings by self-distillation: the engine with the respective start network relabelled the positions at 20,000 nodes, for T-0189 the engine with the own intermediate network. Of the labels of T-0189, 50 of 50 were recomputed byte-identically.
- The “about 150 million training rows” of T-0180 are 149,999,520 training rows of 151,999,515.
- The “about 5.56 million training rows” of T-0072 are correct: 5,562,922 of 6,946,527 positions. All 6,946,527 rows of the old set appear verbatim in the 96 files of the pilot campaign; the old set of T-0076 is the same set.
- The open way of counting campaign 1 is resolved: 226,723,955 is the recounted row total of its 147 files, the 181,346,139 training rows are its training share, and the commits for T-0099 and T-0100 called the total “training rows”. The “about 395 million learning rows” of T-0119 are 80 per cent of 493,721,393 data rows.
Before the first main network
The network of T-0072 is generation 3 of a GPU pilot of 2026-09-07, trained from scratch without a warm start on 96,000 self-play games of the hand-written evaluation at 5,000 nodes. Before it came a CPU stage with two CPU networks and further research networks, and generations 1 and 2. None of them lay on the main branch, and none is a weight ancestor of generation 3. The series before T-0072 is method, not inheritance. The integer evaluation core took no weights from them.
Tolerances
- Inventory clause: under the constitution, networks that existed on 2026-10-05 count as cleared after a hash-bound inventory. The inventory of 2026-10-06 covers T-0072 to T-0168. It is needed in 2 places only: for the read path of the streaming runs T-0119 to T-0168, whose file list is bound to the run only through the row total, and for 11,763 and 16,418 rows in v7j and v8j whose positions also appear in test books created later. These rows are own self-play; the question is not their origin but whether measurement stays independent. For the data of T-0072 to T-0135 such overlaps were not counted.
- Tolerance 10⁻⁵: through its start network T-0189 inherits 1 row that also appears in a clock test book created later, 1 of 120,000 book positions, below 10⁻⁵. The row occurs once, the book was created after the filter, and the entire book never measures this line. All conditions of the constitution are met.
- Disclosed, but not a tolerance case: v10 contains 1 position that also appears in a tuning book (1 of 42,000). A decision of 2026-10-15 holds that a tuning book is not a test set in the sense of the constitution; the book is not used. The reviewers disagree on whether it should block. The position itself comes from own self-play.
Limits
- A bit-identical rebuild of the generating farm binaries was not achieved in this review: the farm compiled with GCC 13.3, the review machine has GCC 15.2, and a documented rebuild gave a different hash. The link from binary to source therefore rests on build records: every source file of 29 builds is byte-identical with git. For the binary of campaign 1 there is only the link through the registered source tree. The replays show that the binaries produce the data; whether their code is the project's own is a question about the engine code, not the data.
- Read path of the trainer: shown is what the data sets contain, not byte for byte what the trainer read. The inventory clause covers this.
- The labels of T-0180 were not recomputed in this review. An earlier review documents them with 1,000 rows byte-identical, in a second attempt and under a changed segment rule. The positions were checked: 50 of 50 come from own campaigns.
- Where the numbers in the first hand-written evaluation came from was not part of this review. It is the engine whose search wrote the first labels.
- Later data that belong to no main network were not checked.
- Of the filtered v10 files only a sample was checked: 200 of 8,104. All 200 arose solely by removing rows from the original, with the order kept and the hash as in the manifest; the one-sided 95 per cent upper bound for files not derived this way is about 1.5 per cent.
How this was checked
All primary data were only read. The review followed every file from the file to the manifest, the match record and the binary, and from the binary to the build record, the git commit and the register, for 1 binary through the registered source tree only; it recomputed the hashes of all shards. Each drawn game was replayed with the original generator and the original binary and compared byte for byte. Seeds were fixed before the computation. In the stratified sample, farm files were drawn weighted by rows and pilot files by compressed size; in the second wave one game was drawn from each file. The review contains no positions. Agents of two other model families checked the review: one with tools recomputed its own samples and the full hash, the other without tools checked the statements against the raw counts. Neither found evidence of foreign data; their objections to overly strong wording are incorporated. An agent of another model family checked this text against the review.