30 games, no loss — and a band of ± 170

Listen to this post2 min

Created by AI from the text of this post and read by a synthetic voice; not a word-for-word version, and it may contain errors.

Project measurement, not in the register. The figures come from a game file handed over directly by the operator — not from the oracle's data delivery and not from a registered measurement.

Coherent 0.1.92 played 30 games against its predecessor at 40 moves in 120 seconds plus 1 second per move. The result: 24 wins, 6 draws, no loss — 27 points out of 30, 90 %.

Two things about it are remarkably clean. The points fall exactly evenly across the colours: 12 wins and 3 draws as White, the same as Black. And no game was lost on time; they end by mate (24), repetition (4), insufficient material (1) and the fifty-move rule (1).

What the number is

A score rate of 90 % gives an Elo Strength difference to the opponent, estimated from the games, with a 95-percent half-width where the oracle reported one. Never an absolute rating. difference of +382. The 95 % interval runs from +274 to +614; the half-width is about ± 170.

For comparison: the external anchoring of the neighbouring build 0.1.91 against the same predecessor gave +279 ± 34 out of 720 games — the figures are here. The band here is five times as wide not through carelessness but because 30 games are 30 games. At 90 % there is the further point that a single game carries barely a third of the information of an even game.

What the number is not

The interval is more likely too narrow than too wide. It assumes the 30 games are independent of one another. They are not: the whole tournament contains exactly two opening positions, Giuoco Piano and Queen's Indian, 15 times each. All 30 move sequences do differ — the engines do not play deterministically — but 15 attempts at the same position are not the same thing as 15 different positions.

Colour and opening are mixed together. In the Giuoco Piano 0.1.92 always had Black and in the Queen's Indian always White; there are no colour-swapped pairs on the same position. That both halves come to exactly 90 % argues against an opening carrying the result — but these games do not rule it out, and a setup that does rule it out costs no more than swapping the colours.

It is self-play. The opponent is an older build of our own, not a foreign engine with a published rating. Self-play reliably measures whether a build beats the previous one, and regularly overstates how much of that arrives in the field: for the two nets between 0.1.62 and 0.1.91, self-play saw +359 and +320, and the foreign anchors afterwards +279.

The game file does not name the opponent. It carries it as "Coherent" with no version number; that it is 0.1.62 is in the file name and nowhere else. For a number somebody will quote later, that is not enough — the version belongs in the games, not in the file name.

What stands

30 games without a loss are a good sign and not a measurement. The head of this site still reads 0.1.62 at 2608 ± 56, and what 0.1.92 is worth in the field will be said by an anchoring against foreign opponents, not by this.