An Elo number needs its opponent

Every number below comes from the public data contract: register.json in the snapshot of 2026-09-05, with the register ID given beside it. The two accepted candidates ran a second stage against a different opponent; where that stage reports no value, none is invented here.

Six attempts in one day, two of them accepted. One of those carries two EloElo Strength difference to the opponent, estimated from the games, with a 95-percent half-width where the oracle reported one. Never an absolute rating. numbers that differ by a factor of two hundred and fifty — and both are correct.

On 31 August 2026 six candidates went through the test gate. Four were rejected, two were accepted. That is an ordinary day, and it would not be worth an entry — except that one of the two accepted candidates carries two Elo numbers at once, and they differ by a factor of about two hundred and fifty.

Both are correct. They answer different questions, and the difference between those questions is the most useful thing on this page.

The same change, measured twice

T-0049 changed how the search reuses transposition bounds during quiescence. It was measured twice, against two different opponents:

Both rows describe the same code change (T-0049, register.json). The short stage asks: is this build better than the one before it? The long stage asks: how far has the engine come since the beginning? The second question has a much larger answer, because it sums up everything since v0 — not because this change did more.

An Elo number without its opponent is therefore not a statement. It is half a statement, and the missing half decides the order of magnitude.

The opponent is not the only thing that differs between those two rows: the long stage also runs at a five times longer time control. Both stages change at once, so neither number isolates the effect of the opponent alone. What the pair does show is that a single change can carry two legitimate Elo figures at different orders of magnitude, and that the figure is meaningless without saying which comparison produced it.

Why the large number does not add up

The frozen reference v0 never changes, which is what makes it useful: gains against a fixed point can be compared over time. But the number measured against it belongs to the whole series, not to the candidate that happened to run that day. Adding up the long-stage numbers of several accepted candidates would count the same distance many times over.

The number that belongs to a single change is the short one — the one against the predecessor. For T-0049 that is +4.79 Elo, with a 95 % half-width of 3.80 (register.json). It is small, and it is the honest one.

What the six attempts actually measured

All six ran the same short stage against their own predecessor, at 8+0.08. Sorted by estimate (T-0048 to T-0053, register.json):

Read the width and games columns together and the mechanism of the test becomes visible. The widest interval belongs to the smallest sample; the narrowest belongs to the largest. Precision is bought with games, and the price rises steeply as the effect gets closer to zero.

The four rejected candidates are as much a result as the two accepted ones. Three of them estimate below zero. The fourth does not.

A rejection that measured positive

T-0053 was rejected with an estimate of +0.38 ± 2.52 Elo over 36,362 games (register.json) — the largest sample of the six, and a point estimate on the positive side of zero.

That is not a contradiction. The sequential test does not ask is the estimate positive? It asks whether the evidence has crossed one of two thresholds. The stage was set up to separate “no gain” from “a gain of 5 nElonElo Normalised Elo: the Elo difference divided by the spread of the game results, so that the SPRT bounds mean the same at different time controls.” (elo0 0.0, elo1 5.0 nElo, α = β = 0.05, log-likelihood bounds at ∓2.94; T-0053, register.json), and this run crossed the lower one. An interval running from roughly −2.1 to +2.9 contains zero comfortably; a change that cannot be distinguished from no change is not accepted, whichever side of zero its midpoint happens to fall on.

Two consequences follow, and the second matters more:

• A point estimate from a sequential test is not an unbiased effect size. The run stopped when a threshold was reached, and that stopping rule biases the estimate. Where such numbers appear here, they are estimates from a stopped test, not measurements of a size.
• Establishing that something does nothing is expensive. T-0053 needed roughly nine times the games of T-0052 to reach its verdict. The candidate with the largest point estimate (T-0052, +17.17 ± 8.88 Elo, register.json) reached its verdict after 3,828. What these two runs show is that difference and nothing more: both are early-stopped observations, and no general rule about sample size follows from a pair of them.

T-0053’s hypothesis, as published in the register, concerned reporting forced mates as signed move counts so that protocol clients read them correctly — explicitly without changing the search tree. The run was rejected, which is consistent with that expectation — it did not establish that the effect is zero, only that the evidence reached the lower bound. That is why the run sits in the register rather than in a changelog.

What is missing, and stays missing

The long stage of T-0052 reports no Elo value at all, and the long stage of T-0049 reports a value without an error half-width (register.json). Neither gap is filled here. A missing error bar is not zero uncertainty, and a missing value is not zero — both mean unknown, and that is what the register says.

For the long stage the gap matters more than it looks. At that distance from v0 the score rate sits close to its ceiling, where a handful of games decides whether a finite Elo value exists at all — and 486 games is a small sample for a quantity that behaves that way. The number is reported because the register reports it, not because it is precise.

What this does not say

It does not say the engine gained 1194.89 Elo on 31 August (T-0049, register.json). That number is the distance from a build frozen at the beginning of the series to the build measured that day, and it is reported once per accepted long stage — not earned once per candidate.

It does not say T-0049 and T-0052 are worth +4.79 and +17.17 Elo permanently. Both are estimates from stopped tests, both carry intervals wide enough to matter, and neither has been re-measured since.

It does not compare this engine to any other. Every number here comes from a candidate playing against another build of itself, which says nothing about how it would do against anyone else.

And it does not say the four rejections were mistakes. Three of them estimate below zero and one just above it; all four crossed the lower bound. The four rejections in this series are informative results, and they are reported here for that reason.