Show the speed
Listen to this post6 min
Created by AI from the text of this post and read by a synthetic voice; not a word-for-word version, and it may contain errors.
Loading…
Created by AI from the text of this post and read by a synthetic voice; not a word-for-word version, and it may contain errors.Position and speed stay in this browser and are sent nowhere.
The recording could not be played. Open the audio file
Some changes do not change what the engine plays. The search visits the same positions, chooses the same moves and reports the same numbers. Such a change can only help in one way: by making the engine faster, so that it sees more in the same time. Until late September, these changes went straight to the oracle, the measurement that decides whether a change is adopted. This entry explains why that stopped.
Why the pre-test cannot see them
Before a change reaches the oracle, it normally plays a pre-test against the current version at a fixed number of nodes. For a change that leaves the search identical, every one of those games is the same as before, so the pre-test learns nothing. Under the old rules such a candidate, with a proof that its output was identical, was therefore sent straight on to the oracle.
What it cost
The numbers come from the project's own records, in a window from 2026-09-21 to 2026-09-26:
| Route to the oracle | Passed | Oracle hours per pass |
|---|---|---|
| Sent on by the pre-test with a clear result | six of eight | about one point three |
| Sent on only at the pre-test's cap | one of seven | not given |
| Proof of identical output only | two of eleven | about ten point seven |
Counted over the whole history, the identical candidates had passed seven times out of nineteen. The ones that failed occupied the oracle for almost sixteen hours in that window alone.
One observation decided the matter. All seven identical candidates that had ever passed had a measured speed gain written into their hypothesis. Of the twelve that failed, only one did.
The rule
On 2026-09-26 the project decided: a candidate with a proof of identical output goes to the oracle only with speed evidence. That is either a measured, paired gain in speed, or a label saying that the change only affects time management and can only be seen on a real clock. Without either, it is rejected with "no speed evidence". Changes without any speed claim, such as clean-ups, take other routes without the oracle, provided they are not slower, or are closed without being merged.
Looking back, the rule would not have cost a single pass: all seven earlier passes carried a speed gain. It is one condition in the decision function. A test with a deliberately broken variant had to fail without it, and did. The rule went into the code the same day.
Something else was examined and left alone: the cap at which the pre-test stops without a decision. No threshold separated good cases from bad ones there, the cap cases cost little oracle time, and the stricter options would have lost the only candidate that had passed after hitting the cap.
How speed is measured
The first method timed the same searches with the old and the new version, alternating which one went first, and took the median ratio with a confidence band. The decision said "a gain above zero". The implementation asks for more: the lower edge of the band has to be above zero. Otherwise a change with no speed effect at all would get through on a coin flip.
From 2026-09-30 a second method applies. It uses registered builds and three runs, each with a control run that compares the old version with itself. The pooled control band has to stay within half a percent, and the lower edge of the gain's band has to lie above that. This means that gains smaller than about half a percent cannot be shown at all.
What happened next
From 2026-09-27 to 2026-10-03, thirteen candidates were rejected with "no speed evidence". In the oracle, six identical candidates have been tested since the rule came in, and four of them passed, at about seven oracle hours per pass. The three that came in on a measured speed gain all passed. Of the three that came in with the time-management label, one passed, although all three also carried a small measured gain.
One of the thirteen rejections shows the rule at its strictest. Its measured gain was clear, but its control run ended just under three ten-thousandths of a percentage point above the half-percent limit. It was rejected all the same.
What this does not say
The numbers are small: eleven and nineteen cases before, six after. Other changes to the pipeline came in during the same days, so the effects are mixed. The look back relies on the gains written into the hypotheses, not on a new measurement under today's stricter method. Speed is not measured on the oracle's own machine, and speed evidence is not evidence of strength. Small real gains may now be turned away. Both the cost of that and the time saved will be reviewed on 2026-10-10.